Trouble with watch together / groups

[Nublys]

Hello, I have trouble with the reliability of the groups and would like to get this fixed, as my gf and I will be long-distance for a bit, but want to keep up with our shows. I know my server can handle 2+ streams handily, but when I create a group, I have random buffering, stuttering, and desynchronization due to said issues.
It is a TrueNAS server, transcoding via an Arc A310, a Z2 RAID, and a Cloudflare tunnel. When looking, no resources are pinned, and no errors are in the log.