QSV terribly slow on TrueNAS Scale

[Thorshani]

I am only accessing it through Tailscale, the server is on another continent.

That would probably explain this. This is no longer an issue about network bandwidth, but also of network latency. You're at the mercy of all the connections providing you access to your server when you're that far apart from your server. Your ping to and from the server might be really high. As in, by the time your server gets notified that your transcode needs to progress, you have been waiting on a buffering screen for some time.

Lets take something like youtube as an example. Youtube has multiple server scattered across the world. The most popular videos are cached on each server, while the more obscure videos are stored on fewer servers. When you request a video on youtube, your redirected to the closest server to your area that will handle giving you the data that you requested. This is to avoid latency issues.

I would check to see if you have the "throttle transcodes" option enabled under your dashboard playback settings. If that's on, your server will stop transcoding until it gets signal that a client is reaching the end of the provided buffer.

I don't know what your situation is with being in another continent than your server (which I assume you own), but I would assume physically moving the server to where you're located would lessen this issue. That or making a new server where you're located, then redownloading your media to it over the internet.

Though as I said with my experience using Tailscale, being in the same location as my server and connecting to it through Tailscale with my mobile phone's 4G network, it takes a while to load anything to view through Jellyfin. Sometimes it takes a few seconds, sometimes it takes like a minute to load the video.

I see what you mean. I was hoping my NAS would work as a Jellyfin server no matter where I was in the world. I was already considering creating a second NAS to have at my home, just decided to deploy the remote one first. I did however determine that my pool being encrypted could have a big impact on speed. Unfortunately decided to make the whole pool encrypted rather than individual datasets, so I would need to rebuild the pool and reconfigure everything.

I do not have throttle transcodes enabled.

With your last point, would that mean simply waiting 2 minutes at the start of a 4k movie, or constant buffering for something of that quality?