Ahead-of-time x265 transcode looks much worse than Jellyfin QSV transcode

[StaticUnit97]

I think part of this depends on the route you want to go: Handbrake or ffmpeg? HEVC or AV1? Hardware-accelerated or not? What sort of hardware are you running these encodes on?

Thanks for all the good info! Some replies to your final questions:

1. No preference on Handbrake vs. ffmpeg. I'm very comfortable with CLI tools, I just have only used handbrake up to this point.
   
2. Again, no preference. I'm interested in AV1 from a quality per bitrate standpoint, so I'd definitely be interested in that script you mentioned.
   
3. I would imagine not HW accelerated? The goal of this is to maximize quality for a given bitrate, and it seems like software gives the best. The server is obviously using hardware acceleration when doing the real-time transcoding.
   
4. I'm running this on a 5700X3D, so 8C/16T, on Windows. I'd probably use ffmpeg in WSL. The Jellyfin server has a i5-9500T, which is 6C/6T.
   

Thanks!