Choppy transcoding of high bitrate files with Hardware Acceleration enabled

Hi there,

I’m experiencing choppy transcoding of high bitrate files with Hardware Acceleration enabled. The file Jellyfin is transcoding has the following properties:

Size47167 MB
TitleMPEG-H HEVC Video / 64973 kbps / 2160p / 24 fps / 16:9 / Main 10 @ Level 5.1 @ High / 10 bits / HDR10 / BT.2020 - 4K
ProfileMain 10
Aspect ratio16:9
Bitrate68814 kbps
Video rangeHDR
Color transfersmpte2084
Color primariesbt2020
Pixel formatyuv420p10le
Ref frames1
TitleDTS-HD Master Audio / 5.1 / 48 kHz / 4174 kbps / 24-bit - English - Default
ProfileDTS-HD MA
Channels6 ch
Sample rate48000 Hz
Bit depth24 bit

Jellyfin transcodes this to 1080p but at just 16fps. However, the bitrate is 115Mbps, even though I’ve set the streaming bitrate limit to 75Mbps. The transcode log can be found here:
I’ve played around with various settings, such as the FFMpeg presets ultrafast, slow etc. as well as the CRF numbers to no avail.

The client is the Jellyfin webinterface running in FireFox on Windows 10. The server is running Jellyfin version: 10.7.0 on Docker (Operating System: Linux, Architecture: X64) on my Synology DS918+. This NAS is capable of hardware transcoding the following codcs:

I’ve enabled hardware transcoding by mapping the appropriate devices to the Docker container, assigning them the correct permissions, and selecting the appropriate codecs in in Jellyfin itself.

What could this be causing? Is the source file simple too big to handle? I don’t see my CPU suffering, it hovers between 40-70 percent. I would’ve expected to see 100% concurrent CPU usage if it was CPU limited.

Thank you!