Hardware Transcoding Performance Tanking After a Few Minutes (Iris Xe + Jellyfin LXC

[jero537]

Hey all, looking for some guidance.
Setup:

• Host OS: Proxmox 9
  
• Container: LXC running Jellyfin (2 vCPU / 2GB RAM — also tried 4 and 6 vCPU, same issue)
  
• Hardware: Minisforum NPB7 with Intel Iris Xe
  
• Storage: USB-C DAS, 6 HDDs (3+3 mirrored ZFS) holding all media
  

Issue:
When transcoding large Dolby Vision files (50–70GB) through Jellyfin, playback starts smooth and the GPU kicks in fine (CPU stays low).
But after a few minutes, the framerate slowly drops until it’s around 10–15 FPS, making playback unusable.
What I've tried:

• Switched hardware acceleration:
  • ✅ QSV — proper tone-mapping, but stutters after a few mins
    
  • ⚠️ VA-API — tone-mapping issues, same FPS drop
    
• Tested encoding to H.264 and HEVC — same behavior
  
• Increased container CPU/RAM — no change
  

So it seems like hardware acceleration starts off fine and then chokes or throttles.
Question:
Anyone run into this? Any tips on proper Intel GPU passthrough for Jellyfin on LXC? BIOS setting / driver / VA-API/QSV config / throttling issue?
Any direction or known fixes would be super appreciated!

[sosch]

Hi,

I am having similar issues with my setup:

Host OS: Proxmox 9.0.11
Container: Docker Container inside Proxmox VM (Debian 13, latest) running Jellyfin (4 vCPU / 12GB RAM)
Hardware: N150 Mini-PC, 16 GB RAM, SSD
Storage: RAID-1 NAS 1Gbit

Logs and intel_gpu_top proof iGPU passthrough is working fine.
Symptoms are pretty much the same as OP: Startup Jellyfin, play any kind of media and the Transcoding works perfectly fine, for a very short time. I have up to 10x transcoding ratio, but shortly after it drops to <1x and it ultimately results in stuttering playback.

I have tested both, QSV and VAAPI, but my own benchmarks show QSV a little more performant.
I observe high RAM usage, that also not really drops lower, at least according to the Proxmox Monitoring UI.