5 hours ago
Hello,
I'm trying without success to enable hardware acceleration on my jellyfin media server, enabling it in jellyfin causes the player to crash. Here's my config :
- Jellyfin build 10.11.1
- Running as docker-compose image within a debian 13 VM on a proxmox server.
- Intel i-5 6600 wich has an Intel® HD Graphics 530 integrated GPU
I managed to succesfully pass the gpu to the debian VM and then to the container running jellyfin. Drivers are installed on the debian host.
When I try to uses ffmpeg manually it works and I see gpu activity with intel_gpu_top.
When I run the following command within the container : /usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi \
-i /filmsetseries/complete/Oceans.Eleven.2001.1080p.mp4 \
-c:v h264_vaapi -vf 'scale_vaapi=w=1280:h=720' -f null -
I can see gpu activity and there the file is actually being transcoded within the container.
Trying to fix the issue with chat gpt, it identified two potential issues causing this crash. It's related to jellyfin use of ffmeg, especially two arguments : "scale_vaapi=w=1280:h=532:format=nv12:extra_hw_frames=24"
(I highlighted it in orange in the log snippet bellow).
It would be either the abnormal scaling (1280x532) or the number of frames requested (24). But there is no option to set the scaling manually or request less frames (if that's really the issue). Chat GPT advises me to write a custom script that would bypass jellyfin default utilization of ffmpeg but I'm not sure that would work.
Here is a relevant snippet of the log, showing the ffmpeg crash.
MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -f mov,mp4,m4a,3gp,3g2,mj2 -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:\"/filmsetseries/complete/Oceans.Eleven.2001.1080p.BluRay.H264.AAC-RARBG/Oceans.Eleven.2001.1080p.BluRay.H264.AAC-RARBG.mp4\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_vaapi -rc_mode VBR -b:v 1276002 -maxrate 1276002 -bufsize 2552004 -profile:v:0 high -sei -a53_cc -force_key_frames:0 \"expr:gte(t,n_forced*3)\" -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=w=1280:h=532:format=nv12:extra_hw_frames=24\" -codec:a:0 libfdk_aac -ac 2 -vbr:a 5 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"83075ea8c1532ba12964b1294f81aa0c-1.mp4\" -start_number 0 -hls_segment_filename \"/config/custom-ffmpeg/83075ea8c1532ba12964b1294f81aa0c%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -hls_segment_options movflags=+frag_discont -y \"/config/custom-ffmpeg/83075ea8c1532ba12964b1294f81aa0c.m3u8\""
[2025-10-29 22:54:44.702 +01:00] [ERR] [15] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: FFmpeg exited with code 187
[2025-10-29 22:54:44.762 +01:00] [ERR] [15] Jellyfin.Api.Middleware.ExceptionMiddleware: Error processing request. URL "GET" "/videos/284d3d44-c859-a5e0-4e55-ed9a1f953b13/hls1/main/-1.mp4".
I tried to use a custom path for the transcode files served to clients. Where the file would require that vaapi scaling to be a specific resolution (720p or 1080p) but that did not work.
Using a different device cause similar crash.
Any help on the matter would be much appreciated.
I'm trying without success to enable hardware acceleration on my jellyfin media server, enabling it in jellyfin causes the player to crash. Here's my config :
- Jellyfin build 10.11.1
- Running as docker-compose image within a debian 13 VM on a proxmox server.
- Intel i-5 6600 wich has an Intel® HD Graphics 530 integrated GPU
I managed to succesfully pass the gpu to the debian VM and then to the container running jellyfin. Drivers are installed on the debian host.
When I try to uses ffmpeg manually it works and I see gpu activity with intel_gpu_top.
When I run the following command within the container : /usr/lib/jellyfin-ffmpeg/ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi \
-i /filmsetseries/complete/Oceans.Eleven.2001.1080p.mp4 \
-c:v h264_vaapi -vf 'scale_vaapi=w=1280:h=720' -f null -
I can see gpu activity and there the file is actually being transcoded within the container.
Trying to fix the issue with chat gpt, it identified two potential issues causing this crash. It's related to jellyfin use of ffmeg, especially two arguments : "scale_vaapi=w=1280:h=532:format=nv12:extra_hw_frames=24"
(I highlighted it in orange in the log snippet bellow).
It would be either the abnormal scaling (1280x532) or the number of frames requested (24). But there is no option to set the scaling manually or request less frames (if that's really the issue). Chat GPT advises me to write a custom script that would bypass jellyfin default utilization of ffmpeg but I'm not sure that would work.
Here is a relevant snippet of the log, showing the ffmpeg crash.
MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -f mov,mp4,m4a,3gp,3g2,mj2 -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:\"/filmsetseries/complete/Oceans.Eleven.2001.1080p.BluRay.H264.AAC-RARBG/Oceans.Eleven.2001.1080p.BluRay.H264.AAC-RARBG.mp4\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_vaapi -rc_mode VBR -b:v 1276002 -maxrate 1276002 -bufsize 2552004 -profile:v:0 high -sei -a53_cc -force_key_frames:0 \"expr:gte(t,n_forced*3)\" -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=w=1280:h=532:format=nv12:extra_hw_frames=24\" -codec:a:0 libfdk_aac -ac 2 -vbr:a 5 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"83075ea8c1532ba12964b1294f81aa0c-1.mp4\" -start_number 0 -hls_segment_filename \"/config/custom-ffmpeg/83075ea8c1532ba12964b1294f81aa0c%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -hls_segment_options movflags=+frag_discont -y \"/config/custom-ffmpeg/83075ea8c1532ba12964b1294f81aa0c.m3u8\""
[2025-10-29 22:54:44.702 +01:00] [ERR] [15] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: FFmpeg exited with code 187
[2025-10-29 22:54:44.762 +01:00] [ERR] [15] Jellyfin.Api.Middleware.ExceptionMiddleware: Error processing request. URL "GET" "/videos/284d3d44-c859-a5e0-4e55-ed9a1f953b13/hls1/main/-1.mp4".
I tried to use a custom path for the transcode files served to clients. Where the file would require that vaapi scaling to be a specific resolution (720p or 1080p) but that did not work.
Using a different device cause similar crash.
Any help on the matter would be much appreciated.

