SOLVED: Jellyfin hw transcoding not working

[canguy247]

Hey Folks,

I am hoping someone can help me with an issue as I am at the end of my knowledge here.

I have a server running NixOS and have a few containers running and all is well except for one thing.

The issue I have is that i want to use gpu acceleration for transcoding in Jellyfin and cannot for the life of me get it working.

Here is a summary of what I have done, with the logs/details below:
-confirmed Jellyfin running well without hardware transcoding
-installed NVidia drivers on host and container and passed devices through
-confirmed drivers working on host and container
-enabled NVENC hardware acceleration in Jellyfin - I get an error when playing media which uses the hardware
-ffmpeg log shows an error wrt no cuda device
-copying and pasting the exact ffmpeg command from the log into the container works fine
-changed the systemd service to run as root with no luck
-reached wit's end and came here

Details of what I have tried:
I have installed the drivers on the host and confirmed them as running with nvidia-smi:
NVIDIA-SMI 565.77 Driver Version: 565.77 CUDA Version: 12.7

I also did same in the container and confirmed them running as above.

The issue is that playing any media which requires gpu transcoding fails…ffmpeg log shows:
https://pastebin.com/V84W4i25


Here is the config for the container for reference:
https://pastebin.com/rWJKej8W

Checked ffmpeg sees the hardware:

Code:

/nix/store/n6jgkqjx03jalh683vxfv7mhl4xnhlc4-jellyfin-ffmpeg-7.0.2-5-bin/bin/ffmpeg -hide_banner -hwaccels
Hardware acceleration methods:
vdpau
cuda
vaapi
qsv
drm
opencl
vulkan

And then successfully used the jellyfin-ffmpeg within the container to do a conversion including using the full command from the ffmpeg log.

I do not know where to go from here...journalctl shows the following so it definitely can run ffmpeg successfully:

Code:

.
jellyfin[320]: [08:57:06] [INF] [1] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Available decoders: ["libdav1d", "av1", "av1_cuvid", "av1_qsv", "h264", "h264_qsv", "h264_cuvid", "hevc", "hevc_qsv", "hevc_cuvid", "mpeg2video", "mpeg2_qsv", "mpeg2_cuvid", "mpeg4", "mpeg4_cuvid", "msmpeg4", "vc1_qsv", "vc1_cuvid", "vp8", "libvpx", "vp8_cuvid", "vp8_qsv", "vp9", "libvpx-vp9", "vp9_cuvid", "vp9_qsv", "aac", "ac3", "ac4", "dca", "flac", "mp3", "truehd"]
.
.

Code:

.
MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Available encoders: ["libsvtav1", "av1_nvenc", "av1_amf", "av1_vaapi", "libx264", "h264_amf", "h264_nvenc", "h264_qsv", "h264_v4l2m2m", "h264_vaapi", "libx265", "hevc_amf", "hevc_nvenc", "hevc_qsv", "hevc_vaapi", "mjpeg_qsv", "mjpeg_vaapi", "aac", "ac3", "alac", "dca", "flac", "libmp3lame", "libopus", "truehd", "libvorbis", "srt"]
.
.

Changed bind mounts to allowedDevices and all is good thanks to TheDreadPirate

Not sure why bind mounts are suggested in most NixOS docs I read or why I could still use hw transcoding from ffmpeg within the container. I am thinking the allowedDevices does something that is needed by the systemd.services sandboxing.

[TheDreadPirate]

Which GPU are you using? And did you install the nvidia encode and decode packages?

[canguy247]

Thanks for the suggestion

Quadro P400

I am running NixOS so no extra packages needed. I also confirmed ffmpeg works with hardware accel works within the container

I have done everything i can think of to deal with systemd sandboxing but that is the only difference between running ffmpeg myself or Jellyfin teying it...

[TheDreadPirate]

I'm not familiar with NixOS, but the only thing that looks odd is that you passed in several, or all, of the Nvidia devices as volumes. In plain docker you pass in devices as devices and not volumes.

[canguy247]

Thank you so much for this. When I first read it I almost ignored it because binding the /dev devices is what evry example I saw said to do

I looked a little deeper thanks to your post and switch it to allowedDevices and it works!

Thanks again eh!

I'm not familiar with NixOS, but the only thing that looks odd is that you passed in several, or all, of the Nvidia devices as volumes.  In plain docker you pass in devices as devices and not volumes.