SOLVED: Not possible to play media when NVIDIA NVENC

[MutzelBruzzel]

Hi folks,

I just got my new 'mini PC (intel i5-10300H / NVIDIA RTX 2060-mobile)' for jellyfin, but unfortunately it's not working as intended:

- Using only the CPU results in lags and 100% usage -> Ok nvm
- Using the NVIDIA NVENC (with H264, VC1, HEVC 10bit and VP9 10bit) results in an error message.

Trancsoder-log:

[h264_nvenc @ 0x5a5afe243b40] Cannot load libnvidia-encode.so.1
[h264_nvenc @ 0x5a5afe243b40] The minimum required Nvidia driver for nvenc is 520.56.06 or newer
[vost#0:0/h264_nvenc @ 0x5a5afe47b380] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x5a5afe3c7100] Error sending frames to consumers: Operation not permitted
[vf#0:0 @ 0x5a5afe3c7100] Task finished with error code: -1 (Operation not permitted)
[vf#0:0 @ 0x5a5afe3c7100] Terminating thread with return code -1 (Operation not permitted)
[hevc @ 0x5a5afe423a40] Skipping NAL unit 63
    Last message repeated 1 times
[hevc @ 0x5a5afe410bc0] Skipping NAL unit 63
    Last message repeated 1 times
[hevc @ 0x5a5afe914880] Skipping NAL unit 63
    Last message repeated 1 times
[hevc @ 0x5a5afe332200] Skipping NAL unit 63
    Last message repeated 1 times
[vost#0:0/h264_nvenc @ 0x5a5afe47b380] Could not open encoder before EOF
[vost#0:0/h264_nvenc @ 0x5a5afe47b380] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_nvenc @ 0x5a5afe47b380] Terminating thread with return code -22 (Invalid argument)
[out#0/hls @ 0x5a5afe34ce40] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=      0KiB time=N/A bitrate=N/A speed=N/A   
Conversion failed!

'nvidia-smi' says:

NVIDIA-SMI 550.127.08            Driver Version: 550.127.08    CUDA Version: 12.4 

I am using the jellyfin docker and my 'docker-compose.yaml' looks like this:

                                   
services:
  jellyfin:
    image: lscr.io/linuxserver/jellyfin:latest
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Germany/Berlin
    volumes:
      - jellyfin-configconfig
      - /mediamedia-lokal
    ports:
      - 8096:8096
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities:
                - gpu
                - utility
                - compute
volumes:
  jellyfin-config:

What did I do wrong?

[TheDreadPirate]

Did you also install the nvidia decode and encode libraries? And nvidia container toolkit.

[MutzelBruzzel]

Did you also install the nvidia decode and encode libraries?  And nvidia container toolkit.

Sure:

root@srv-mediaopt/jellyfin# dpkg -l | grep libnvidia
ii  libnvidia-cfg1-550-server:amd64      550.127.08-0ubuntu0.24.04.1            amd64        NVIDIA binary OpenGL/GLX configuration library
ii  libnvidia-common-550-server          550.127.08-0ubuntu0.24.04.1            all          Shared files used by the NVIDIA libraries
rc  libnvidia-compute-535-server:amd64    535.216.03-0ubuntu0.24.04.1            amd64        NVIDIA libcompute package
ii  libnvidia-compute-550-server:amd64    550.127.08-0ubuntu0.24.04.1            amd64        NVIDIA libcompute package
ii  libnvidia-container-tools            1.17.4-1                                amd64        NVIDIA container runtime library (command-line tools)
ii  libnvidia-container1:amd64            1.17.4-1                                amd64        NVIDIA container runtime library
ii  libnvidia-decode-550-server:amd64    550.127.08-0ubuntu0.24.04.1            amd64        NVIDIA Video Decoding runtime libraries
ii  libnvidia-egl-wayland1:amd64          1:1.1.13-1build1                        amd64        Wayland EGL External Platform library -- shared library
ii  libnvidia-encode-550-server:amd64    550.127.08-0ubuntu0.24.04.1            amd64        NVENC Video Encoding runtime library
ii  libnvidia-extra-550-server:amd64      550.127.08-0ubuntu0.24.04.1            amd64        Extra libraries for the NVIDIA Server Driver
ii  libnvidia-fbc1-550-server:amd64      550.127.08-0ubuntu0.24.04.1            amd64        NVIDIA OpenGL-based Framebuffer Capture runtime library
ii  libnvidia-gl-550-server:amd64        550.127.08-0ubuntu0.24.04.1            amd64        NVIDIA OpenGL/GLX/EGL/GLES GLVND libraries and Vulkan ICD

root@srv-mediaopt/jellyfin# dpkg -l | grep nvidia-container
ii  libnvidia-container-tools            1.17.4-1                                amd64        NVIDIA container runtime library (command-line tools)
ii  libnvidia-container1:amd64            1.17.4-1                                amd64        NVIDIA container runtime library
ii  nvidia-container-toolkit              1.17.4-1                                amd64        NVIDIA Container toolkit
ii  nvidia-container-toolkit-base        1.17.4-1                                amd64        NVIDIA Container Toolkit Base

[TheDreadPirate]

Missed this the first time. Add this to your docker compose.

Code:

runtime: nvidia

https://jellyfin.org/docs/general/admini...ualization

[MutzelBruzzel]

First of all - Thanks for your help

Did:

1. Docker-compose down
2. Added the missing line
3. Docker-compose up -d

Still the same error:

[h264_nvenc @ 0x5c114b2f9b40] Cannot load libnvidia-encode.so.1
[h264_nvenc @ 0x5c114b2f9b40] The minimum required Nvidia driver for nvenc is 520.56.06 or newer
[vost#0:0/h264_nvenc @ 0x5c114b531380] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0x5c114b47d100] Error sending frames to consumers: Operation not permitted
[vf#0:0 @ 0x5c114b47d100] Task finished with error code: -1 (Operation not permitted)
[vf#0:0 @ 0x5c114b47d100] Terminating thread with return code -1 (Operation not permitted)
[hevc @ 0x5c114b4d9a40] Skipping NAL unit 63
Last message repeated 1 times
[hevc @ 0x5c114b4c6bc0] Skipping NAL unit 63
Last message repeated 1 times
[hevc @ 0x5c114b9ca880] Skipping NAL unit 63
Last message repeated 1 times
[hevc @ 0x5c114b3e8200] Skipping NAL unit 63
Last message repeated 1 times
[vost#0:0/h264_nvenc @ 0x5c114b531380] Could not open encoder before EOF
[vost#0:0/h264_nvenc @ 0x5c114b531380] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_nvenc @ 0x5c114b531380] Terminating thread with return code -22 (Invalid argument)
[out#0/hls @ 0x5c114b402e40] Nothing was written into output file, because at least one of its streams received no packets.
frame= 0 fps=0.0 q=0.0 Lsize= 0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!

Important part is '[h264_nvenc @ 0x5c114b2f9b40] Cannot load libnvidia-encode.so.1' or?

Any other idea?

[TheDreadPirate]

What GPU do you have and are they any additional layers of virtualization? Is this docker in a LXC?

[MutzelBruzzel]

What GPU do you have and are they any additional layers of virtualization?  Is this docker in a LXC?

- nvidia RTX 2060 mobile
- Just a normal docker on barebone

[TheDreadPirate]

What distro is this installed on? If plain Debian, did you install the non-free-firmware package? And do you have the linux-headers package installed?

[MutzelBruzzel]

Installed on ubuntu server 24.04.2 LTS and for nvidia I used the normal 550 (without open).
Didn't install anything else.

[TheDreadPirate]

In the guide I linked, did you also complete steps 4, 5, and 6?

Code:

Add your username to the video group:

sudo usermod -aG video $USER

Update dynamic links and restart the Docker service:

docker exec -it jellyfin ldconfig
sudo systemctl restart docker

Check the NVIDIA GPU's status by using nvidia-smi:

docker exec -it jellyfin nvidia-smi