SOLVED: Nvidia not being used for transcoding

[mati123321]

I've been trying to set up hardware acceleration for my jellyfin, with no luck. Below there are specs of the current system that appears to be working the most.
I'm not pasting the jellyfin codecs because I tried disabling all except supported ones, all except h264, enabling all, etc etc...

I've been following these 2 tutorials with their images:
https://jellyfin.org/docs/general/admini...ion/nvidia
https://github.com/linuxserver/docker-jellyfin

Here's my compose file (.env just sets puid and pgid to 1000 and some paths)
yml
    jellyfin:
        image: jellyfin/jellyfin:latest
        container_name: jellyfin
        restart: unless-stopped

        env_file:
            - .env
        runtime: nvidia
        environment:
            - DOCKER_MODS=ghcr.io/themepark-dev/theme.park:jellyfin
            - TP_THEME=${TP_THEME}
            - NVIDIA_VISIBLE_DEVICES=all
        volumes:
            - ${CONFIG_DIR}/jellyfinconfig
            - ${MEDIA_DIR}/mediastackdata/media

        devices:
            - /dev/dridev/dri

        deploy:
            resources:
                reservations:
                    devices:
                        - driver: nvidia
                          count: all
                          capabilities: [gpu]


Here I verified that GPU is passed as ffmpeg parameter:
sh
$ ps -Flww -p 340907
F S UID          PID    PPID  C PRI  NI ADDR SZ WCHAN    RSS PSR STIME TTY          TIME CMD
0 S root      340907  332122 98  80  0 - 2553902 -    204516  0 18:54 ?        00:05:07 /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -ss 00:07:03.000 -f matroska -init_hw_device cuda=cu:0 -filter_hw_device cu -hwaccel cuda -hwaccel_output_format cuda -noautorotate -hwaccel_flags +unsafe_output -threads 1 -i filedata/media/movies/Despicable Me 3 (2017)/Despicable Me 3 (2017) 1080p BrRip x264 -VPPV.mkv -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_nvenc -preset p1 -b:v 336000 -maxrate 336000 -bufsize 672000 -profile:v:0 high -g:v:0 72 -keyint_min:v:0 72 -vf setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_cuda=w=960:h=402:format=yuv420p -codec:a:0 libfdk_aac -ac 2 -ab 256000 -af volume=2 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 141 -hls_segment_filename /cache/transcodes/ee1aad0dcfad553c0820bfe18ecac5ac%d.ts -hls_playlist_type vod -hls_list_size 0 -y /cache/transcodes/ee1aad0dcfad553c0820bfe18ecac5ac.m3u8


Here are some screenshots of the system. Surprisingly, ffmpeg is visible in nvidia-smi only on host, not in docker. GPU will draw around 30W once video is loaded, but it shows 0% usage, and cpu is almost 100% all the time

[TheDreadPirate]

What are the full server specs? What CPU and GPU? What distro is the host running? Besides docker, are there additional layers of virtualization?

Can you share one of the ffmpeg logs generated? Those logs are located in the same directory as Jellyfin's logs.

[mati123321]

What are the full server specs?  What CPU and GPU?  What distro is the host running?  Besides docker, are there additional layers of virtualization?

Can you share one of the ffmpeg logs generated?  Those logs are located in the same directory as Jellyfin's logs.

Everything is in the attached images, but I guess they don't load, it works when you right-click and open in new tab.

Basically old laptop with external gtx1060, running on home ubuntu 24. CPU maybe 10% below recommended specs for hwaccel. I think there's no further virtualizaion rather than docker itself. Nvidia properly showing up in ls /dev/dri in both host and container. Everything streaming over cloudflare zero-trust (but I think that's not relevant as there's no sign on that being a bottleneck)

[mati123321]

and now I think the GPU is actually used despite it not being visible in ncivia-smi (drawing twice as much at 4k video transcoding), but still CPU is a bottleneck to a point that I can't watch 2 1080p streams at once. I think that because of that cpu being just about right, maybe something is still wrong with ffmpeg.

[TheDreadPirate]

Had to download them on my cloud VM. My work PC was blocking them. Your CPU usage is coming from the audio transcoding, which will always be on the CPU. Keep in mind that your CPU is still in a low power state even though it is reporting "91.5%" loading. Meaning that the workload is so light that it doesn't need to boost to 2.4Ghz.

---

Your CPU should be able to handle 2 audio transcodes, no problem. I'd need to see your ffmpeg logs to figure out what it is actually doing.

[mati123321]

ok, how can I access them?

[TheDreadPirate]

Go to where you mounted /config. The ffmpeg and jellyfin logs are in /config/log and the file names start with "FFmpeg.Transcode".

[mati123321]

https://paste.mateusz.zip/upload/emu-emu-squid

[TheDreadPirate]

In addition to audio transcoding, it is also burning in your subtitles. Subtitle burn-in also only happens on the CPU.

GPU decodes frame --> CPU composes and renders subtitle on frame --> GPU encodes frame

But the GPU is being used for both the decode and encoding of the video.

Decoding on GPU

Code:

-hwaccel cuda

Encoding on the GPU, "h264_nvenc", with subtitle burn-in.

Code:

Stream mapping:
  Stream #0:0 (hevc) -> setparams:default (graph 0)
  Stream #0:2 (pgssub) -> scale:default (graph 0)
  overlay_cuda:default (graph 0) -> Stream #0:0 (h264_nvenc)
  Stream #0:1 -> #0:1 (aac (native) -> aac (libfdk_aac))

[mati123321]

Thanks for the help! Is there anything that I can do with audio with this GPU architecture? I've disabled baking subtitles and cpu usage dropped a bit. WIth that cpu clocks, I think it's just thermal throttling