SOLVED: Proxmox LXC: iGPU Transcoding issue

[Vinny956]

Update: I finally managed to get it working even in a non-privileged container. The issue was an incorrect UID/GID mapping. Here is what I run and what I added to make it work in case someone faces a similar issue:

Gathering the necessary information for the mapping
ls -l /dev/dri on Proxmox node

Code:

crw-rw---- 1 root video  226,  0 Oct 28 17:36 card0
crw-rw---- 1 root render 226, 128 Oct 28 17:36 renderD128

cat /etc/group on Proxmox node

Code:

video:x:44:
render:x:993:

cat /etc/group on LXC container

Code:

video:x:44:
render:x:104:

Adding the mapping to LXC-configuration

I added the following mapping to /etc/pve/lxc/110.conf (adjust the name according to the number of your container):

Code:

lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/card0 dev/dri/card0 none bind,optional,create=file
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.idmap: u 0 100000 44
lxc.idmap: g 0 100000 44
lxc.idmap: g 44 44 1 # mapping GID 44 in LXC to GID 44 on Proxmox node
lxc.idmap: u 44 100044 1
lxc.idmap: u 45 100045 59
lxc.idmap: g 45 100045 59
lxc.idmap: g 104 993 1 # mapping GID 104 in LXC to GID 993 on Proxmox node
lxc.idmap: u 104 100104 1
lxc.idmap: u 105 100105 900
lxc.idmap: g 105 100105 900
lxc.idmap: u 1005 1005 1 # here I was mapping a storage mount as per [url=https://pve.proxmox.com/wiki/Unprivileged_LXC_containers#Using_local_directory_bind_mount_points]Proxmox wiki[/url]. You can skip it if you just want to map do the transcoding.
lxc.idmap: g 1005 1005 1
lxc.idmap: u 1006 101006 64530
lxc.idmap: g 1006 101006 64530

Now, you need to allow the root user to use the render and video UIDs and GIDs we just mapped. Open /etc/subuid and /etc/subgid on your Proxmox node and add the respective group number (in my case - 44 and 993), so it looks like the following:

Code:

root:100000:65536 #will be there by default
root:1005:1 #additional mapping for the storage mount
root:44:1 #Proxmox node video GID
root:993:1 #Proxmox node render GID

Restart the LXC and the transcoding should work.

Docker setup

In case you want to run Jellyfin in a docker container, you will need to pass the LXC render GID (in my case 104 ) and the /dev/dri/renderD128 device to the container. After adjustments, my jellyfin docker-compose file looked as following:

Code:

services:
  jellyfin:
    image: jellyfin/jellyfin
    container_name: jellyfin
    group_add:
      - "104" #my LXC render GID we got earlier from running cat /etc/group inside the LXC
    environment:
      - TZ=Europe/Vienna
    ports:
      - 8096:8096/tcp
      - 7359:7359/udp
    volumes:
      - /docker/jellyfin/config:/config
      - /docker/jellyfin/cache:/cache
      - type: bind
        source: /mnt
        target: /media
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128 #passing the render-device from LXC to docker-container
    restart: unless-stopped
networks: {}

After that, transcoding inside docker-container worked without any issues.

I hope it helps! 

----- Original Post (solved) -----

Hi everyone! Unfortunately I cannot get hardware transcoding to work with my Proxmox setup.  Currently, I have created a privileged LXC container and passed the iGPU via the Web-GUI->Resources:

Code:

/dev/dri/renderD128,gid=992,mode=0666

Then, in docker compose, I followed the "Configure With Linux Virtualization" chapter of the official documentation and created the following docker-compose file:

Code:

services:
  jellyfin:
    image: jellyfin/jellyfin
    container_name: jellyfin
    user: 1000:1000
    group_add:
      - "992"
    ports:
      - 8096:8096/tcp
      - 7359:7359/udp
    volumes:
      - /docker-compose/jellyfin/config:/config
      - /docker-compose/jellyfin/cache:/cache
      - /media:/media
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    restart: unless-stopped
networks: {}

Then, I checked the codecs (docker exec -it jellyfin /usr/lib/jellyfin-ffmpeg/vainfo):

Code:

Trying display: drm
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.22 (libva 2.22.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 25.3.4 (192fe0f)
vainfo: Supported profile and entrypoints
      VAProfileNone                  : VAEntrypointVideoProc
      VAProfileNone                  : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main              : VAEntrypointVLD
      VAProfileH264Main              : VAEntrypointEncSliceLP
      VAProfileH264High              : VAEntrypointVLD
      VAProfileH264High              : VAEntrypointEncSliceLP
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileJPEGBaseline          : VAEntrypointVLD
      VAProfileJPEGBaseline          : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileHEVCMain              : VAEntrypointVLD
      VAProfileHEVCMain              : VAEntrypointEncSliceLP
      VAProfileHEVCMain10            : VAEntrypointVLD
      VAProfileHEVCMain10            : VAEntrypointEncSliceLP
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile1            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileVP9Profile3            : VAEntrypointVLD
      VAProfileHEVCMain422_10        : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10        : VAEntrypointVLD
      VAProfileHEVCMain444_10        : VAEntrypointEncSliceLP

and checked the OpenCL runtime status (docker exec -it jellyfin /usr/lib/jellyfin-ffmpeg/ffmpeg -v verbose -init_hw_device vaapi=va -init_hw_device opencl@va):

Code:

ffmpeg version 7.1.2-Jellyfin Copyright (c) 2000-2025 the FFmpeg developers
  built with gcc 14 (Debian 14.2.0-19)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59. 39.100 / 59. 39.100
  libavcodec    61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample  5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
[AVHWDeviceContext @ 0x7839e4480bc0] Trying to use DRM render node for device 0.
[AVHWDeviceContext @ 0x7839e4480bc0] libva: VA-API version 1.22.0
[AVHWDeviceContext @ 0x7839e4480bc0] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x7839e4480bc0] libva: Found init function __vaDriverInit_1_22
[AVHWDeviceContext @ 0x7839e4480bc0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x7839e4480bc0] Initialised VAAPI connection: version 1.22
[AVHWDeviceContext @ 0x7839e4480bc0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 25.3.4 (192fe0f).
[AVHWDeviceContext @ 0x7839e4480bc0] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x7839e44eb480] 0.0: Intel(R) OpenCL Graphics / Intel(R) UHD Graphics
[AVHWDeviceContext @ 0x7839e44eb480] Intel QSV to OpenCL mapping function found (clCreateFromVA_APIMediaSurfaceINTEL).
[AVHWDeviceContext @ 0x7839e44eb480] Intel QSV in OpenCL acquire function found (clEnqueueAcquireVA_APIMediaSurfacesINTEL).
[AVHWDeviceContext @ 0x7839e44eb480] Intel QSV in OpenCL release function found (clEnqueueReleaseVA_APIMediaSurfacesINTEL).

So far, everything appears normal. However, after enabling either the QVC or VAAPI breaks everything and leads a fatal playback error:

Code:

[2025-10-28 12:29:57.511 +00:00] [INF] [30] Jellyfin.Api.Helpers.MediaInfoHelper: User policy for "jellyfin". EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
[2025-10-28 12:29:57.525 +00:00] [INF] [24] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: Deleting partial stream file(s) "/cache/transcodes/88de280cd9c475e464967fd35537388f.m3u8"
[2025-10-28 12:29:57.679 +00:00] [INF] [24] Jellyfin.Api.Controllers.DynamicHlsController: Current HLS implementation doesn't support non-keyframe breaks but one is requested, ignoring that request
[2025-10-28 12:29:57.680 +00:00] [INF] [24] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -probesize 1G -f matroska -init_hw_device vaapi=va:/dev/dri/renderD128,driver=iHD -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv -noautorotate -c:v hevc_qsv -i file:\"/media/movies/Die.Rosenschlacht.2025.GERMAN.DL.2160p.HDR.DV.WEB.H265-MGE/die.rosenschlacht.2025.german.dl.2160p.hdr.dv.web.h265-mge.mkv\" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_qsv -preset veryfast -mbbrc 1 -b:v 35170103 -maxrate 35170104 -rc_init_occupancy 70340206 -bufsize 140680412 -profile:v:0 high -level 51 -g:v:0 72 -keyint_min:v:0 72 -vf \"setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,vpp_qsv=format=nv12\" -codec:a:0 libfdk_aac -ac 2 -ab 256000 -af \"volume=2\" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename \"c124e9bb8f7625e8e420f79b03cc7d40-1.mp4\" -start_number 0 -hls_segment_filename \"/cache/transcodes/c124e9bb8f7625e8e420f79b03cc7d40%d.mp4\" -hls_playlist_type vod -hls_list_size 0 -hls_segment_options movflags=+frag_discont -y \"/cache/transcodes/c124e9bb8f7625e8e420f79b03cc7d40.m3u8\""
[2025-10-28 12:29:58.260 +00:00] [ERR] [30] MediaBrowser.MediaEncoding.Transcoding.TranscodeManager: FFmpeg exited with code 218
[2025-10-28 12:29:58.303 +00:00] [ERR] [30] Jellyfin.Api.Middleware.ExceptionMiddleware: Error processing request. URL "GET" "/videos/75dc82f2-e871-baee-869c-b4ca3c0aecbc/hls1/main/-1.mp4".
MediaBrowser.Common.FfmpegException: FFmpeg exited with code 218

I would really appreciate any help!