HW Encoding Proxmox - LXC

[Talle]

Ive been trying to get HW encoding to work but i always end up with this error message when ive been following guides or trying to sort out outher errors.

Running in LXC on proxmox
Proxmox: 8.2.4 (Kernel Version Linux 6.8.8-4-pve)
LXC: Debian 12
Docker image: jellyfin/jellyfin
Jellyfin 10.9.8
Gpu: Intel Arc A380
Cpu: Xeon® Gold 6136 CPU (iGPU disabled)

LXC config

Code:

arch: amd64
cores: 12
features: nesting=1
hostname: Media
memory: 62512
mp0: /dev/sdb1,mp=/storage,backup=0
nameserver: *hidden*
net0: *hidden*
ostype: debian
rootfs: local-lvm:vm-100-disk-0,size=100G
searchdomain: *hidden*
swap: 512
unprivileged: 1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:1 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.cgroup2.devices.allow: c 235:* rwm
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry: /dev/dri/renderD128 dev/renderD128 none bind,optional,create=file
lxc.idmap: u 0 100000 65536
lxc.idmap: g 0 100000 44
lxc.idmap: g 44 44 1
lxc.idmap: g 45 100045 62
lxc.idmap: g 107 104 1
lxc.idmap: g 108 100108 65428

LXC ls -l /dev/dri

Code:

total 0
drwxr-xr-x 2 nobody nogroup      80 Aug  2 22:05 by-path
crw-rw---- 1 nobody video  226,  1 Aug  2 22:05 card1
crw-rw-rw- 1 nobody _ssh    226, 128 Aug  2 22:05 renderD128

Docker compose

Code:

services:
  jellyfin:
    image: jellyfin/jellyfin:latest
    container_name: jellyfin
    group_add:
      - '107'  # This needs to be the group id of your GPU, e.g., `stat -c '%g' /dev/dri/renderD128` on the docker host for iGPU
    environment:
      - TZ=Europe/London
    volumes:
      - ./jellyfin/config:/config
      - ./jellyfin/cache:/cache
      - /storage/filmer:/filmer
      - /storage/serier:/serier
    ports: 
      - 8096:8096
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    restart: unless-stopped

Error when encoding

Code:

{"Protocol":0,"Id":"0761f7c17c3e7a496af2b68ab68929cd","Path":"Story.EAC3.5.1.1080p.Bluray.x265-iVy.mkv","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"mkv","Size":1428245778,"Name":"FFFStory.EAC3.5.1.1080p.Bluray.x265-iVy","IsRemote":false,"ETag":"0503b1b9de4e43a85fa15d7f86eda7c1","RunTimeTicks":29500360000,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"hevc","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":"bt709","ColorTransfer":"bt709","ColorPrimaries":"bt709","DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":1,"VideoRangeType":1,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"1080p HEVC SDR","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":3873161,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":23.976025,"RealFrameRate":23.976025,"Profile":"Main","Type":1,"AspectRatio":"16:9","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":120,"IsAnamorphic":false},{"Codec":"eac3","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":"Standard","LocalizedForced":null,"LocalizedExternal":"Extern","LocalizedHearingImpaired":null,"DisplayTitle":"English - Dolby Digital\u002B - 5.1 - Standard","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"5.1","BitRate":640000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":6,"SampleRate":48000,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"PGSSUB","CodecTag":null,"Language":"eng","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":"Odefinierad","LocalizedDefault":"Standard","LocalizedForced":"Tvingad","LocalizedExternal":"Extern","LocalizedHearingImpaired":"H\u00F6rselskadad","DisplayTitle":"English - PGSSUB","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":2,"AspectRatio":null,"Index":2,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"PGSSUB","CodecTag":null,"Language":"ger","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":"Odefinierad","LocalizedDefault":"Standard","LocalizedForced":"Tvingad","LocalizedExternal":"Extern","LocalizedHearingImpaired":"H\u00F6rselskadad","DisplayTitle":"Ger - PGSSUB","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":1080,"Width":1920,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":2,"AspectRatio":null,"Index":3,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":4513161,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":0,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -i file:"/Story.EAC3.5.1.1080p.Bluray.x265-iVy.mkv" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 hevc_qsv -tag:v:0 hvc1 -preset veryfast -b:v 292000 -maxrate 292000 -bufsize 584000 -profile:v:0 main -level 40 -g:v:0 72 -keyint_min:v:0 72 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=w=426:h=240:format=nv12:extra_hw_frames=24,hwmap=derive_device=qsv,format=qsv" -codec:a:0 libfdk_aac -ac 6 -ab 128000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "830687def38a9f4d0c8c4354d156ecce-1.mp4" -start_number 0 -hls_segment_filename "/cache/transcodes/830687def38a9f4d0c8c4354d156ecce%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/cache/transcodes/830687def38a9f4d0c8c4354d156ecce.m3u8"


ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec    60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter    9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample  4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
Input #0, matroska,webm, from 'file:/Story.EAC3.5.1.1080p.Bluray.x265-iVy.mkv':
  Metadata:
    creation_time  : 2023-11-22T06:11:39.000000Z
    ENCODER        : Lavf59.27.100
  Duration: 00:49:10.04, start: 0.000000, bitrate: 3873 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 272.355000
      Metadata:
        title          : Chapter 1
    Chapter #0:1: start 272.355000, end 1218.718000
      Metadata:
        title          : Chapter 2
    Chapter #0:2: start 1218.718000, end 1800.173000
      Metadata:
        title          : Chapter 3
    Chapter #0:3: start 1800.173000, end 2067.899000
      Metadata:
        title          : Chapter 4
    Chapter #0:4: start 2067.899000, end 2424.631000
      Metadata:
        title          : Chapter 5
    Chapter #0:5: start 2424.631000, end 2915.037000
      Metadata:
        title          : Chapter 6
    Chapter #0:6: start 2915.037000, end 2949.989000
      Metadata:
        title          : Chapter 7
  Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      DURATION        : 00:49:10.036000000
  Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      DURATION        : 00:49:10.016000000
  Stream #0:2(eng): Subtitle: hdmv_pgs_subtitle, 1920x1080
    Metadata:
      DURATION        : 00:49:06.031000000
  Stream #0:3(ger): Subtitle: hdmv_pgs_subtitle, 1920x1080
    Metadata:
      DURATION        : 00:49:06.031000000
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_qsv))
  Stream #0:1 -> #0:1 (eac3 (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[hevc @ 0x614d20d42dc0] Failed to create decode context: 2 (resource allocation failed).
[hevc @ 0x614d20d42dc0] Failed setup for format vaapi: hwaccel initialisation returned error.
Impossible to convert between the formats supported by the filter 'Parsed_setparams_0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
[libfdk_aac @ 0x614d20d95340] 2 frames left in the queue on closing
Conversion failed!

[TheDreadPirate]

Starting with kernel 6.8.5 there is a kernel bug that almost entirely breaks transcoding with Intel Arc GPUs.

You need to roll back the kernel you are running to 6.8.4. Or YOLO and push ahead to 6.9.4+, which has patches that resolve the kernel bug for Intel Arc transcoding.

[Talle]

Thank you now i feel stupid but it works now, I pinned my kernal to 6.8.4-2-pve using this script and rebooted then it all workt.

Code:

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/kernel-pin.sh)"