Transcoding fails in Unraid Docker with AMD GPU + VAAPI

[spicer]

Hello,
I just can't get the hw accelerated transcoding to work.

Code:

# /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128
Trying display: drm
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/r600_drv_video.so
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so
libva info: Trying to open /usr/lib/dri/r600_drv_video.so
libva info: Trying to open /usr/local/lib/dri/r600_drv_video.so
libva info: va_openDriver() returns -1
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_20
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.20 (libva 2.20.0)
vainfo: Driver version: Mesa Gallium driver 23.2.1 for PITCAIRN (, LLVM 13.0.1, DRM 2.50, 6.1.64-Unraid)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264Main              : VAEntrypointVLD
      VAProfileH264High              : VAEntrypointVLD
      VAProfileNone                  : VAEntrypointVideoProc

Jf transcoding log:

Code:

/videos/2d07aece-e2f4-e01d-c6e5-ab2f33ff1e05/hls1/main/0.ts

{"Protocol":0,"Id":"2d07aecee2f4e01dc6e5ab2f33ff1e05","Path":"/data/Filme/A/Aguirre, der Zorn Gottes (1972).mkv","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"mkv,webm","Size":9155945472,"Name":"Aguirre, der Zorn Gottes (1972)","IsRemote":false,"ETag":"a6fc313f97890750debcac7c23ee1933","RunTimeTicks":56740851712,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"h264","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"Aguirre, der Zorn Gottes (1972)","VideoRange":"SDR","VideoRangeType":"SDR","VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"Aguirre, der Zorn Gottes (1972) - 1080p - H264 - SDR","NalLengthSize":"4","IsInterlaced":false,"IsAVC":true,"ChannelLayout":null,"BitRate":12909141,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"Height":1080,"Width":1440,"AverageFrameRate":23.976025,"RealFrameRate":23.976025,"Profile":"High","Type":1,"AspectRatio":"4:3","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p","Level":41,"IsAnamorphic":null},{"Codec":"dts","CodecTag":null,"Language":"ger","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"DTS","VideoRange":null,"VideoRangeType":null,"VideoDoViTitle":null,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"DisplayTitle":"DTS - Ger - 5.1 - Default","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":"5.1","BitRate":1536000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":6,"SampleRate":48000,"IsDefault":true,"IsForced":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":"DTS","Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":14445141,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":null,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -i file:"/data/Filme/A/Aguirre, der Zorn Gottes (1972).mkv" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_vaapi -rc_mode VBR -b:v 7360000 -maxrate 7360000 -bufsize 14720000 -profile:v:0 high -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=nv12:extra_hw_frames=24" -codec:a:0 libfdk_aac -ac 6 -ab 640000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/config/transcodes/04f112a4a73b9c99bd8f6ec9b2b5bc9d%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/config/transcodes/04f112a4a73b9c99bd8f6ec9b2b5bc9d.m3u8"


ffmpeg version 5.1.4-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      57. 28.100 / 57. 28.100
  libavcodec    59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter    8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample  4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from 'file:/data/Filme/A/Aguirre, der Zorn Gottes (1972).mkv':
  Metadata:
    title          : Aguirre, der Zorn Gottes (1972)
    encoder        : libebml v1.2.3 + libmatroska v1.3.0
    creation_time  : 2012-09-02T04:52:19.000000Z
  Duration: 01:34:34.09, start: 0.000000, bitrate: 12909 kb/s
  Chapters: <snip>
 
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1440x1080 [SAR 1:1 DAR 4:3], 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      title          : Aguirre, der Zorn Gottes (1972)
  Stream #0:1(ger): Audio: dts (DTS), 48000 Hz, 5.1(side), fltp, 1536 kb/s (default)
    Metadata:
      title          : DTS
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (dts (dca) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[h264 @ 0x556ed1f3d500] Hardware does not support image size 1440x1088 (constraints: width 64-0 height 64-0).
[h264 @ 0x556ed1f3d500] Failed setup for format vaapi: hwaccel initialisation returned error.
Impossible to convert between the formats supported by the filter 'Parsed_setparams_0' and the filter 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
[libfdk_aac @ 0x556ed1f48200] 2 frames left in the queue on closing
Conversion failed!

Tone mapping is all deactivated if that matters.

Any ideas?

Thanks!

[TheDreadPirate]

You added jellyfin to the video and render group?

[nyanmisaka]


vainfo: Driver version: Mesa Gallium driver 23.2.1 for PITCAIRN (, LLVM 13.0.1, DRM 2.50, 6.1.64-Unraid)

[h264 @ 0x556ed1f3d500] Hardware does not support image size 1440x1088 (constraints: width 64-0 height 64-0).


https://www.techpowerup.com/gpu-specs/amd-pitcairn.g420

Your AMD GPU is too old to support hardware transcoding. It doesn't even have the ability to decode 1080p videos.

[spicer]

Your AMD GPU is too old to support hardware transcoding

No that can't be it. It has UVD 4.0 and VCE 1.0 and according to https://www.x.org/wiki/RadeonFeature/#ra...erhardware it does have hw decode and encode capabilities. In particular both for MPEG 4 AVC aka H264

[spicer]

You added jellyfin to the video and render group?

Code:

root@unraid:~# getent group input | cut -d: -f3
71
root@unraid:~# getent group video | cut -d: -f3
18

In Unraid's docker settings for "bbergle-jellyfin", which is apparently the official image, as Extra Parameters:

Code:

--group-add="18"  --group-add="71" --env ROC_ENABLE_PRE_VEGA=1

Docker command:

Code:

docker run
  -d
  --name='bbergle-jellyfin'
  --net='host'
  -e TZ="Europe/Berlin"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="unraid"
  -e HOST_CONTAINERNAME="bbergle-jellyfin"
  -e 'TCP_PORT_8096'='8096'
  -e 'TCP_PORT_8920'='8920'
  -e 'UDP_PORT_7359'='7359'
  -e 'UDP_PORT_1900'='1900'
  -e 'JELLYFIN_PublishedServerUrl'='192.168.178.188'
  -e 'PUID'='99'
  -e 'PGID'='100'
  -e 'UMASK'='022'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:8096]'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/linuxserver/docker-templates/master/linuxserver.io/img/jellyfin-logo.png'
  -v '/mnt/user/':'/data/':'rw'
  -v '':'/xmltv':'rw'
  -v '/mnt/cache/appdata/bbergle-jellyfin':'/config':'rw'
  --device='/dev/dri/renderD128'
  --group-add="18"
  --group-add="71"
  --env ROC_ENABLE_PRE_VEGA=1
 'jellyfin/jellyfin:latest'
17eb6cf93b67b3221fbd642fa4497e56f80642b8b34e80c39addbb0031d3db13

[spicer]

Sorry for bumping, but it feels so close ... vainfo loads the driver and reports the various entrypoints. But ffmpeg just errors out. I googled the ffmpeg messages, but frankly those minutiae of ffmpeg and vaapi just go over my head.
Any ideas how to further diagnose the issue?
Thanks

[TheDreadPirate]

At this point, I think asking on the Unraid forum would probably be more productive. There aren't a lot of Unraid users here, even fewer that also use AMD GPUs with it, and we have no official documentation to try to work with.

If people on the Unraid forum are able to help you, do come back and share what you learned.

[spicer]

At this point, I think asking on the Unraid forum would probably be more productive.  There aren't a lot of Unraid users here, even fewer that also use AMD GPUs with it, and we have no official documentation to try to work with.

If people on the Unraid forum are able to help you, do come back and share what you learned.

Thanks for the response. So you think the problem lies in the docker configuration (maybe)?
I mean, everything seems to work - device gets passed, driver loaded, etc. - until ffmpeg gives an error. And since this is as far as I'm aware a Jf specific fork / patch this would be the appropriate place to ask.
Is there nothing interesting at all in the ffmpeg error messages?

Anyway, thanks again, I will bang my head a little more against this problem

[TheDreadPirate]

If this was plain docker, I think it is more straight forward. Mu understanding is that Unraid introduces some variables I don't know how to navigate.