SOLVED: Subtitle burn-in with hardware decoding error (v10.9)

[Maybe0842]

Hi.

I am running the official docker image on Debian x64 with AMD CPU. After upgrading to v10.9, transcoding with subtitle burn-in would fail when hardware decoding is turned on (software decoding or hardware encoding are both fine). I tried SRT and ASS subtitles and both of them failed. Any help would be appreciated. Thanks.

Here is the error message in the ffmpeg log (full ffmpeg log attached):

Code:

...
[Parsed_subtitles_2 @ 0x55ef0527af80] libass API version: 0x1701000
[Parsed_subtitles_2 @ 0x55ef0527af80] libass source: commit: 0.17.1-0-ge8ad72accd3a84268275a9385beb701c9284e5b3
[Parsed_subtitles_2 @ 0x55ef0527af80] Shaper: FriBidi 1.0.13 (SIMPLE) HarfBuzz-ng 8.4.0 (COMPLEX)
[Parsed_subtitles_2 @ 0x55ef0527af80] Using font provider fontconfig
[AVHWDeviceContext @ 0x55ef05277900] Cannot map DRM frame to Vulkan: VK_ERROR_FORMAT_NOT_SUPPORTED
[Parsed_hwmap_6 @ 0x55ef04fe4700] Failed to map frame: -542398533.
Error while filtering: Generic error in an external library
Failed to inject frame into filter network: Generic error in an external library
Error while processing the decoded data for stream #0:0
Conversion failed!

Output of vainfo:

Code:

docker exec -it jellyfin /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128
Trying display: drm
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_21
amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description.
If they do, bad things may happen!
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.21 (libva 2.21.0)
vainfo: Driver version: Mesa Gallium driver 24.0.5 for AMD Radeon Graphics (radeonsi, renoir, LLVM 16.0.6, DRM 3.40, 5.10.0-28-amd64)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main              : VAEntrypointVLD
      VAProfileH264Main              : VAEntrypointEncSlice
      VAProfileH264High              : VAEntrypointVLD
      VAProfileH264High              : VAEntrypointEncSlice
      VAProfileHEVCMain              : VAEntrypointVLD
      VAProfileHEVCMain              : VAEntrypointEncSlice
      VAProfileHEVCMain10            : VAEntrypointVLD
      VAProfileHEVCMain10            : VAEntrypointEncSlice
      VAProfileJPEGBaseline          : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                  : VAEntrypointVideoProc

Output of ffmpeg vulkan info:

Code:

docker exec -it jellyfin /usr/lib/jellyfin-ffmpeg/ffmpeg -v debug -init_hw_device vulkan
ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec    60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter    9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample  4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Splitting the commandline.
Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
Reading option '-init_hw_device' ... matched as option 'init_hw_device' (initialise hardware device) with argument 'vulkan'.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option v (set logging level) with argument debug.
Applying option init_hw_device (initialise hardware device) with argument vulkan.
[AVHWDeviceContext @ 0x5615f610e500] Supported validation layers:
[AVHWDeviceContext @ 0x5615f610e500]    VK_LAYER_MESA_device_select
[AVHWDeviceContext @ 0x5615f610e500]    VK_LAYER_MESA_overlay
[AVHWDeviceContext @ 0x5615f610e500] GPU listing:
[AVHWDeviceContext @ 0x5615f610e500]    0: AMD Radeon Graphics (RADV RENOIR) (integrated) (0x164c)
[AVHWDeviceContext @ 0x5615f610e500] Device 0 selected: AMD Radeon Graphics (RADV RENOIR) (integrated) (0x164c)
[AVHWDeviceContext @ 0x5615f610e500] Queue families:
[AVHWDeviceContext @ 0x5615f610e500]    0: graphics compute transfer (queues: 1)
[AVHWDeviceContext @ 0x5615f610e500]    1: compute transfer (queues: 4)
[AVHWDeviceContext @ 0x5615f610e500]    2: sparse (queues: 1)
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_KHR_push_descriptor
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_KHR_sampler_ycbcr_conversion
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_KHR_synchronization2
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_KHR_external_memory_fd
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_EXT_external_memory_dma_buf
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_EXT_image_drm_format_modifier
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_KHR_external_semaphore_fd
[AVHWDeviceContext @ 0x5615f610e500] Using device extension VK_EXT_external_memory_host
[AVHWDeviceContext @ 0x5615f610e500] Using device: AMD Radeon Graphics (RADV RENOIR)
[AVHWDeviceContext @ 0x5615f610e500] Alignments:
[AVHWDeviceContext @ 0x5615f610e500]    optimalBufferCopyRowPitchAlignment: 1
[AVHWDeviceContext @ 0x5615f610e500]    minMemoryMapAlignment:              4096
[AVHWDeviceContext @ 0x5615f610e500]    minImportedHostPointerAlignment:    4096
[AVHWDeviceContext @ 0x5615f610e500] Using queue family 0 (queues: 1) for graphics
[AVHWDeviceContext @ 0x5615f610e500] Using queue family 1 (queues: 4) for compute transfers
Successfully parsed a group of options.
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

[TheDreadPirate]

What kind of AMD GPU and CPU do you have?

[Maybe0842]

CPU is AMD Ryzen 3 5300U. There is no dedicated GPU.

[TheDreadPirate]

Are you using Vulkan or VAAPI?

[Maybe0842]

I believe VAAPI. I've attached a screenshot of the transcoding settings page.

[TheDreadPirate]

@nyanmisaka - Any ideas?

[Maybe0842]

Is there any update? Anybody looking into this?

[nyanmisaka]

5.10.0-28-amd64

This is a fairly old kernel version, and some features required by Vulkan<->VAAPI interop are not yet supported in it.

Can you upgrade to 5.15+ or 6.1+ kernel and try again?

[Maybe0842]

Yea that seemed to be the cause. I upgraded my kernel to 6.1 and initial test is working great. I will report back if I encounter the issue again. Thank you very much.