10.9 broke QSV VPP encoding

[bitmap]

Is this every video or one in particular? I'm running both an i7-13700k and an Arc A380 and have ZERO issues with QSV.

The "crucial lines" obfuscate the true cause of the issue, as a full, verbose ffmpeg log would provide more context, mediainfo would provide info on the file as well. This error, seems to happen when there is something incompatible about the ffmpeg command sent or when the file is not compatible with the decoder (e.g., issues with VC-1, Hi10p using QSV decoders). You can check a few things...

* What all is enabled in your HWA codec stack if you use QSV?
* Do you have LPE or prefer native enabled?
* Is VPP tonemapping enabled?
* Could it be related to this bug: https://github.com/jellyfin/jellyfin/issues/11380

Seems like there are kernel issues with the i915 driver. All I saw said that 6.8.4 should be fine, but if the patches applied to the driver were backported, you'd be stuck. You may see if you can run a different kernel on the VM as a test.

[nyanmisaka]

Please provide the complete ffmpeg log.

[TheMaFa]

I used the official Intel page as a reference for what the UHD 770 supports.
For decoding I have enabled everything. Encoding only for HEVC since AV1 encoding is not supported (I don't use it anyway).

Things are kind of complex (LPE always enabled, disabling it doesn't change anything):
As I said VA-API works fully with LPE and VPP, but relatively slow.
QSV with VPP and prefer native works fine but also slow which makes sense, I guess.
QSV with prefer native disabled and VPP enabled crashes (full error log below) for HDR content (4K and 1080p), SDR works fine (4K and 1080p).
QSV with prefer native disabled and VPP disabled works fine for 4k, 1080p HDR and SDR, just without the tone mapping.

It's important to mention that VPP used to work prior to the 10.9 upgrade and I would like to keep using VPP if possible.
Although using the other tone mapping method works. Is there a speed difference between BT.2390 and VPP?
If there are no other benefits like less power consumption then I would just call this resolved since I don't really care.
I get the high speeds of QSV and LPE works too.

Here is the full ffmpeg log of HDR content trying to be transcoded (QSV, VPP enabled):
{"Protocol":0,"Id":"f2ee90a70aabcaa27bb5b7bfb9ad250c","Path":"<moviefilepath>","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"mkv,webm","Size":1078788480,"Name":"<moviename>","IsRemote":false,"ETag":"2e16930e9bad6e86d0bcea99f7051d83","RunTimeTicks":60411301888,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":0,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"hevc","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":"bt2020nc","ColorTransfer":"smpte2084","ColorPrimaries":"bt2020","DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":null,"VideoRange":2,"VideoRangeType":2,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"1080p HEVC HDR","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":null,"BitRate":1428591,"BitDepth":10,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":804,"Width":1920,"AverageFrameRate":24,"RealFrameRate":24,"Profile":"Main 10","Type":1,"AspectRatio":"2.40:1","Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuv420p10le","Level":120,"IsAnamorphic":null},{"Codec":"ac3","CodecTag":null,"Language":"ger","ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/1000","CodecTimeBase":null,"Title":"Surround","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"Surround - Ger - Dolby Digital - 5.1 - Default","NalLengthSize":null,"IsInterlaced":false,"IsAVC":null,"ChannelLayout":"5.1","BitRate":640000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":6,"SampleRate":48000,"IsDefault":true,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null}],"MediaAttachments":[],"Formats":[],"Bitrate":2068591,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":0,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv -c:v hevc_qsv -noautorotate -i file:"<moviefilepath>" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:s -codec:v:0 h264_qsv -low_power 1 -preset veryfast -look_ahead 0 -b:v 5952463 -maxrate 5952463 -bufsize 11904926 -g:v:0 72 -keyint_min:v:0 72 -vf "setparams=color_primaries=bt2020:color_trc=smpte2084:colorspace=bt2020nc,hwmap=derive_device=vaapi,procamp_vaapi=b=16:c=1,tonemap_vaapi=format=nv12:p=bt709:t=bt709:m=bt709:extra_hw_frames=32,hwmap=derive_device=qsv" -codec:a:0 libfdk_aac -ab 256000 -af "volume=4.25,pan=stereo|c0=0.5*c2+0.707*c0+0.707*c4+0.5*c3|c1=0.5*c2+0.707*c1+0.707*c5+0.5*c3" -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "eb52b5f404d2054c4fcee93579b319f3-1.mp4" -start_number 0 -hls_segment_filename "/var/lib/jellyfin/transcodes/eb52b5f404d2054c4fcee93579b319f3%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/var/lib/jellyfin/transcodes/eb52b5f404d2054c4fcee93579b319f3.m3u8"


ffmpeg version 6.0.1-Jellyfin Copyright © 2000-2023 the FFmpeg developers
built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
Input #0, matroska,webm, from 'file:<moviefilepath>':
Metadata:
title : <themovie>
creation_time : 2024-04-09T19:54:10.000000Z
ENCODER : Lavf60.16.100
Duration: 01:40:41.13, start: 0.000000, bitrate: 1428 kb/s
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 1920x804 [SAR 1:1 DAR 160:67], 24 fps, 24 tbr, 1k tbn (default)
Metadata:
DURATION : 01:40:41,130000000
Side data:
Content Light Level Metadata, MaxCLL=4000, MaxFALL=1000
Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2650,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.000100, max_luminance=1000.000000
Stream #0:1(ger): Audio: ac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
Metadata:
title : Surround
DURATION : 01:40:41,120000000
Stream mapping:
Stream #0:0 -> #0:0 (hevc (hevc_qsv) -> h264 (h264_qsv))
Stream #0:1 -> #0:1 (ac3 (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
libva info: VA-API version 1.21.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_21
libva info: va_openDriver() returns 0
[h264_qsv @ 0x567a21e49d00] Error querying (IOSurf) the encoding parameters: invalid video parameters (-15)
[vost#0:0/h264_qsv @ 0x567a21e04480] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[libfdk_aac @ 0x567a21e01cc0] 2 frames left in the queue on closing
Conversion failed!

[nyanmisaka]

Please disable the "Low-Power encoder" and try again.

[TheMaFa]

Same exact error with VPP enabled but LPE disabled.
I will not mark this as resolved because it isn't but I found a solution that works for me right now.
QSV, prefer native disabled, LPE enabled and BT.2390 for tone-mapping.
Thanks for your help. Of course I'll still be grateful if we could resolve this issue.

[nyanmisaka]

This is a really weird issue. It makes me wonder if the new ffmpeg6, media-driver(iHD) and libdrm have some incompatibility with the LXC environment. I've been using it on host and docker and never seen this error, could you also try it on host for example (Ubuntu 24.04 LTS or Debian 12)?

[TheMaFa]

Update:
I migrated Jellyfin to a fresh Ubuntu 24.04 LXC and everything works fine now.
I don't know wheter it was the new Ubuntu version or a new Jellyfin (ffmpeg6) release.
VPP and LPE are both confirmed working.

Thanks for your help!