2023-09-06, 03:56 PM
Just wanted to revisit this and say that I'm an idiot. Actually I'm just sleep-deprived and work four different jobs, so this stuff happens in bits between all the different things I do. So thank you @nyanmisaka for your help. Every time I search for help online, it's a post in a forum or on a Github issue from you helping some dumbass like me. To be fair, they're mostly more knowledgeable than I am since they're on the bleeding edge and I'm riding coattails.
I didn't remove the -pix_fmt option. I did a search before I started this morning and found a Tdarr reddit post from when the ARC cards were brand new and folks were struggling to get AV1 encoding working with the exact error message I'm getting. They found that both -pix_fmt and -profile:v options were causing issues. So I tried adding -provile:v main10 out of desperation. It failed, as expected. Then I noticed I still had the -pix_fmt option in my running document.
Wait....no...
Goddammit.
Gonna try a few more things (wanna check on other options available and try to break things) as well as figure out how to write metadata for things as I'm encoding, as I truly hate having incorrect stream information when I've re-encoded something.
Bottom line? This 19 GB+ remux came down to 815 MB in AV1 OPUS stereo at QP 20. I need to watch and verify it's not awful quality, may try a few other options that are *supposedly* available, but this was pretty stable at about 5.25x speed, which is pretty awesome. I need to see if I can get this working with ab-av1 as well...I don't think so, since I don't have VMAF compiled with jellyfin-ffmpeg but I could make that happen. It's always something stupid when you're at the one-yard line...
I didn't remove the -pix_fmt option. I did a search before I started this morning and found a Tdarr reddit post from when the ARC cards were brand new and folks were struggling to get AV1 encoding working with the exact error message I'm getting. They found that both -pix_fmt and -profile:v options were causing issues. So I tried adding -provile:v main10 out of desperation. It failed, as expected. Then I noticed I still had the -pix_fmt option in my running document.
Wait....no...
Code:
bitmap@server:$ /usr/lib/jellyfin-ffmpeg/ffmpeg -hide_banner -v verbose \
-hwaccel qsv -hwaccel_output_format qsv -qsv_device /dev/dri/renderD129 \
-i "/mnt/media/re-encode/testbed/input/media.mkv" \
-map 0:v -map 0:m:language:eng \
-c:v av1_qsv -preset slower -qp 20 -c:a libopus -ac 2 \
-b:a 256k -c:s copy "/mnt/media/re-encode/testbed/output/media-AV1.mkv.mkv"
[AVHWDeviceContext @ 0x556f1b0350c0] libva: VA-API version 1.19.0
[AVHWDeviceContext @ 0x556f1b0350c0] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x556f1b0350c0] libva: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x556f1b0350c0] libva: Found init function __vaDriverInit_1_19
[AVHWDeviceContext @ 0x556f1b0350c0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x556f1b0350c0] Initialised VAAPI connection: version 1.19
[AVHWDeviceContext @ 0x556f1b0350c0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.3.2 (95dc859).
[AVHWDeviceContext @ 0x556f1b0350c0] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x556f1b034bc0] Use Intel(R) oneVPL to create MFX session, API version is 2.9, the required implementation version is 1.3
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
libva info: VA-API version 1.19.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_19
libva info: va_openDriver() returns 0
[AVHWDeviceContext @ 0x556f1b034bc0] Initialize MFX session: implementation version is 2.9
[h264 @ 0x556f1b163d40] Reinit context to 1920x1088, pix_fmt: yuv420p
[matroska,webm @ 0x556f1bd98580] Could not find codec parameters for stream 3 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x556f1bd98580] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x556f1bd98580] Could not find codec parameters for stream 5 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x556f1bd98580] Could not find codec parameters for stream 6 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Selecting decoder 'h264_qsv' because of requested hwaccel method qsv
Input #0, matroska,webm, from '/mnt/media/re-encode/testbed/input/media.mkv':
Metadata: [REMOVED]
Stream mapping:
Stream #0:0 -> #0:0 (h264 (h264_qsv) -> av1 (av1_qsv))
Stream #0:1 -> #0:1 (dts (dca) -> opus (libopus))
Stream #0:3 -> #0:2 (copy)
Press [q] to stop, [?] for help
[AVHWDeviceContext @ 0x556f1bbe8bc0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.3.2 (95dc859).
[AVHWDeviceContext @ 0x556f1bbe8bc0] Driver not found in known nonstandard list, using standard behaviour.
[h264_qsv @ 0x556f1b2884c0] Decoder: output is video memory surface
[h264_qsv @ 0x556f1b2884c0] Use Intel(R) oneVPL to create MFX session with the specified MFX loader
[AVHWDeviceContext @ 0x556f1b123f80] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.3.2 (95dc859).
[AVHWDeviceContext @ 0x556f1b123f80] Driver not found in known nonstandard list, using standard behaviour.
[h264_qsv @ 0x556f1b2884c0] Decoder: output is video memory surface
[h264_qsv @ 0x556f1b2884c0] Use Intel(R) oneVPL to create MFX session with the specified MFX loader
[graph_1_in_0_1 @ 0x556f1ba53cc0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:5.1(side)
[format_out_0_1 @ 0x556f1b26c7c0] auto-inserting filter 'auto_aresample_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_1'
[auto_aresample_0 @ 0x556f1b8d2cc0] ch:6 chl:5.1(side) fmt:fltp r:48000Hz -> ch:2 chl:stereo fmt:flt r:48000Hz
[graph 0 input from stream 0:0 @ 0x556f1bddf7c0] w:1920 h:1080 pixfmt:qsv tb:1/1000 fr:24000/1001 sar:1/1
[av1_qsv @ 0x556f1b6651c0] Using input frames context (format qsv) with av1_qsv encoder.
[av1_qsv @ 0x556f1b6651c0] Encoder: input is video memory surface
[av1_qsv @ 0x556f1b6651c0] Use Intel(R) oneVPL to create MFX session with the specified MFX loader
[av1_qsv @ 0x556f1b6651c0] Using the variable bitrate (VBR) ratecontrol method
[av1_qsv @ 0x556f1b6651c0] profile: av1 main; level: 40
[av1_qsv @ 0x556f1b6651c0] GopPicSize: 250; GopRefDist: 4; GopOptFlag:; IdrInterval: 0
[av1_qsv @ 0x556f1b6651c0] TargetUsage: 2; RateControlMethod: VBR
[av1_qsv @ 0x556f1b6651c0] BufferSizeInKB: 750; InitialDelayInKB: 125; TargetKbps: 1000; MaxKbps: 1000; BRCParamMultiplier: 1
[av1_qsv @ 0x556f1b6651c0] NumRefFrame: 3
[av1_qsv @ 0x556f1b6651c0] IntRefType: 0; IntRefCycleSize: 0; IntRefQPDelta: 0; IntRefCycleDist: 0
[av1_qsv @ 0x556f1b6651c0] MaxFrameSize: 0;
[av1_qsv @ 0x556f1b6651c0] BitrateLimit: unknown; MBBRC: OFF; ExtBRC: unknown
[av1_qsv @ 0x556f1b6651c0] VDENC: ON
[av1_qsv @ 0x556f1b6651c0] BRefType: pyramid
[av1_qsv @ 0x556f1b6651c0] PRefType: default
[av1_qsv @ 0x556f1b6651c0] MinQPI: 0; MaxQPI: 0; MinQPP: 0; MaxQPP: 0; MinQPB: 0; MaxQPB: 0
[av1_qsv @ 0x556f1b6651c0] FrameRateExtD: 1001; FrameRateExtN: 24000
[av1_qsv @ 0x556f1b6651c0] NumTileRows: 1; NumTileColumns: 1; NumTileGroups: 1
[av1_qsv @ 0x556f1b6651c0] WriteIVFHeaders: OFF
[av1_qsv @ 0x556f1b6651c0] LowDelayBRC: OFF
[av1_qsv @ 0x556f1b6651c0] MaxFrameSize: 0;
Output #0, matroska, to '/mnt/media/re-encode/testbed/output/media-AV1.mkv':
Metadata: [REMOVED]
Automatically inserted bitstream filter 'pgs_frame_merge'; args=''e= 824.3kbits/s speed=5.13x
Error while decoding stream #0:0: Invalid data found when processing input
[matroska,webm @ 0x556f1bd98580] 0x00 at pos 12498345078 (0x2e8f59c76) invalid as first byte of an EBML number
Error while decoding stream #0:0: Invalid data found when processing input
[matroska,webm @ 0x556f1bd98580] 0x00 at pos 14859755260 (0x375b5defc) invalid as first byte of an EBML number
EOF in input file 04 q=-0.0 size= 832000kB time=01:32:24.04 bitrate=1229.4kbits/s speed=5.18x
Terminating demuxer thread 0
[h264_qsv @ 0x556f1b2884c0] A decode call did not consume any data: expect more data at input (-10)
Last message repeated 2 times
No more output streams to write to, finishing.
[out#0/matroska @ 0x556f1b663a00] All streams finished
[out#0/matroska @ 0x556f1b663a00] Terminating muxer thread
[AVIOContext @ 0x556f1bbaf400] Statistics: 854162871 bytes written, 2 seeks, 3260 writeouts
frame=132917 fps=124 q=-0.0 Lsize= 834140kB time=01:32:26.54 bitrate=1232.0kbits/s speed=5.18x
video:657213kB audio:165383kB subtitle:8970kB other streams:0kB global headers:0kB muxing overhead: 0.309612%
Input file #0 (/mnt/media/re-encode/testbed/input/media.mkv):
Input stream #0:0 (video): 132918 packets read (17272902706 bytes); 132917 frames decoded;
Input stream #0:1 (audio): 519731 packets read (1045698772 bytes); 519731 frames decoded (266102272 samples);
Input stream #0:2 (audio): 205 packets read (412460 bytes);
Input stream #0:3 (subtitle): 1218 packets read (9185094 bytes);
Input stream #0:4 (subtitle): 0 packets read (0 bytes);
Input stream #0:5 (subtitle): 0 packets read (0 bytes);
Input stream #0:6 (subtitle): 0 packets read (0 bytes);
Total: 654072 packets (18328199032 bytes) demuxed
Output file #0 (/mnt/media/re-encode/testbed/output/media-AV1.mkv):
Output stream #0:0 (video): 132917 frames encoded; 132917 packets muxed (672985953 bytes);
Output stream #0:1 (audio): 277190 frames encoded (266102272 samples); 277191 packets muxed (169352064 bytes);
Output stream #0:2 (subtitle): 1218 packets muxed (9185094 bytes);
Total: 411326 packets (851523111 bytes) muxed
[AVIOContext @ 0x556f1b6237c0] Statistics: 19393601045 bytes read, 5 seeksGoddammit.
Gonna try a few more things (wanna check on other options available and try to break things) as well as figure out how to write metadata for things as I'm encoding, as I truly hate having incorrect stream information when I've re-encoded something.
Bottom line? This 19 GB+ remux came down to 815 MB in AV1 OPUS stereo at QP 20. I need to watch and verify it's not awful quality, may try a few other options that are *supposedly* available, but this was pretty stable at about 5.25x speed, which is pretty awesome. I need to see if I can get this working with ab-av1 as well...I don't think so, since I don't have VMAF compiled with jellyfin-ffmpeg but I could make that happen. It's always something stupid when you're at the one-yard line...
Jellyfin 10.10.0 LSIO Docker | Ubuntu 24.04 LTS | i7-13700K | Arc A380 6 GB | 64 GB RAM | 79 TB Storage
![[Image: AIL4fc84QG6uSnTDEZiCCtosg7uAA8x9j1myFaFs...qL0Q=w2400]](https://lh3.googleusercontent.com/pw/AIL4fc84QG6uSnTDEZiCCtosg7uAA8x9j1myFaFsx_0wM5mKgTleZb466r4NtOHIRePq_CxyhsB72qsd-xuYFkqCGogLltAMN62BXsm-yFsSZcDuryqL0Q=w2400)