Encoding Discussion Megathread (ffmpeg, Handbrake, AV1, etc...)

[TheDreadPirate]

Here is the ffmpeg command from a tone mapped stream. Perhaps something can be adapted for your purposes?

Code:

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -probesize 1G -init_hw_device vaapi=va:,kernel_driver=i915,driver=iHD -init_hw_device qsv=qs@va -init_hw_device opencl=ocl@va -filter_hw_device qs -hwaccel vaapi -hwaccel_output_format vaapi -noautorotate -canvas_size 1920x1080 -i file:"/media/library/Movies/Dune (2021)/Dune.mkv" -noautoscale -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -map -0:0 -codec:v:0 h264_qsv -low_power 1 -preset veryfast -look_ahead 0 -b:v 13305202 -maxrate 13305202 -bufsize 26610404 -g:v:0 72 -keyint_min:v:0 72 -filter_complex "[0:6]scale=-1:1080:fast_bilinear,scale,crop,pad=max(2592\,iw):max(1080\,ih):(ow-iw)/2:(oh-ih)/2:black@0,crop=2592:1080,format=bgra,hwupload=derive_device=qsv:extra_hw_frames=64[sub];[0:0]setparams=color_primaries=bt2020:color_trc=smpte2084:colorspace=bt2020nc,procamp_vaapi=b=16:c=1,tonemap_vaapi=format=nv12:p=bt709:t=bt709:m=bt709:extra_hw_frames=32,hwmap=derive_device=qsv,format=qsv[main];[main][sub]overlay_qsv=eof_action=pass:repeatlast=0:w=3840:h=1600" -start_at_zero -codec:a:0 libfdk_aac -ac 2 -ab 256000 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type fmp4 -hls_fmp4_init_filename "2a73e6555e0f365751b7712a4297833f-1.mp4" -start_number 0 -hls_segment_filename "/config/transcodes/2a73e6555e0f365751b7712a4297833f%d.mp4" -hls_playlist_type vod -hls_list_size 0 -y "/config/transcodes/2a73e6555e0f365751b7712a4297833f.m3u8"

In the filter complex it calls out both VAAPI and QSV. And since earlier in the command it configures "-init_hw_device opencl=ocl@va", could you do something similar for your command? Instead of "scale_opencl" it would be "scale_vaapi"? Etc.

[bitmap]

This is somewhat helpful for a number of reasons, but I tried several iterations and was unsuccessful. I'm not sure I fully understand the syntax of the filter_complex (as I always get an error there if I'm not doing something like in the snippet you provided: overlaying subs). I tried a regular filter with deriving the ocl device, then ending with hwmap for qsv and that didn't work. Added format, no dice. Even tried vpp_qsv to set the format.

I guess I'm not wrapping my head around the ffmpeg side of things...I want to QSV decode --> OpenCL filter --> QSV encode. Seems like it should be possible, but I dunno. Not interested in chasing ghosts, though...

[bitmap]

Okay, so I was able to put together a hybrid encoding script to allow the use of nlmeans_opencl. The impact to performance is SIGNIFICANT, so I may end up looking for other solutions, but this is pretty cool and I did use suggestions from our resident ffmpeg expert earlier in the thread. Take a look!

Code:

cd "/path/to/input/SOURCE_MEDIA (1991)" && \
mkdir -p "/path/to/output/FINAL_MEDIA (1991)" && \
jffmpeg -y -hide_banner -v verbose -stats \
  -init_hw_device vaapi=hw:/dev/dri/renderD129 -init_hw_device qsv=qs@hw -init_hw_device opencl=ocl@hw -filter_hw_device ocl \
  -analyzeduration 200M -probesize 1G \
  -i "SOURCE_MEDIA.mkv" \
  -map 0 -map -0:a -map -0:s -map -0:t? -map 0:a:1 -map 0:a:0 -map 0:s:1 -map 0:s:0 \
  -c copy -c:v:0 "av1_qsv" -preset "veryslow" \
  -global_quality:v:0 22 -low_delay_brc 1 -extbrc 1 -look_ahead_depth 100 \
  -filter:v:0 "hwupload=extra_hw_frames=64,tonemap_opencl=tonemap=bt2390:format=yuv420p,nlmeans_opencl=1.0:7:5:3:3,scale_opencl=algo=lanczos:h=1080:w=-1:format=yuv420p,hwdownload,format=yuv420p" \
  -c:a libopus -b:a:0 160k -ac:a:0 2 -b:a:1 256k -ac:a:1 6 \
  -metadata title="FINAL_MEDIA (1991)" \
  -metadata:s:v:0 title='AV1 Encoding by bitmap' \
  -metadata:s:a:0 title='Stereo' -metadata:s:a:0 language='eng' -disposition:a:0 default \
  -metadata:s:a:1 title='5.1 Surround' -metadata:s:a:1 language='eng' -disposition:a:1 0 \
  -metadata:s:s:0 title='English' -metadata:s:s:0 language='eng' -disposition:s:0 0 \
  -metadata:s:s:1 title='English (SDH)' -metadata:s:s:1 language='eng' -disposition:s:1 0 \
  "/path/to/output/FINAL_MEDIA (1991)/FINAL_MEDIA (1991).mkv"

This snippet shows the use of tonemap_opencl, scale_opencl (<3 lanczos), and nlmeans_opencl all verified working. The nlmeans_opencl variables were taken from a recommendation seen elsewhere...I'm not an expert on modifying those, but these are a good starting point.

[TheDreadPirate]

What do these two options do?

Code:

-low_delay_brc 1 -extbrc 1

Also, are you tone mapping to SDR? Is that for compatibility? Or is that just a way to deal with Dolby Vision since AV1 doesn't really support it?

[bitmap]

My understanding -- which is limited by the sparse documentation -- is that low_delay_brc allows for better (more responsive, maybe?) control of bitrate during encoding. Meaning if there's a massive shift in bitrate (e.g., a standoff followed by a chaotic bar fight), low_delay_brc enables ffmpeg to adjust to that shift more effectively.

For extbrc there's no real explanation, however, it enables you to use look_ahead_depth which selects LA_ICQ (intelligent constant quality with look ahead) as the determining quality factor for encoding. Look ahead -- again, my understanding -- allows the encoder to keep frames in memory and look ahead in the file (100 frames is the limit in QSV) to assess the best approach to encoding.

I use both of these because I've found that LA_ICQ provides higher quality, smoother transitions during action scenes, and smaller files overall. I'd say it also probably bloats some files that are insanely grainy if they're not denoised properly (I suck at denoising).

EDIT: This is probably a better question to ask nyanmisaka (not tagging since it's not super important).

[TheDreadPirate]

Interesting. I get files that are exactly the same size but different check sums.

My script's ffmpeg command

Code:

ffmpeg \
                -init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 -y -hide_banner -stats -fflags genpts -loglevel error \
                -init_hw_device qsv=qs@va \
                -filter_hw_device qs \
                -hwaccel qsv \
                -hwaccel_output_format qsv \
                -i "$INPUTFILE" \
                $INPUT_LIST \
                -ss $startTime \
                -to $endTime \
                -map_chapters 0 \
                $MAPPING \
                -c:v av1_qsv \
                -preset veryslow \
                -vf vpp_qsv="$FILTERS" \
                -c:a copy \
                -c:s copy \
                -disposition:s 0 \
                -global_quality:v 26 \
                -look_ahead 1 \
                -look_ahead_depth 100 \
                "$outputFile"

All I did was add -low_delay_brc 1 -extbrc 1.

Code:

-rwxr--r-- 1 chris jellyfin  5671988708 Jun 25 20:51 'Sailor Moon Sailor Stars Disc 4_t01.mkv'
-rw-rw-r-- 1 chris jellyfin  1430809789 Jun 29 01:38 new.mkv
-rw-rw-r-- 1 chris jellyfin  1430809789 Jun 29 01:34 og.mkv

08d01f409b0d9039cfb7bb97bdb923fa  new.mkv
34d352dfe600b3196e590b4f6e30cdd8  og.mkv

[bitmap]

The look_ahead_depth flag only functions if you enable extbrc as well. Add extbrc and see what happens. It honestly could be confirmation bias and nothing changes, but that seems like a weird feature to add...

EDIT: Additionally, you should remove "look_ahead 1" because that's not supported by av1_qsv (and seems to only be supported with h264_qsv and the VBR quality options). In other words:

Code:

ffmpeg \
                -init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 -y -hide_banner -stats -fflags genpts -loglevel error \
                -init_hw_device qsv=qs@va \
                -filter_hw_device qs \
                -hwaccel qsv \
                -hwaccel_output_format qsv \
                -i "$INPUTFILE" \
                $INPUT_LIST \
                -ss $startTime \
                -to $endTime \
                -map_chapters 0 \
                $MAPPING \
                -c:v av1_qsv \
                -preset veryslow \
                -vf vpp_qsv="$FILTERS" \
                -c:a copy \
                -c:s copy \
                -disposition:s 0 \
                -global_quality:v 26 \
                -extbrc 1
                -look_ahead_depth 100 \
                "$outputFile"

[TheDreadPirate]

@bitmap - Since you're also running 24.04, just a heads up that kernel 6.8.0-38 that was released earlier this week inherited an upstream regression that affects Intel Arc GPUs. But only when you are scaling the resolution of the video. Transcodes/encoding that does not scale the output resolution is not affected, AFAICT.

Nyan submitted a ticket to Ubuntu's kernel team to backport upstream patches into Ubuntu's kernel. One of the kernel devs released a test kernel with the patches that I will be testing later tonight. If successful, Ubuntu will include the backports in a soon to be released kernel update.

Stay on 6.8.0-36 for now.

[bitmap]

Appreciate the heads up @TheDreadPirate! I'm still on 6.8.0-36, I'll make sure I hold that for now. I'm running into a bug with the vpp_qsv for a few encodes I'm running that's forcing me to utilize SVT-AV1 and/or vaapi alternatives, which don't produce results of the same quality IMO. Most of these are MPEG-2 media, but I did some searching and found a bug report nyan submitted to ffmpeg as a regression due to vpp_qsv refactoring. It sucks because the QSV deinterlacer is pretty spiffy with older media.

I am getting pretty good with mixed-acceleration encoding. I can use SVT-AV1 with nlmeans_opencl and the speed impact is ZERO (although SVT-AV1 runs slow as hell anywhere below preset 4 where great results and small sizes collide). I've been able to mix opencl and qsv filters as well. Now I need to figure out how to mess with vulkan and placebo....

[TheDreadPirate]

@bitmap I can't remember if I asked this before, do you have any recommendations for dealing with film grain while sticking with QSV?

I have an anime movie that is really struggling to compress.  Lots of film grain and the output is terrible.

Left is the AV1 encode, right is the original.