Encoding Discussion Megathread (ffmpeg, Handbrake, AV1, etc...)

[bitmap]

I wanted to get this started and I'll throw in some of my basic code that I've been using for AV1 encodes recently, both with ffmpeg as well as ab-av1, which I've grown to really like. I don't have a lot of time to spare at the moment, but want to welcome anybody who wants to contribute with questions they have, suggestions to revise code that's posted, programs/scripts to use, better routes to go, guides they've utilized in the past, or even struggles they're having.

This is an open topic that I don't think fits anywhere else but is something we all work with and around. Open discussion is welcome, but please keep it friendly and constructive as well as within forum rules. Maybe replace your file names with Big Buck Bunny or something like that. ;-)

Here's a sample ffmpeg bash for loop I used to encode a season (could easily be adapted using find to do an entire folder or drive):

Code:

for i in *.mkv; do \
    ffmpeg -i "${i}" \
    -map 0:v -map 0:m:language:eng \
    -c:v libsvtav1 \
    -svtav1-params "preset=5:crf=27:film-grain=2:input-depth=10:tune=0" \
    -pix_fmt yuv420p10le \
    -c:a libopus -ac 2 -b:a 160k \
    -af "pan=stereo|FL<FC+0.30*FL+0.30*BL|FR<FC+0.30*FR+0.30*BR"
    -c:s copy \
    "${i%.mkv} [AV1 10bit OPUS 2.0][EN][EN]-rgName.mkv"; done

This is designed to grab only English-language streams, encode using the SVT-AV1 encoder with some params I've since found aren't particularly necessary (input-depth may not be required) as well as the "midnight" downmixing for better dialog mix in stereo audio. This provided a VMAF score of about 94.5% which was reasonable, but not amazing and you could definitely tell. I used it on a series that was low-risk for me and this was part of my experimentation. I have since refined, but this is a good, clean look at what a fairly simple (yet complex) ffmpeg command can look like.

Here's a similar ab-av1 command:

Code:

for i in *.mkv; do \
ab-av1 auto-encode \
--encoder libsvtav1 --enc map=0:v --enc map=0:m:language:eng --preset 4 \
--pix-format yuv420p10le \
--svt enable-hdr=1 --svt tune=0 --svt film-grain=10 \
--acodec libopus --downmix-to-stereo --enc b:a=160k \
--enc af="pan=stereo|FL<FC+0.30*FL+0.30*BL|FR<FC+0.30*FR+0.30*BR" \
-i "${i}" -o "${i%.mkv} [WEBDL-2160p AV1 HDR10Plus 10bit OPUS 2.0][EN][EN]-rgName.mkv"; done

You'll see some similarities. Notice film-grain has changed, HDR is enabled, input-depth is gone, and the preset has been lowered. This is a 4K source and I went a lot more granular and did testing to ensure I could get a VMAF of at least 95%. I tried for 96% and that would've taken three days per episode on CPU. I haven't gotten QSV working with either my CPU or my A380 yet, so if anybody has experience there, I'd appreciate a boost.

Also looking for any advanced tuning that folks suggest...I haven't touched my anime as of yet because I'm a little scared and most of my sources aren't remux quality, but I do have the ability to rip some remuxes and re-encode them, which I would like to do if I can tune a profile well enough to get good AV1 encodes. From what I understand, folks use some pretty complex software with crazy profiles to improve upon the BD rips they get, though. Not sure I'm at that level...

So please, share, ask questions, start the discussion!

[TheDreadPirate]

Continuing our convo from the other thread. I'm not at home yet to post my Handbrake json. But here is the gist of my two Handbrake presents.

"Blurays" preset
MKV container
no filters
Automatically detect and trim black bars, otherwise don't change resolution
Variable frame rate, match source (always results in constant frame rate)
NVENC HEVC 8-bit, CQ26, slow preset, main profile, auto level
Direct copy all audio streams
Direct copy all subtitles
Direct copy chapters

"DVDs" preset
MKV container
De-interlace filter (Will edit with specifics later)
Automatically detect and trim black bars, otherwise don't change resolution
Variable frame rate, match source (always results in constant frame rate)
NVENC HEVC 8-bit, CQ26, slow preset, main profile, auto level
Direct copy all audio streams
Direct copy all subtitles
Direct copy chapters


What would that ffmpeg command look like? Keeping in mind that I will be doing this with QSV on my server.

[bitmap]

"Blurays" preset
MKV container
no filters
Automatically detect and trim black bars, otherwise don't change resolution
Variable frame rate, match source (always results in constant frame rate)
NVENC HEVC 8-bit, CQ26, slow preset, main profile, auto level
Direct copy all audio streams
Direct copy all subtitles
Direct copy chapters

The only hard part here is the cropping of black bars. I both understand and don't why folks do this -- if you wouldn't mind sharing your thoughts? I'll share what I have so far, need to get back to work, but the rest is just putting this together into a bash script, which is fairly easy.

This will output exactly what you need for the ffmpeg video filter to crop out the black bars -- keep in mind it's automated, so may over- or under-crop a bit. It also doesn't zoom so you're not getting a full resolution, you're getting whatever comes out. You can correct this, but it gets more complicated and you dilute your video quality if you mess with it any further.

Code:

ffmpeg -ss 10 -i 'input_media.mkv' -t 1 -vf cropdetect -f null - 2>&1 | awk '/crop=/ {a=$NF} END{print a}'

The explanation here is it skips the first 10 seconds (to get past a likely black intro card -- you can customize this to skip further if needed, uses the cropdetect filter, directs the output away from the console, and uses awk to only grab the final output of the command, which is the cropping information formatted exactly as the video filter needs it. Here it is in context, thrown into a bash variable to see if I could set it up that way so I could then use it inside of a script with ffmpeg:

Code:

bitmap@server:$ crop=$(ffmpeg -ss 10 -i 'input_media.mkv' -t 1 -vf cropdetect -f null - 2>&1 | awk '/crop=/ {a=$NF} END{print a}')
bitmap@server:$ echo $crop
crop=1904:800:10:0

Next is the ffmpeg command to actually crop it:

Code:

ffmpeg -i input_media.mkv -vf "${crop}" cropped_media.mkv

Obviously that needs the rest of the customization...I'll come back later when I have more time. The auto-cropping was the worst part of it, the rest is fairly simple and I don't think I'll need to look up as much to write it out...

[TheDreadPirate]

The only hard part here is the cropping of black bars. I both understand and don't why folks do this -- if you wouldn't mind sharing your thoughts?

On TVs and my computer monitors, it doesn't make a difference.  But most phones today have an aspect ratio around 21:9.  Including my Pixel 6a.  Leaving the black bars in results in the video not filling the screen if the movie is wider than 16:9.  Which is not ideal on an already small screen.  I stopped leaving black bars in my rips almost from the get go when I was still using Plex.  So I don't know if Jellyfin allows zoom to fill or something like that.

From an encoding efficiency standpoint, I know there is little to no gain from removing them.

[bitmap]

Continuing our convo from the other thread.  I'm not at home yet to post my Handbrake json.  But here is the gist of my two Handbrake presents.

"Blurays" preset
MKV container
no filters
Automatically detect and trim black bars, otherwise don't change resolution
Variable frame rate, match source (always results in constant frame rate)
NVENC HEVC 8-bit, CQ26, slow preset, main profile, auto level
Direct copy all audio streams
Direct copy all subtitles
Direct copy chapters

Okay, back at it. I had the main stuff put together with a few things to go. Note that this is untested at the moment as I just wrote it. I'll grab something like Big Buck Bunny or something to throw at it later, but I can't test the NVENC version, but I'm sure most of it is right aside from the pix_fmt option as I don't know which formats are supported by the encoder...since it's not documented on the ffmpeg site and I had to find a Github page with the help printout to get what I needed. Anyway. Here's the NVENC version. All that's going to change are a few options between this and the QSV version.

Code:

# Written to emulate Handbrake Bluray rip preset for HEVC NVENC, crop black bars, auto-level, main8
for i in *.mkv; do \
    crop=$(ffmpeg -ss 10 -i 'input_media.mkv' -t 1 -vf cropdetect -f null - 2>&1 | awk '/crop=/ {a=$NF} END{print a}') \
    ffmpeg -hwaccel_output_format cuda [-gpu <index>] \
    -i "${i}" -vf "pp=al, ${crop}" -map 0:v -map 0:a -map 0:s -map_chapters 0 \
    -c:v hevc_nvenc -profile main -preset slow -cq 26 -pix_fmt yuv420p \
    -c:a copy -c:s copy "${$i%mkv}_processed.mkv"; done

So we get the crop value, start the encode by telling ffmpeg we're using NVENC -- if you only have one device, leave out the GPU option. From there, specify the input file and apply two video filters. The first is a sort of crude auto-leveling analog (I have not used this one before, so testing it would be advised before using it on something you cherish). Next is the black bar cropping with a call to our crop variable, which is already formatted exactly as we need. From there, we map all the video, audio, subs, and chapters to make sure we don't miss anything. Specify the video codec, profile, preset, CQ, pix_fmt (standard pix_fmt for 8-bit video), then specify the codec is COPY for audio and subs, and put our output file with "_processed" at the end of the file name. You can also specify a different directory or whatever you like here, this is where I'm sure you don't need help.

Frame rate is always matched, unless your source is variable framerate. You can specify -vsync (which is technically deprecated, but still works) or -fps_mode (which replaced -vsync) if you run across this issue.

I'll tackle the DVD preset soon. Deinterlacing is personal preference, but YADIF does a good job, I just ran Mission Hill through it and got great results. So I have something to go off, and since it's essentially the same preset, just adding that filter, it will look very similar. For reference, here's the QuickSync version, but I won't explain that one in detail:

Code:

# Written to emulate Handbrake Bluray rip preset for HEVC QSV, crop black bars, auto-level, main8
for i in *.mkv; do \
    crop=$(ffmpeg -ss 10 -i 'input_media.mkv' -t 1 -vf cropdetect -f null - 2>&1 | awk '/crop=/ {a=$NF} END{print a}') \
    ffmpeg -hwaccel_output_format -qsv_device /dev/dri/renderD128 \
    -i "${i}" -vf "pp=al, ${crop}" -map 0:v -map 0:a -map 0:s -map_chapters 0 \
    -c:v hevc_qsv -profile main -preset slow -global_quality:v 26 -pix_fmt yuv420p \
    -c:a copy -c:s copy "${$i%mkv}_processed.mkv"; done

Please ask any questions you have or if there are things I missed, more customizations you're interested in, things you don't know that you might want, etc... This was fun, it stretched me a little bit!

[TheDreadPirate]

Couldn't get the crop filter to work. The resulting crop variable always ended up being null. Here is what I ended up using for a simple QSV recompression. Based on what you posted + using the QSV device parameters from jellyfin ffmpeg logs.

Code:

ls *.mkv | while read file ; do
     sudo /usr/lib/jellyfin-ffmpeg/ffmpeg -init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 \
     -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv -i "$file" -map 0:v -map 0:a \
     -map 0:s -map_chapters 0 -c:v hevc_qsv -profile main -preset slow -cq 26 -pix_fmt yuv420p \
     -c:a copy -c:s copy "recompressed.mkv" ;
done

But I will continue experimenting with the crop filter.

[bitmap]

Bummer! I need to run some tests, as with my latest compiled ffmpeg it's bloating absolutely everything beyond belief and I'm not sure why. I was getting excellent results previously.

While I'm at it I'll see if I can refine what I put together before. What's in the Jellyfin logs definitely offers a glimpse of things a bit past where I'm at, but I always hesitate to throw ffmpeg options I wasn't able to find documentation for, so I'll do a deeper dive. I always like learning new things about encoding.

[TheDreadPirate]

Do note that I am using jellyfin-ffmpeg. I didn't want to install vanilla ffmpeg and risk breaking something.

[TheDreadPirate]

Here is my progress.  I added functionality to split files if there is a known chapter count per episode.  USUALLY the same.  Also accepts "all" for movies and such or if the MKVs came off the disc as 1 episode per file.

No deinterlacing or cropping.  I am leaning towards letting clients do the deinterlacing for my content going forward.  Still need to get cropping working.

Code:

#!/bin/bash

chaptersEpisode=$1

if [ -z "$chaptersEpisode" ]; then
        echo "Chapters per episode was not provided"
        exit 1
fi

ls *.mkv | while read file; do
        sudo /usr/lib/jellyfin-ffmpeg/ffprobe -i $file -print_format json -show_chapters -loglevel error > chapters.json
        rawChapters=$(jq .chapters[].start_time chapters.json | wc -l)
        totalChapters=$(echo $(($rawChapters - 1)) | bc)
        i=0
        startStop=""
        if [ "$chaptersEpisode" == "all" ]; then
                startStop="0:$totalChapters"
        else
                while [ $i -lt $totalChapters ]; do
                        first=$i
                        last=$(echo $(($i + $chaptersEpisode - 1)) | bc)
                        startStop=$(echo "$startStop $first:$last")
                        i=$(echo $(($i + $chaptersEpisode)) | bc)
                done
        fi
        for chapter in `echo $startStop` ; do
                begin=$(echo $chapter | awk -F ':' '{print $1}')
                end=$(echo $chapter | awk -F ':' '{print $2}')
                startTime=$(jq .chapters[$begin].start_time chapters.json | sed 's/\"//g')
                endTime=$(jq .chapters[$end].start_time chapters.json | sed 's/\"//g')
                sudo /usr/lib/jellyfin-ffmpeg/ffmpeg -init_hw_device vaapi=va:,driver=iHD,kernel_driver=i915 \
                -init_hw_device qsv=qs@va -filter_hw_device qs -hwaccel qsv -hwaccel_output_format qsv \
                -i $file -map 0:v -map 0:a -map 0:s -map_chapters 0 -c:v hevc_qsv -profile main \
                -preset slow -cq 26 -pix_fmt yuv420p -c:a copy -c:s copy -ss $startTime -to $endTime "$file-chapter-$begin-$end.mkv"
        done
done

[bitmap]

So does the awk command not work or does the crop video filter not work?