Use custom compiled FFmpeg

[Hoof]

Hello fellow Jelly-people,
So I recently purchased a HDHomeRun Flex for the sole purpose of getting OTA ATSC 3.0 stations. In my area all my local major stations are now using this and the signal, for whatever reason, is way better than the ATSC 1.0 stations. Little did I know when I purchased this, although I have beautiful video, I wouldn't have sound. This as I understand it is because FFmpeg does not currently officially support AC4 audio and there is some difficulty with Dolby to get approval for adding support. 

There is a fork of the FFmpeg project (https://github.com/sub3/FFmpeg/tree/ac4) that claims to have a working AC4 patch, so I though I would just pull this repo and compile a custom ffmpeg.

I did this then copied the newly compiled  ffmpeg and ffprobe files to a new path on my Jellyfin docker container. I then went into Jellyfin and changed the ffmpeg path. After clicking save I get the following error message:



Looking at the log I see this:

Code:

[11:03:33] [INF] [414] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Attempting to update encoder path to /customffmpeg/ffmpeg. pathType: Custom
[11:03:33] [ERR] [414] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: FFmpeg validation: The process returned no result
[11:03:33] [ERR] [414] Jellyfin.Server.Middleware.ExceptionMiddleware: Error processing request. URL POST /System/MediaEncoder/Path.
MediaBrowser.Common.Extensions.ResourceNotFoundException: Exception of type 'MediaBrowser.Common.Extensions.ResourceNotFoundException' was thrown.

To compile I used an Ubuntu 22.04 server running the following commands:

Code:

sudo apt-get update -qq && sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libgnutls28-dev \
  libmp3lame-dev \
  libsdl2-dev \
  libtool \
  libva-dev \
  libvdpau-dev \
  libvorbis-dev \
  libxcb1-dev \
  libxcb-shm0-dev \
  libxcb-xfixes0-dev \
  meson \
  ninja-build \
  pkg-config \
  texinfo \
  wget \
  yasm \
  zlib1g-dev

git clone -b ac4 --single-branch https://github.com/sub3/FFmpeg.git
cd FFmpeg/
./configure
make

Envoronment:
Jellyfin 10.8.10 Docker x64 - Unraid

Any idea why I am getting this ffmpeg verification error when selecting the custom ffmpeg?

[skribe]

This might be a permissions error. Make sure that jellyfin has read and execute permissions for the new binary.

This being said, it should also be noted that jellyfin's ffmpeg is already a custom fork, and without those patches as well you may be missing functionality.

[Hoof]

Yeah I thought the same thing but I ran a "chmod -R 777 /customffmpeg/" and "chown -R root:root /customffmpeg/" but still getting the same error. Maybe I'll try to pull the patch into the latest fork of jellyfin's repo and compile it.

[skribe]

That's a good idea probably, but I don't think it's related to the issue you're experiencing. As you can use jellyfin with the standard ffmpeg packages as well, if you want. It'll work. You'll just lack functionality like tonemapping.

But give it a try and we can go from there.

[bitmap]

So I wonder if this has to do with the fact that you're not adding your customffmpeg dir to the PATH in your container? I know the instructions I followed for compiling my own ffmpeg always included adding the dir where you compiled the bin to the PATH. Unsure whether that matters for how Jellyfin calls the tool for transcoding. The last part of my custom compile script that I developed and saved for future reference looks like this:

Code:

PATH="$HOME/bin:$PATH" make -j$(nproc) && \
make install && \
hash -r

You might verify that your custom ffmpeg actually works by doing a docker exec into the container and trying to run it just to see the banner. If it doesn't run, there's something wrong with the compile.

[Hoof]

You might verify that your custom ffmpeg actually works by doing a docker exec into the container and trying to run it just to see the banner. If it doesn't run, there's something wrong with the compile.

You nailed it! Apparently I have no idea what I'm doing

When I run ffmpeg from the Ubuntu server where I compiled it it works fine.
I copied the ffmpeg and ffprobe files over to a volume I setup "/customffmpeg/", but when I run it from the docker container console it gives me this error:

Code:

./ffmpeg: error while loading shared libraries: libxcb-shape.so.0: cannot open shared object file: No such file or directory

Which leads me to believe I'm not building it properly to contain all the dependencies.

Is there some bash script I should be running to compile this with all the options that are defaulted for Jellyfin?

I'm probably way over my head here and should just send the HDHomeRun back.

[bitmap]

To make this a little easier to read, I formatted it a bit, but here's what jellyfin-ffmpeg has configured when compiled in the container:

Code:

configuration:
    --extra-libs=-lfftw3f
    --extra-version=Jellyfin
    --disable-doc
    --disable-ffplay
    --disable-ptx-compression
    --disable-static
    --disable-libxcb
    --disable-sdl2
    --disable-xlib
    --enable-lto
    --enable-gpl
    --enable-version3
    --enable-shared
    --enable-gmp
    --enable-gnutls
    --enable-chromaprint
    --enable-libdrm
    --enable-libass
    --enable-libfreetype
    --enable-libfribidi
    --enable-libfontconfig
    --enable-libbluray
    --enable-libmp3lame
    --enable-libopus
    --enable-libtheora
    --enable-libvorbis
    --enable-libopenmpt
    --enable-libdav1d
    --enable-libwebp
    --enable-libvpx
    --enable-libx264
    --enable-libx265
    --enable-libzvbi
    --enable-libzimg
    --enable-libfdk-aac
    --enable-libsvtav1
    --enable-libshaderc
    --enable-libplacebo
    --enable-vulkan
    --enable-opencl
    --enable-vaapi
    --enable-amf
    --enable-libmfx
    --enable-ffnvcodec
    --enable-cuda
    --enable-cuda-llvm
    --enable-cuvid
    --enable-nvdec
    --enable-nvenc

That's a lot. You likely don't need all of it, e.g., if you don't have any NVIDIA hardware you won't use anything cuda*/nv* but it's kinda hard to sort through. Looking at the fork you linked, it's nearly 7000 commits behind the master branch of ffmpeg, which is behind jellyfin-ffmpeg in a number of ways. So...I went on a bit of a hunt and found that there's really only one change they made...a single commit that supposedly offers AC4 support. This is about where I get in over my head.

Best guess? You could try to fork jellyfin-ffmpeg, figure out what exactly happened in this commit, duplicate that work in your fork of jellyfin-ffmpeg, and see if it compiles. The unfortunate part here is that there's not really any documentation on the fork you linked or the jellyfin-ffmpeg fork for how to compile with the correct configuration, nor do I know if the AC4 support requires an additional callout during compile. Only way to know is to try I guess.

From here, all I can do is share the script I used to compile...the last part is the most useful, though...it was sort of custom made for my purposes. This script DOES NOT cover all of the sources above and DOES NOT utilize jellyfin-ffmpeg, but SHOULD be able to adapt to compiling jellyfin-ffmpeg with enough elbow grease. This was adapted from the Ubuntu compilation script, automating several portions, adding in the Netflix VMAF library, and fixing several inaccuracies and issues with their instructions. Wish I could be more helpful, maybe I'll take a stab at compiling a custom version of jellyfin-ffmpeg (I am interested in the idea of setting a VMAF score instead of using CRF as an indicator, though I don't think that's possible with how transcoding works currently).

Good luck!

Code:

mkdir -p ~/ffmpeg_sources ~/ffmpeg_build ~/bin && \
sudo apt-get update -qq && sudo apt-get -y install \
  autoconf \
  automake \
  build-essential \
  cmake \
  git-core \
  libass-dev \
  libfreetype6-dev \
  libgnutls28-dev \
  libmp3lame-dev \
  libsdl2-dev \
  libtool \
  libva-dev \
  libvdpau-dev \
  libvorbis-dev \
  libxcb1-dev \
  libxcb-shm0-dev \
  libxcb-xfixes0-dev \
  meson \
  ninja-build \
  pkg-config \
  texinfo \
  wget \
  yasm \
  zlib1g-dev \
  libunistring-dev \
  libaom-dev \
  libdav1d-dev \
  nasm \
  libx264-dev \
  libx265-dev \
  libnuma-dev \
  libvpx-dev \
  libfdk-aac-dev \
  libopus-dev && \
cd ~/ffmpeg_sources && \
wget https://github.com/Netflix/vmaf/archive/v2.3.1.tar.gz && \
tar xvf v2.3.1.tar.gz && \
mkdir -p vmaf-2.3.1/libvmaf/build &&\
cd vmaf-2.3.1/libvmaf/build && \
meson setup -Denable_tests=false -Denable_docs=false --buildtype=release --default-library=static .. --prefix "$HOME/ffmpeg_build" --bindir="$HOME/ffmpeg_build/bin" --libdir="$HOME/ffmpeg_build/lib" && \
ninja && \
ninja install && \
cd ~/ffmpeg_sources && \
git -C SVT-AV1 pull 2> /dev/null || git clone https://gitlab.com/AOMediaCodec/SVT-AV1.git && \
mkdir -p SVT-AV1/build && \
cd SVT-AV1/build && \
PATH="$HOME/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$HOME/ffmpeg_build" -DCMAKE_BUILD_TYPE=Release -DBUILD_DEC=OFF -DBUILD_SHARED_LIBS=OFF .. && \
PATH="$HOME/bin:$PATH" make -j$(nproc) && \
make install && \
cd ~/ffmpeg_sources && \
wget -O ffmpeg-6.0.tar.bz2 https://ffmpeg.org/releases/ffmpeg-6.0.tar.bz2 && \
tar xjvf ffmpeg-6.0.tar.bz2 && \
cd ffmpeg-6.0 && \
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --ld="g++" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-gnutls \
  --enable-libaom \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libsvtav1 \
  --enable-libdav1d \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-libwebp \
  --enable-libxvid \
  --enable-libvmaf \
  --enable-vaapi \
  --enable-libvpl \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make -j$(nproc) && \
make install && \
hash -r