GFX1150 - Hardware acceleration transcoding error

[fld]

Hey, I had trouble with jellyfin working on the iGPU as well. Main problem seems to be that this iGPU is too new and most prepackaged drivers and software don't know how to use it.

For me what worked was getting newer LLVM and Mesa drivers (VA-API, Vulkan) as well as DRM libraries from Debian backports. Then i swapped out Jellyfin's pre-packaged AMD VAAPI driver (radeonsi_drv_video.so) with the newer one from Mesa.

Now hardware accel works quite well including AV1 encoding and tone mapping with Vulkan.

I am running jellyfin in docker so you will have to adapt to your own setup.

Here is my Dockerfile for your reference:

Code:

# Fix AMD Radeon 890M (gfx1150) support in Jellyfin container
# Required for VAAPI hardware transcoding and HDR tone mapping on newer AMD iGPUs
# - Install Mesa 25+ drivers from backports (adds gfx1150 support)
# - Install latest LLVM from backports (required for shader compilation)
# - Replace Jellyfin's bundled radeonsi driver with Mesa's newer version

FROM jellyfin/jellyfin:latest

RUN set -ex && \
    # Detect Debian codename and add backports repository
    DEBIAN_VERSION=$(grep VERSION_CODENAME /etc/os-release | cut -d= -f2) && \
    echo "Detected Debian version: $DEBIAN_VERSION" && \
    echo "deb http://deb.debian.org/debian ${DEBIAN_VERSION}-backports main" >> /etc/apt/sources.list.d/backports.list && \
    apt-get update && \
    # Find the newest available LLVM version in backports
    LLVM_PKG=$(apt-cache search --names-only '^libllvm[0-9]+$' | grep -o 'libllvm[0-9]\+' | sort -V | tail -n1) && \
    echo "Found newest LLVM package: $LLVM_PKG" && \
    # Install Mesa drivers and dependencies from backports
    apt-get install -y -t ${DEBIAN_VERSION}-backports \
        mesa-va-drivers \
        mesa-vulkan-drivers \
        libva2 \
        libva-drm2 \
        libdrm2 \
        libdrm-amdgpu1 \
        ${LLVM_PKG} && \
    # Replace Jellyfin's bundled radeonsi driver with Mesa's version
    if [ -f /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so ]; then \
        GALLIUM_LIB=$(find /usr/lib/x86_64-linux-gnu/ -name "libgallium*.so" -type f | sort -V | tail -n1) && \
        if [ -n "$GALLIUM_LIB" ]; then \
            echo "Found gallium library: $GALLIUM_LIB" && \
            mv /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so \
              /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so.bak && \
            cp "$GALLIUM_LIB" /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so && \
            echo "Replaced radeonsi driver with $GALLIUM_LIB"; \
        fi; \
    fi && \
    # Cleanup
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*