Transcoding intermittent issue

[Hemz]

I've been having this issue since I got transcoding working.
It works great for anywhere from 1-5 days but then requires a re-boot before it transcodes anymore. 
My users tell me that usually working h264 content gives them a playback error or source error and when I check the FFMPEG logs it says this CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected. 
I restart the Jellyfin yaml and woolah it's all functional again until it's not.

I've read on other threads that something like this will work in the grub config on proxmox. 

This line has been added/edited in my /etc/default/grub file:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash systemd.unified_cgroup_hierarchy=0"

I haven't changed that yet and mine currently says this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafbff,efifbff"

Can anyone that knows what is going throw their 2 cents in.

My hardware is an intel cpu and nvidia p2000 gpu dedicated purely for transcoding and has nothing connected to the outputs of it. 

Cheers legends

[TheDreadPirate]

Does this issue sound like yours? Some people in the comments had some success changing their setup a bit.

https://github.com/jellyfin/jellyfin/issues/9702

Let us know if any of the suggestions in the git issue work for you.

[Hemz]

I have changed the grub cmd line to start with and it seems to have worked since I made the post until now.....will try your reply if that doesn't work.

[Hemz]

I've still been having issues with this unfortunately.

It's intermittent as per the other symptoms and it is only fixed when I restart the server.

Anyone have an answer?

Jellyfin log throwing this line

cu->cuInit(0) failed -> CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

When I restart it works as per normal!

So frustrating.

[TheDreadPirate]

IMO, this doesn't sound like a software problem. Perhaps its a physical problem. Try re-seating the GPU in its slot. Many frustrating problems I had were resolved by reseating a CPU, RAM stick, GPU, power connectors.

[unicron_au]

Sorry this is such a late reply but did you ever resolve this?
I'm seeing similar behaviour in my Jellyfin LXC on a Proxmox host with GPU passthrough. GPU (nVidia GTX1* something) works great for transcoding for a while, the next day I try to play something and i get a failure message. Direct play content with no transcoding is working fine.

If i reboot the Jellyfin server, transcoding works again.

I am thinking of setting a nightly reboot schedule (with Cron maybe? or can Proxmox trigger this?) but that's not really "fixing" the problem.

How do i check the ffmpeg logs?

[TheDreadPirate]

You'd probably have more luck searching through your system logs for the host instead of Jellyfin's ffmpeg logs. Jellyfin would likely only say that permission to access the GPU was denied or that there is no valid Nvidia GPU available. Not necessarily why.

Does Nvidia have its own service or something? It sounds like a race condition where something Nvidia related isn't available before your LXC starts.

[Mel_Gibson_Real]

I had similar issues with my old P400, it would randomly become unavailable. This was on VMs and LXCs with jellyfin installed, and through a reinstall of proxmox and multiple updates. Im sure its possibly a driver or hardware issue with older nvidia cards.

[Obijc]

I have the same issue with a Jellyfin installation on Docker using an Nvidia card. I have to restart the container every night, otherwise I start to experience transcoding issues after a few days. I’m almost sure the problem comes from the Nvidia drivers on Linux, as I don’t remember having this issue with the same card on Jellyfin with Windows. On top of that, I also have the same problem with Ollama, another Docker container that uses the same graphics card on the same docker env.

[kuckenberg]

Hey there,

its been a while since you posted, just want to let others know I am having the same issue. Proxmox with GPU (1650 ti) passthrough into vm into jellyfin docker container.
I cant find any disconnects or errors, neither on VM level nor on Proxmox OS level, but the container at some point loses connection to that GPU.

I am solving it with a small bash script that looks at the log files in my log dir, and if there is this message it restarts the container, solving this issue for me.
I use a txt file to store the info on which logs have been scanned.

Code:

#!/bin/bash
# Directory containing your transcode logs
LOG_DIR="/home/user/jellyfin/config/log"
# File that records the processed log files
PROCESSED_LIST="/home/user/jellyfin/config/log/processedlist.txt"
# Docker container name
CONTAINER="jellyfin"

# Ensure the processed list file exists
touch "$PROCESSED_LIST"

# Flag to indicate if any file encountered the error
ERROR_FOUND=0

# Loop over all .log files in the directory
for logfile in "$LOG_DIR"/*.log; do
    # Check if this file has already been processed
    if ! grep -Fxq "$logfile" "$PROCESSED_LIST"; then
        # Search for the specific error message in the file
        if grep -q "CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected" "$logfile"; then
            echo "Error detected in $logfile"
            ((ERROR_FOUND++))
        fi
        # Add the log file's name to the processed list to avoid rechecking in future runs
        echo "$logfile" >> "$PROCESSED_LIST"
    fi
done

# If any file had the error, restart the container once
if [ "$ERROR_FOUND" -gt 0 ]; then
    echo "Restarting container $CONTAINER due to detected error."
    docker restart "$CONTAINER"
fi

Please note that you have to initially add the existing logs to that file processedlist.txt, otherwise you get a restart for existing errors in log files (which doesnt hurt but...)

just schedule this as cron every few minutes