Login

LordRatner

Hi there. I just wrote up a guide for getting Jellyfin to work on Proxmox in an LXC with GPU transcoding. You can see the whole thread here, in case this thread is missing the answers you seek: https://forum.proxmox.com/threads/jellyf...ge.138873/

Sorry for the formatting, I couldn't figure out if this forum has inline code functionality. It looks cleaner on the ProxMox forum.

The Overview:

We're looking for an unprivileged LXC to serve as a Jellyfin server. We want to use an Nvidia GPU (in this case a GTX 1070Ti) that is also used by other LXCs for other services. The sharing of the GPU is why we are not considering using VMs. Additionally, this setup uses an NFS share for the video library, which has a couple other requirements.

Phase one: Installing Jellyfin

We'll start by creating a Jellyfin LXC using on of the TTECK scripts available here: https://tteck.github.io/Proxmox/
- Follow the instructions using the host shell in the Proxmox GUI
- I recommend using the Advanced options, so you know what you're getting. I use:
  - 2 or 4 cores
  - 4096M RAM (two gb should suffice)
  - 16G Disk (this will need to be much larger if you plan on many simultaneous transcodes until Jellyfin 10.9 comes out with live transcode clearing)
  - Debian 12 (Ubuntu is fine if that's your preference)
  - Static IP address
  - IP6 Disabled
  - Root SSH enabled - You'll need this if you are rebuilding Jellyfin and want to save all your previous data and settings.
Once created, open the console and stop the jellyfin service
Code:
systemctl stop jellyfin
(Optional - Transfer old settings) If you are transferring from a previous install, you need to transfer the
Code:
/var/lib/jellyfin
directory (ignoring the transcodes folder) from the old server to the new one.
- WinSCP works well for this, if you remembered to enable root SSH access. If you forgot, open
  Code:
  /etc/ssh/sshd_config
  and change the
  Code:
  PermitRootLogin
  option to 'yes' and uncomment it if applicable.
  Code:
  systemctl restart sshd
  to make it active.
- Transfer the directories
- In the new Jellyfin LXC, navigate to
  Code:
  /var/lib/jellyfin
  and fix the ownership with
  chown -R jellyfin:jellyfin *
(Optional - Shared Media folder) Create the mount folder where your shared media library is accessed on the Jellyfin LXC. In this example:
Code:
mkdir /mnt/theater
Shutdown the LXC
(Optional - Shared Media folder) We need to give the LXC access to your media library. I'm using NFS to do this, so you'll need to make sure whatever server is hosting your media is set up to accept NFS sharing. We are going to set up the NFS client access on the proxmox host, not the LXC. This will make migrating and snapshotting a bit easier. You can use CIFS, but you'll have to make sure the permissions and user/group settings allow for read and write access.
- On all of your proxmox nodes, create a folder to mount to. In this example we are going to use
  Code:
  /mnt/lxc-share
  . It must be the same on all nodes.
- Mount the folder to your shared media using the method of your choosing. Test to make sure your files are visible, and that you can add and edit files from the proxmox host. In my case NFS via autofs creates
  Code:
  /mnt/lxc-share/theater
- In the proxmox host, open the LXC configuration file (using LXC 140 in this example)
  Code:
  nano /etc/pve/lxc/140.conf
- Mount the media folder into the LXC using this line at the end of your configuration:
  Code:
  lxc.mount.entry: /mnt/lxc-share/theater mnt/theater none bind 0 0
  - We are using this mounting method because it allows for LXC snapshots.
  - By mounting the NFS share on the Proxmox host instead of the actual LXC, you have fewer share clients to create and setup. One connection per node, then as many LXCs as you want can access the share using the configuration entry.
Start the LXC and confirm.
- In the LXC console, navigate to your mounted media drive (Example:
  Code:
  /mnt/theater
  ) and confirm that you can see your media and add/edit files.
- Code:
  systemctl start jellyfin
  Then check to see you are able to navigate to and access jellyfin. If this is a fresh install, you can start setting everything up through Jellyfin, or wait till after we've added the GPU. If this is a transfer, you should be able to log in using your old credentials, but it will ask you to set the admin password. Note, LDAP settings will transfer and work successfully.
Take a snapshot of the LXC. In this example we are calling the snapshot "post-install."
Shutdown the LXC
Did you take a snapshot?
Seriously, make sure you took a snapshot.

Phase Two: Adding the GPU.

This part is a pain. The problem is that Jellyfin requires two libraries, libnvcuvid1 and libnvidia-encode1, to work. You can read the documentation here: https://jellyfin.org/docs/general/admini...nux-setups

But installing these libraries forces whatever Nvidia drivers your current repositories consider current. This is a problem because the drivers you use on your host must match the drivers you use in the LXC. You have to install the GPU on the host first to be able to install it in the LXC, but you might not end up with the same drivers, so you end up having to go through this process. Do not start doing this yet, just read the bullets below to get an idea of the process:

Install drivers on the host. There is a whole world of how-tos explaining how to install an Nvidia GPU on Proxmox. Get to the point where you can run
Code:
nvidia-smi
and see your card listed. I strongly recommend using this process for selecting the drivers: https://github.com/keylase/nvidia-patch#...step-guide. Just the drivers for now, we will address the patch later. Pick some relatively newer drivers.
Make sure you snapshotted your Jellyfin LXC! Install the two libraries on the LXC following the Jellyfin Instructions: https://jellyfin.org/docs/general/admini...nux-setups This will download the drivers identified as required packages, and you will be able to see what drivers you need to install on the host. Make note of the driver version.
Rollback the LXC to the snapshot, removing the drivers and packages.
Back on the host, unsinstall the previous drivers
Code:
./NVIDIA-Linux-x86_64-430.50.run --uninstall
(edit for the version you have) and go through the directions for downloading and installing the needed version.
Repeat the steps from the jellyfin instructions to install the two libraries.
Test to make sure the card is transcoding.

Got it? It sucks, I know, but if you can figure out how to install the two required libraries using the Nvidia drivers of your choice, instead of the repository drivers, please let me know.

Here we go:

Install the GPU on your Proxmox host(s). I won't cover this process, you can find the steps online. Use the driver download, not the repository, found here: https://github.com/keylase/nvidia-patch#...step-guide . You will need to disable nuveau, and all sorts of other obnoxious steps. Google "proxmox lxc plex transcode" and you should find guides, just do the host steps, not the LXC steps.
Shut down the LXC.
On the host,
Code:
ls -l /dev/nvidia*
. We have to pass these to the LXC. Take note of the two numbers separated by a comma, after the group and before the date. You can use both numbers, or just the first number with an asterisk. You should see the following lines, and we are going to pass them all:
- /dev/nvidia0
- /dev/nvidiactl
- /dev/nvidia-modset
- /dev/nvidia-uvm
- /dev/nvidia-uvm-tools
- /dev/nvidia-caps/nvidia-cap1
- /dev/nvidia-caps/nvidia-cap12
To pass these to the LXC you need the following lines in the configuration file
Code:
/etc/pve/lxc/140.conf
. Note that you need to replace the bold/underlined numbers with the ones from your system:
- lxc.cgroup2.devices.allow: c 195:* rwm
- lxc.cgroup2.devices.allow: c 234:* rwm
- lxc.cgroup2.devices.allow: c 235:* rwm
- lxc.cgroup2.devices.allow: c 238:* rwm
- lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
- lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
- lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
- lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file
- lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file
- lxc.mount.entry: /dev/nvidia-caps/nvidia-cap1 dev/nvidia-caps/nvidia-cap1 none bind,optional,create=file
- lxc.mount.entry: /dev/nvidia-caps/nvidia-cap2 dev/nvidia-caps/nvidia-cap2 none bind,optional,create=file
Take a snapshot!! "gpu-mounted" or something like that.
Start the LXC. Install the proprietary drivers as described in the Jellyfin instructions here: https://jellyfin.org/docs/general/admini...nux-setups
- https://wiki.debian.org/NvidiaGraphicsDrivers and go to the LXC distribution. I'm using debian 12
- Add "contrib", "non-free" and "non-free-firmware" components to
  Code:
  /etc/apt/sources.list
- Code:
  apt update
- Code:
  apt install firmware-misc-nonfree
  NOTE: I removed nvidia-driver from the instructions. We will let the next step take care of this.
- Back to the jellyfin instructions https://jellyfin.org/docs/general/admini...nux-setups
- Code:
  apt install -y jellyfin-ffmpeg5
  (Should already be installed)
- Code:
  apt install -y libnvcuvid1 libnvidia-encode1
  This is where you'll see drivers installed, take note of the version. Mine was 525.147.05-4.
Now roll back to the "gpu-mounted" snapshot once you noted the correct driver version.
On the host, uninstall the drivers from step one using the same command you used to install them, but add the
Code:
--uninstall
flag.
Download the correct drivers from the site in step one, and install them. Make sure the GPU registers when you run
Code:
nvidia-smi
On the LXC, repeat step 6 again.
Reboot the LXC
You'll be able to run
Code:
nvidia-smi
from the LXC, though you won't see any processes (these only show up on the host).

At this point you should be able to transcode something using the GPU. Set up Nvidia transcoding in the Jellyfin settings, start a movie, and change the resolution to something in the 480 range just force a transcode. Now, on the host, run

Code:
nvidia-smi

and you should see the transcoding process listed. It'll have "jellyfin-ffmpeg" in the process name. Open up a second window and start another movie simultaneously, drop the resolution, and run

Code:
nvidia-smi

on the host. You should now see two processes indicating the second transcode is working.

The final step: Unlimited encoding streams

Nvidia has a cap on their consumer cards. This can be easily removed using this repo: https://github.com/keylase/nvidia-patch

This needs to be done on both the host and the LXC

Go to
Code:
/opt/nvidia
which might already exist
Code:
wget https://raw.githubusercontent.com/keylase/nvidia-patch/master/patch.sh
Code:
bash ./patch.sh

Now you should be able to open many instances of jellyfin and have them all transcoding at the same time. Running nvidia-smi on the host will show you how many transcoding streams are actually on the GPU. If you can get to 6 you know you applied the patch correctly.

Good luck!

jellyshield · 2024-01-11, 12:51 AM

It didn't work for me.

You can see similar issues with others : https://forum.jellyfin.org/t-hardware-en...5#pid13315

For me it used to work with just installing the latest driver from NVIDIA in both host and lxc. I didn't even had to install libnvcuvid1 libnvidia-encode1 separately. After some kernel update jellyfin-ffmpeg5/6 is broken. I did what was mentioned and downgraded the driver to match the Debian version. I was also able to install the libnvcuvid1 libnvidia-encode1 in lxc and successfully ran nvidia-smi in both host and lxc. Still I have the same error.

Both the libraries are installed

Code:
root@jellyfin:~# apt install -y libnvcuvid1 libnvidia-encode1

Reading package lists... Done

Building dependency tree... Done

Reading state information... Done

libnvcuvid1 is already the newest version (525.147.05-4~deb12u1).

libnvidia-encode1 is already the newest version (525.147.05-4~deb12u1).

0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

NVIDIA-SMI

Code:
root@jellyfin:~# nvidia-smi

Thu Jan 11 00:43:19 2024

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 525.147.05  Driver Version: 525.147.05  CUDA Version: 12.0    |

|-------------------------------+----------------------+----------------------+

| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |

| Fan  Temp  Perf  Pwr:Usage/Cap|        Memory-Usage | GPU-Util  Compute M. |

|                              |                      |              MIG M. |

|===============================+======================+======================|

|  0  Quadro M2000        On  | 00000000:21:00.0 Off |                  N/A |

| 56%  28C    P8    6W /  75W |      1MiB /  4096MiB |      0%      Default |

|                              |                      |                  N/A |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes:                                                                  |

|  GPU  GI  CI        PID  Type  Process name                  GPU Memory |

|        ID  ID                                                  Usage      |

|=============================================================================|

|  No running processes found                                                |

+-----------------------------------------------------------------------------+

Still get the same error in Jellyfin during transcoding

Code:
ffmpeg version 5.1.4-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers

  built with gcc 12 (Debian 12.2.0-14)

  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc

  libavutil      57. 28.100 / 57. 28.100

  libavcodec    59. 37.100 / 59. 37.100

  libavformat    59. 27.100 / 59. 27.100

  libavdevice    59.  7.100 / 59.  7.100

  libavfilter    8. 44.100 /  8. 44.100

  libswscale      6.  7.100 /  6.  7.100

  libswresample  4.  7.100 /  4.  7.100

  libpostproc    56.  6.100 / 56.  6.100

[AVHWDeviceContext @ 0x562bf18b8d00] cu->cuInit(0) failed -> CUDA_ERROR_UNKNOWN: unknown error

Device creation failed: -542398533.

Failed to set value 'cuda=cu:0' for option 'init_hw_device': Generic error in an external library

Error parsing global options: Generic error in an external library

Login

Username/Email:
Password:


Or login with a social network below

Login

Username/Email:
Password:


Or login with a social network below

Proxmox LXC with Nvidia Transcoding and Network Share