Container startup issues with NVidia

[Terence-D]

Hi there, not sure where to post, so thought I'd start here... 

I have Jellyfin running great, along with hw transcoding through my GPU, no issues until I reboot. The container will not restart after reboot until I first run this command by hand, then I can restart the container successfully:

sudo nvidia-container-cli --load-kmods info

I'll be honest I don't know what magic that does, I assume it loads kernel modules? I did check the following file and load-kmods=true is set:

cat /etc/nvidia-container-runtime/config.toml | grep mod
load-kmods = true

I'm looking for some way to either fix whatever this is, OR just run that command on startup somehow.. this might be a better question to ask for the docker or linux guys, but as its Jellyfin I thought I'd start here. Any help is appreciated, thank you.  I've been running like this for awhile and its finally annoyed me enough to start investigating.  I've tried searching on Google to no avail.

Relevant system info

• Host - Ubuntu 22.04
  
• Jellyfin Version: 10.8.13
  
• GPU - nvidia 1060 6gb
  

Dockercompose:

Code:

version: "3"
services:
  jellyfin:
    image: jellyfin/jellyfin
    container_name: jellyfin
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=${TZ}
    runtime: nvidia
    volumes:
      - [deleted]/docker/jellyfin/config:/config
      - [deleted]/jellyfin/cache:/cache
    ports:
        [deleted]
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    devices:
      - /dev/nvidia-caps:/dev/nvidia-caps
      - /dev/nvidia0:/dev/nvidia0
      - /dev/nvidiactl:/dev/nvidiactl
      - /dev/nvidia-modeset:/dev/nvidia-modeset
      - /dev/nvidia-uvm:/dev/nvidia-uvm
      - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools

[TheDreadPirate]

You could a service that systemd runs on startup. As long as the priority is higher than docker it should run before it.

https://askubuntu.com/questions/919054/h...ng-systemd

[Terence-D]

Thanks for the link DreadPirate, that got me on the right path. 

In case anyone ever finds this post in the future, here's the service I created under /etc/systemd/system.  Tested on a reboot everything came up properly:

Code:

[Unit]
Description=Nvidia Kernel Modules
After=network.target local-fs.target
Before=containerd.service docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-container-cli --load-kmods info
TimeoutStopSec=10
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target containerd.service docker.service

Enabled with:

Code:

sudo systemctl enable nvidiamods.service