Web UI becomes unresponsive when updating libraries

[atcommander]

[*]Running Jellyfin 10.8.10

[*]Installed using ppa on Ubuntu 20.04 on VM with 4 vcpus and 16 GB RAM

I have several large libraries with 1000s of items each and two of them have many items added every day. Also there are another 11 libraries that largely stay the same.  Whenever I either manually trigger a library to look for new or updated files or the automatic updates which are twice a day, the web ui bogs down and at times becomes completely unresponsive.  Also at times update itself comes to a complete stop and I have to restart the jellyfin server before I can do anything.

[TheDreadPirate]

What does 'top' and 'iotop' look like during these updates. Memory usage? Is the library storage directly attached to the hypervisor? NAS?

[atcommander]

top

top - 01:00:31 up 3 days, 19:43, 1 user, load average: 1.53, 1.69, 1.44
Tasks: 181 total, 1 running, 180 sleeping, 0 stopped, 0 zombie
%Cpu(s): 9.0 us, 7.5 sy, 0.0 ni, 83.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 15997.2 total, 295.3 free, 2318.4 used, 13383.5 buff/cache
MiB Swap: 4096.0 total, 4095.0 free, 1.0 used. 13338.7 avail Mem

iotop shows 7 to 11 M/s writes

The library is stored on my storage server and shared using an NFS share to the VM. There is a direct 10 GB fiber connection between the storage server and hypervisor.

[TheDreadPirate]

Where is the Jellyfin database located? Local to the VM? What is the file system for the volume that the database is located on?

[atcommander]

The database is on the vm and I believe it is on the root drive which is formatted XFS. I added a second drive at /var/lib/jellyfin/metadata which formatted ext4 as I ran out of space for metadata.

[TheDreadPirate]

I can't say with 100% certainty that this is the problem. But I am pretty sure that XFS is a CoW (copy-on-write) file system. And it is strongly discouraged that you run any kind of database on a CoW filesystem.

Backup /var/lib/jellyfin and then create another volume, using an ext4 fs, and mount /var/lib/jellyfin to that volume.

[atcommander]

So I moved /var/lib/jellyfin to a drive that was formatted ext4 and then ran a scan all libraries and just like before the webui became unresponsive

[Venson]

Hey @atcommander

Jellyfin runs off an SqLite database. That brings some limitations in performance when you hit a certain ceiling of items with the way jellyfin is coded.

[TheDreadPirate]

Hey @atcommander

Jellyfin runs off an SqLite database. That brings some limitations in performance when you hit a certain ceiling of items with the way jellyfin is coded.

Is the jellyfin DB, in /var/lib/jellyfin/, running on a SSD or HDD?

[atcommander]

It is on an SSD. I had a feeling that it was likely a performance limitation of sqlite but just wanted to make sure there wasn't something else that I was missing.