SQLite Database Corruption in Jellyfin 10.10.6 - Sudden Failure Without System Change

[gvns]

Environment

• Unraid Version: 7.0.1
  
• Jellyfin Version: 10.10.6
  
• Container Source: bbergle-jellyfin container template from https://github.com/BBergle/unraid-templa...llyfin.xml
  
• Tailscale 1.80.2
  
  

Issue Description
My Jellyfin instance has suddenly stopped working - specifically, I cannot get playback to function at all. The system was working normally last night, but today it's failing with database corruption errors. What's particularly concerning is that this happened without any:

• Power outages
  
• Container restarts
  
• System updates
  
• Manual intervention
  
• lack of disk space: (I have plenty)- I have ~11TB free on the share Jellyfin is using, and ~1TB free on its cache drive
  

Error Messages
The primary error I'm seeing is:
[ERR] Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 11: 'database disk image is malformed'.

This appears to be a SQLite database corruption issue. Interestingly, I've recently encountered similar SQLite database corruption in other "arr" applications in my stack, suggesting there might be a common underlying cause. In all the other cases, I simply restored from a .bak of the .db though the fact that this is reoccurring across several different containers is concerning. 

Full log from today:  https://forum.jellyfin.org/attachment.php?aid=6979

Troubleshooting Attempted
So far I've identified the error in logs but haven't attempted recovery procedures as I wanted to get community input first.
I've found some potential recovery commands from this Stack Overflow post that suggest running:

 to check integrity:

Code:

sqlite3 library.db "PRAGMA integrity_check;"

 
and I get the output "ok" - which does not tell me much.


or to recover (have not tried this yet)

Code:

sqlite3 broken.db ".recover" | sqlite3 new.db

However, I'm hesitant to try these without guidance as they may just be band-aid solutions if there's a more severe underlying issue causing database corruption across multiple applications.

Questions

1. Has anyone else experienced spontaneous SQLite corruption in Jellyfin recently?
   
2. What's the recommended recovery approach? (Database restore, repair attempt, etc.)
   
3. Given that this has happened across multiple applications using SQLite, should I be concerned about underlying storage issues? I have not noted anything else that would indicate a storage problem (regular parity checks succeed)
   
4. Are there any known issues with SQLite databases in the bbergle-jellyfin container specifically?
   

Additional Information
- I am using an SSD dedicated for my container appdata - this does make me wonder if that is suspect. The SMART Info does not seem to imply it is at fault.

Thank you!

[theguymadmax]

Post the full server log.

[gvns]

Whoops, I did not attach that log as I thought. https://forum.jellyfin.org/attachment.php?aid=6979

[TheDreadPirate]

Code:

MediaBrowser.Common.FfmpegException: FFmpeg exited with code 254

Can you share the ffmpeg log? Also, what are your hardware specs and did you pass in the GPU into the container?

[gvns]

Of course.

In terms of hardware, I do not have a discrete GPU. Instead I am using the iGPU ( Intel UHD Graphics 730 ) from the Intel i5-12400 CPU I have here. I pass that iGPU device into the container using /dev/dri for iGPU transcoding. more detail below.

# Unraid Server Hardware Summary

## CPU
- Processor: Intel Core i5-12400 (12th Generation)
- Cores: 6 physical cores, 12 threads total (via Hyper-Threading)
- Base Frequency: ~800 MHz (idle frequency, boosts higher under load)
- Cache: 18MB (18,432 KB)
- Microarchitecture: Alder Lake (model 151)

## Motherboard
- Manufacturer: ASUSTeK COMPUTER INC.
- Model: ROG STRIX B760-I GAMING WIFI
- Chipset: Intel B760

## Memory
- Total RAM: 32GB (2 x 16GB)
- Type: DDR5
- Speed: 4800 MT/s
- Manufacturer: Lexar
- Configuration: 2 DIMMs populated out of 2 total slots
- Maximum Capacity: 128GB supported

## Graphics
- Integrated GPU: Intel UHD Graphics 730 (Alder Lake-S GT1)

## Network
- Onboard Ethernet: Intel I226 LAN (disabled according to baseboard info)
- Network interfaces were detected in ip addr output

## System Type
- OS: Unraid OS
- Docker Support: Yes (Docker service is running)

Here is a zip of all my logs:
  jellyfin logs.zip (Size: 412.59 KB / Downloads: 31)

A snippet from them, what I think threw that log error:

Code:

ffmpeg version 7.0.2-Jellyfin Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto=auto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libxml2 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libharfbuzz --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      59.  8.100 / 59.  8.100
  libavcodec    61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample  5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
libva info: VA-API version 1.22.0
libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_22
libva info: va_openDriver() returns 0
[in#0 @ 0x557722d84040] Error opening input: No such file or directory
Error opening input file file:/data/torrents/tv/Breaking Bad [jlw]/Breaking Bad S01 [jlw]/Breaking Bad - 1x01 - Pilot.mkv.
Error opening input files: No such file or directory

That was me attempting to test playback, so may not be super useful for troubleshooting purposes. 

Since this initial post, I ran a scan of all media libraries and now have restored playback functionality. I noted the watch state of some items seemed to be reset.

My bigger concern here is having another corruption of the database. Are there some best practices or ideal backup processes to use? Though before that understanding why this occurred is my focus. That may not be possible though given the limited logs at hand.

for context, lscpu and lspci output:
lspcu:

Code:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:          39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  12
  On-line CPU(s) list:    0-11
Vendor ID:                GenuineIntel
  BIOS Vendor ID:        Intel(R) Corporation
  Model name:            12th Gen Intel(R) Core(TM) i5-12400
    BIOS Model name:      12th Gen Intel(R) Core(TM) i5-12400 To Be Filled By O.E.M. CPU @ 3.9GHz
    BIOS CPU family:      205
    CPU family:          6
    Model:                151
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):            1
    Stepping:            5
    CPU(s) scaling MHz:  18%
    CPU max MHz:          4400.0000
    CPU min MHz:          800.0000
    BogoMIPS:            4992.00
    Flags:                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts
                          acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art
                          arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_know
                          n_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdc
                          m sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand la
                          hf_lm abm 3dnowprefetch cpuid_fault epb ssbd ibrs ibpb stibp ibrs_enhanced tpr_shad
                          ow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpci
                          d rdseed adx smap clflushopt clwb intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves sp
                          lit_lock_detect avx_vnni dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_
                          epp hwp_pkg_req hfi vnmi umip pku ospke waitpkg gfni vaes vpclmulqdq rdpid movdiri
                          movdir64b fsrm md_clear serialize arch_lbr ibt flush_l1d arch_capabilities
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                    288 KiB (6 instances)
  L1i:                    192 KiB (6 instances)
  L2:                    7.5 MiB (6 instances)
  L3:                    18 MiB (1 instance)
NUMA:
  NUMA node(s):          1
  NUMA node0 CPU(s):      0-11
Vulnerabilities:
  Gather data sampling:  Not affected
  Itlb multihit:          Not affected
  L1tf:                  Not affected
  Mds:                    Not affected
  Meltdown:              Not affected
  Mmio stale data:        Not affected
  Reg file data sampling: Not affected
  Retbleed:              Not affected
  Spec rstack overflow:  Not affected
  Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS S
                          W sequence; BHI BHI_DIS_S
  Srbds:                  Not affected
  Tsx async abort:        Not affected

lspci

Code:

00:00.0 Host bridge: Intel Corporation 12th Gen Core Processor Host Bridge (rev 05)
00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05)
00:02.0 VGA compatible controller: Intel Corporation Alder Lake-S GT1 [UHD Graphics 730] (rev 0c)
00:06.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x4 Controller #0 (rev 05)
00:0a.0 Signal processing controller: Intel Corporation Platform Monitoring Technology (rev 01)
00:0e.0 RAID bus controller: Intel Corporation Volume Management Device NVMe RAID Controller
00:14.0 USB controller: Intel Corporation Raptor Lake USB 3.2 Gen 2x2 (20 Gb/s) XHCI Host Controller (rev 11)
00:14.2 RAM memory: Intel Corporation Raptor Lake-S PCH Shared SRAM (rev 11)
00:15.0 Serial bus controller: Intel Corporation Raptor Lake Serial IO I2C Host Controller #0 (rev 11)
00:15.1 Serial bus controller: Intel Corporation Raptor Lake Serial IO I2C Host Controller #1 (rev 11)
00:15.2 Serial bus controller: Intel Corporation Raptor Lake Serial IO I2C Host Controller #2 (rev 11)
00:16.0 Communication controller: Intel Corporation Raptor Lake CSME HECI #1 (rev 11)
00:17.0 SATA controller: Intel Corporation Raptor Lake SATA AHCI Controller (rev 11)
00:1a.0 PCI bridge: Intel Corporation Raptor Lake PCI Express Root Port #25 (rev 11)
00:1c.0 PCI bridge: Intel Corporation Raptor Lake PCI Express Root Port #1 (rev 11)
00:1c.2 PCI bridge: Intel Corporation Raptor Point-S PCH - PCI Express Root Port 3 (rev 11)
00:1d.0 PCI bridge: Intel Corporation Device 7a36 (rev 11)
00:1f.0 ISA bridge: Intel Corporation Device 7a06 (rev 11)
00:1f.4 SMBus: Intel Corporation Raptor Lake-S PCH SMBus Controller (rev 11)
00:1f.5 Serial bus controller: Intel Corporation Raptor Lake SPI (flash) Controller (rev 11)
01:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller
02:00.0 Non-Volatile memory controller: Seagate Technology PLC FireCuda 520/IronWolf 525 SSD (rev 01)
05:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 06)

[TheDreadPirate]

Did you relocate your media storage? That is the only explanation for why scanning your libraries resolved the issue.

Code:

Error opening input files: No such file or directory

Backing up a docker install of Jellyfin is pretty simple. Stop the container, make a backup of the directory that contains /config and /cache. /cache is optional, but I've found that it makes the initial startup after restoration much smoother.

[gvns]

Did you relocate your media storage?  That is the only explanation for why scanning your libraries resolved the issue.

No, my media path has remained the same.

Code:

Error opening input files: No such file or directory

Backing up a docker install of Jellyfin is pretty simple.  Stop the container, make a backup of the directory that contains /config and /cache.  /cache is optional, but I've found that it makes the initial startup after restoration much smoother.

Good to know. I found a few different scripts to do this, this one in particular seems decent:  laughingman77/Jellyfin-Backup-Restore: Scripts to backup and restore Jellyfin data. I will probably fork that and improve it a bit for my use case and add it to my regular backup script schedule.

Considering I am back up and running I am going to call this "solved", though I have a gut instinct that I will probably run into some corrupt sqlite db stuff again. There are quite a few topics around this here/reddit/stackoverflow that do not seem to have much in the way of a root cause and instead just focus on recovery. I am not savvy about sqlite db maintenance, though from what I do know I have been using it properly (allowing it to shut down gracefully, etc)

Thanks to the both of you for taking the time!