SOLVED: Could not apply migration 'MigrateActivityLogDatabase'

[scubbo]

The version of Jellyfin you are running: jellyfin/jellyfin:10.8.10
[*]Installation method and platform: Kubernetes, via Helm
[*]Other context: Jellyfin's /config directory is mounted via NFS - this has previously worked with no problems for months at a time
Jellyfin pod repeatedly crashes with the following logs:

Code:

[04:13:43] [INF] [1] Main: Jellyfin version: 10.8.10
[04:13:43] [INF] [1] Main: Environment Variables: ["[JELLYFIN_METRICS_SERVICE_PORT, 8555]", "[JELLYFIN_WEB_DIR, /jellyfin/jellyfin-web]", "[JELLYFIN_METRICS_PORT_8555_TCP, tcp://10.43.6.145:8555]", "[JELLYFIN_PORT_8096_TCP_ADDR, 10.43.29.217]", "[JELLYFIN_PORT_8096_TCP_PORT, 8096]", "[JELLYFIN_DATA_DIR, /config]", "[JELLYFIN_FFMPEG, /usr/lib/jellyfin-ffmpeg/ffmpeg]", "[JELLYFIN_CONFIG_DIR, /config/config]", "[JELLYFIN_METRICS_PORT, tcp://10.43.6.145:8555]", "[JELLYFIN_METRICS_PORT_8555_TCP_PROTO, tcp]", "[JELLYFIN_SERVICE_HOST, 10.43.29.217]", "[JELLYFIN_PORT, tcp://10.43.29.217:8096]", "[JELLYFIN_PORT_8096_TCP, tcp://10.43.29.217:8096]", "[JELLYFIN_METRICS_SERVICE_HOST, 10.43.6.145]", "[DOTNET_SYSTEM_GLOBALIZATION_INVARIANT, 1]", "[JELLYFIN_METRICS_SERVICE_PORT_METRICS, 8555]", "[JELLYFIN_METRICS_PORT_8555_TCP_PORT, 8555]", "[JELLYFIN_METRICS_PORT_8555_TCP_ADDR, 10.43.6.145]", "[JELLYFIN_CACHE_DIR, /cache]", "[JELLYFIN_PORT_8096_TCP_PROTO, tcp]", "[JELLYFIN_LOG_DIR, /config/log]", "[JELLYFIN_SERVICE_PORT, 8096]"]
[04:13:43] [INF] [1] Main: Arguments: ["/jellyfin/jellyfin.dll"]
[04:13:43] [INF] [1] Main: Operating system: Linux
[04:13:43] [INF] [1] Main: Architecture: Arm64
[04:13:43] [INF] [1] Main: 64-Bit Process: True
[04:13:43] [INF] [1] Main: User Interactive: True
[04:13:43] [INF] [1] Main: Processor count: 4
[04:13:43] [INF] [1] Main: Program data path: /config
[04:13:43] [INF] [1] Main: Web resources path: /jellyfin/jellyfin-web
[04:13:43] [INF] [1] Main: Application directory: /jellyfin/
[04:13:43] [DBG] [1] Jellyfin.Server.Migrations.MigrationRunner: Skipping migration 'CreateNetworkConfiguration' since it is already applied
[04:13:43] [INF] [1] Emby.Server.Implementations.AppBase.BaseConfigurationManager: Setting cache path: /cache
[04:13:43] [INF] [1] Emby.Server.Implementations.ApplicationHost: Loading assemblies
(debug logs of "Creating instances" skipped)
[04:13:53] [DBG] [1] Jellyfin.Server.Migrations.MigrationRunner: Skipping migration 'DisableTranscodingThrottling' since it is already applied
[04:13:53] [DBG] [1] Jellyfin.Server.Migrations.MigrationRunner: Skipping migration 'CreateLoggingConfigHeirarchy' since it is already applied
[04:13:53] [INF] [1] Jellyfin.Server.Migrations.MigrationRunner: Applying migration 'MigrateActivityLogDatabase'
[04:13:53] [ERR] [1] Jellyfin.Server.Migrations.MigrationRunner: Could not apply migration 'MigrateActivityLogDatabase'
CannotOpen: SQLitePCL.pretty.SQLiteException: unable to open database file
  at SQLitePCL.pretty.SQLiteException.Throw(Int32 rc, Int32 extended, String msg)
  at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
  at SQLitePCL.pretty.SQLiteDatabaseConnectionBuilder.Build()
  at Jellyfin.Server.Migrations.Routines.MigrateActivityLogDb.Perform()
  at Jellyfin.Server.Migrations.MigrationRunner.PerformMigrations(IMigrationRoutine[] migrations, MigrationOptions migrationOptions, Action`1 saveConfiguration, ILogger logger)
[04:13:53] [FTL] [1] Main: Error while starting server.
CannotOpen: SQLitePCL.pretty.SQLiteException: unable to open database file
  at SQLitePCL.pretty.SQLiteException.Throw(Int32 rc, Int32 extended, String msg)
  at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
  at SQLitePCL.pretty.SQLiteDatabaseConnectionBuilder.Build()
  at Jellyfin.Server.Migrations.Routines.MigrateActivityLogDb.Perform()
  at Jellyfin.Server.Migrations.MigrationRunner.PerformMigrations(IMigrationRoutine[] migrations, MigrationOptions migrationOptions, Action`1 saveConfiguration, ILogger logger)
  at Jellyfin.Server.Migrations.MigrationRunner.Run(CoreAppHost host, ILoggerFactory loggerFactory)
  at Jellyfin.Server.Program.StartApp(StartupOptions options)
[04:13:53] [INF] [1] Main: Running query planner optimizations in the database... This might take a while
[04:13:53] [INF] [1] Emby.Server.Implementations.ApplicationHost: Disposing CoreAppHost
[04:13:53] [INF] [1] Emby.Server.Implementations.ApplicationHost: Disposing SchedulesDirect
[04:13:53] [INF] [1] Emby.Server.Implementations.ApplicationHost: Disposing MusicBrainzAlbumProvider
[04:13:53] [INF] [1] Emby.Server.Implementations.ApplicationHost: Disposing EmbyTV
[04:13:53] [INF] [2] Main: Received a SIGTERM signal, shutting down

Things I've tried, to no avail:

• chmod-ing jellyfin.db and library.db to 777
  
• moving jellyfin.db and library.db to "...db.BACKUP" so that they will be recreated
  
• setting <IsStartupWizardCompleted> in system.xml to false
  

Links I've followed:

• This GitHub Issue (with no answer) looks like the same problem
  
• This looks like the same Exception, though for a different migration
  
• I might be misinterpreting it, but this file suggests that it might be looking for activitylog.db or users.db rather than one of the existing databases - though, if that were the case, I would have expected those databases to be created when I set <IsStartupWizardCompleted> to false
  

[scubbo]

I tried migrating config to a local PersistentVolume just in case this was something to do with NFS - the same issue reoccurred.

[crobibero]

It sounds like your server either already ran the migration and didn’t save that it did. You could manually edit migrations.xml to add the migration id, but I recommend starting from scratch if possible.

[scubbo]

A start-from-scratch worked - after deleting the config and cache directories and restarting the app, it's back online and stable.