SOLVED: Database table randomly locking

[magguzu]

Hi folks! Since today my server has become intermittedly unresponsive. When it happens it seems to be related to database locking
Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.

---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 6: 'database table is locked'.
  at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
  at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
  at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
  at Microsoft.Data.Sqlite.SqliteCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
  at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
  at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReaderAsync(RelationalCommandParameterObject parameterObject, CancellationToken cancellationToken)
  at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.ExecuteAsync(IRelationalConnection connection, CancellationToken cancellationToken)
  --- End of inner exception stack trace ---

I'm not entireley sure but this seems related:
https://github.com/jellyfin/jellyfin/pull/11901

In the meantime can anyone recommend temporary fixes? Any plugins to disable, schedule restarts, etc? This wasn't occurring before today.
Running 10.9.7 which I upgraded to from 10.9.6 today where it was also occurring.

[Efficient_Good_5784]

What version were you on before upgrading to v10.9.6?

Starting with v10.9.4, there's a process that's run on the database that takes a while. If you or something causes the server to turn off during that time, you will run into database issues.

Also, are you using containers for Jellyfin? did you copy the config folder while Jellyfin was running?

[magguzu]

Thank you for your response.

> What version were you on before upgrading to v10.9.6?

I can't say for certain, it upgrades with an Ansible playbook periodically so I would guess 10.9.5.

> Also, are you using containers for Jellyfin?

Yes, in Docker compose.

> did you copy the config folder while Jellyfin was running?

I was in the past (up to about 1.5 months ago). I have since stopped after migrating machines but I did copy over my same database that had been copying nightly. Did I perhaps do permanent damage that I'm seeing the effects of now?

[TheDreadPirate]

Is this happening during a library scan? You can try reducing the number of scan threads to 1. Dashboard > General. Change "Parallel library scan tasks limit" and "Parallel image encoding limit" to 1.

Also, is the jellyfin database on a SSD or HDD?

[magguzu]

Thank you! I have set that limit. Now that you mention it, a few days ago I did change library scan frequency from 12 hours to 2 hours. Was this a bad move? I'm running on 6 cores and 8GB of RAM.

The Jellyfin database is on an SSD.

[TheDreadPirate]

It shouldn't matter how frequent the scans are. As long as the scans aren't taking so long that they overlap. I'm pretty sure only one can happen at a time if that were to happen.

If scheduled scans are taking more than two hours, that is a problem.

This is such a hard problem to diagnose. Some people, like myself and a lot of the devs, have never seen this issue in our setups. And I don't set any thread limits for scans.

Some people, with similarly spec'd systems, can't seem to find away to stop it from happening.

If setting the thread limit to 1 doesn't work, we can try enabling debug logging to try to figure out what activity is tying up the database.

[magguzu]

Thank you. For visibility someone in the Discord suggested I set this environment variable too, so I did:

https://github.com/jellyfin/jellyfin/pull/11823

Once the 11901 pull request is merged I'll test removing that EV and undoing the thread limit mentioned here.