SOLVED: Library scan keeps failing

[pcm]

Hi, 

I every time I am trying to do my library scans, it keeps failing.

Looking at the logs it seems like it fails because of SQLite DB keeps getting locked... The DB seems to keep getting locked while trying to update/save images ... 

Anyone run into this problem before ? And any idea how to fix this easily ? 

[2024-05-24 05:17:38.285 -04:00] [INF] Starting "/usr/lib/jellyfin-ffmpeg/ffprobe" with args "-analyzeduration 200M -probesize 1G -i file:\"<sanitized>\" -threads 0 -v warning -print_format json -show_streams -show_chapters -show_format"
[2024-05-24 05:17:38.285 -04:00] [INF] Starting "/usr/lib/jellyfin-ffmpeg/ffprobe" with args "-analyzeduration 200M -probesize 1G -i file:\"<sanitized>\" -threads 0 -v warning -print_format json -show_streams -show_chapters -show_format"
[2024-05-24 05:17:42.006 -04:00] [INF] Watching directory "<sanitized>"
[2024-05-24 05:17:42.000 -04:00] [ERR] Error executing Scheduled Task
Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: 'database is locked'.
  at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
  at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
  at Microsoft.Data.Sqlite.SqliteConnectionExtensions.ExecuteNonQuery(SqliteConnection connection, String commandText, SqliteParameter[] parameters)
  at Microsoft.Data.Sqlite.SqliteTransaction..ctor(SqliteConnection connection, IsolationLevel isolationLevel, Boolean deferred)
  at Emby.Server.Implementations.Data.SqliteItemRepository.SaveImages(BaseItem item)
  at Emby.Server.Implementations.Library.LibraryManager.UpdateImagesAsync(BaseItem item, Boolean forceUpdate)
  at Emby.Server.Implementations.Library.LibraryManager.RunMetadataSavers(BaseItem item, ItemUpdateType updateReason)
  at Emby.Server.Implementations.Library.LibraryManager.UpdateItemsAsync(IReadOnlyList1 items, BaseItem parent, ItemUpdateType updateReason, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Video.UpdateToRepositoryAsync(ItemUpdateType updateReason, CancellationToken cancellationToken)
  at MediaBrowser.Providers.Manager.MetadataService2.SaveItemAsync(MetadataResult1 result, ItemUpdateType reason, CancellationToken cancellationToken)
  at MediaBrowser.Providers.Manager.MetadataService2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.RefreshChildMetadata(BaseItem child, MetadataRefreshOptions refreshOptions, Boolean recursive, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.<>c__DisplayClass69_01.<<RunTasks>b__1>d.MoveNext()
--- End of stack trace from previous location ---
  at MediaBrowser.Controller.Entities.Folder.RunTasks[T](Func3 task, IList1 children, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.RefreshChildMetadata(BaseItem child, MetadataRefreshOptions refreshOptions, Boolean recursive, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.<>c__DisplayClass69_01.<<RunTasks>b__1>d.MoveNext()
--- End of stack trace from previous location ---
  at MediaBrowser.Controller.Entities.Folder.RunTasks[T](Func3 task, IList1 children, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.RefreshChildMetadata(BaseItem child, MetadataRefreshOptions refreshOptions, Boolean recursive, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.<>c__DisplayClass69_01.<<RunTasks>b__1>d.MoveNext()
--- End of stack trace from previous location ---
  at MediaBrowser.Controller.Entities.Folder.RunTasks[T](Func3 task, IList1 children, IProgress1 progress, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.ValidateChildrenInternal2(IProgress1 progress, Boolean recursive, Boolean refreshChildMetadata, Boolean allowRemoveRoot, MetadataRefreshOptions refreshOptions, IDirectoryService directoryService, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.Folder.ValidateChildrenInternal(IProgress1 progress, Boolean recursive, Boolean refreshChildMetadata, Boolean allowRemoveRoot, MetadataRefreshOptions refreshOptions, IDirectoryService directoryService, CancellationToken cancellationToken)
  at MediaBrowser.Controller.Entities.AggregateFolder.ValidateChildrenInternal(IProgress1 progress, Boolean recursive, Boolean refreshChildMetadata, Boolean allowRemoveRoot, MetadataRefreshOptions refreshOptions, IDirectoryService directoryService, CancellationToken cancellationToken)
  at Emby.Server.Implementations.Library.LibraryManager.PerformLibraryValidation(IProgress1 progress, CancellationToken cancellationToken)
  at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress1 progress, CancellationToken cancellationToken)
  at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
[2024-05-24 05:17:42.021 -04:00] [INF] "Scan Media Library" Failed after 22 minute(s) and 33 seconds
[2024-05-24 05:17:44.298 -04:00] [INF] Watching directory "<sanitized>"
[2024-05-24 05:18:08.043 -04:00] [INF] Watching directory "<sanitized>"
[2024-05-24 05:18:17.793 -04:00] [INF] Finished creation of trickplay files for "<sanitized>"
[2024-05-24 05:18:17.803 -04:00] [INF] Creating trickplay files at 320 width, for "<sanitized>" [ID: 3860de12-87ed-9b09-651e-b7f1a59c51a4]
[2024-05-24 05:18:17.803 -04:00] [INF] Trickplay generation: "/usr/lib/jellyfin-ffmpeg/ffmpeg -loglevel error -init_hw_device cuda=cu:0 -filter_hw_device cu -hwaccel cuda -hwaccel_output_format cuda -hwaccel_flags +unsafe_output -threads 1 -noautorotate -i file:\"<sanitized>\" -noautoscale -an -sn -vf \"fps=0.10000000149011612,setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_cuda=w=320:h=180:format=yuv420p,hwdownload,format=yuv420p\" -threads 9 -c:v mjpeg -qscale:v 4 -f image2 \"/tmp/jellyram/temp/ad17a37cdf74433aa3b21815df29a441/%08d.jpg\""

[TheDreadPirate]

If you have trickplays configured to use more than 1 thread, change it back to 1.

[pcm]

Thanks for your reply.

I changed trickplay to use only 1 thread and reran the scan. Still ran into the same issue.

I disabled trickplay extraction during library scan. Still ran into the same issue.

I disabled trickplay for the library altogether. Still ran into the same issue.

I restarted the server ( via systemctl ) and reverified just to be sure that config was saved and attempted to rescan library and ran into the same issue.

I attempted a 'refresh metadata' in 'replace all metadata' mode and that job also ran into the same issue.

At this point I am considering deleting the library and readding it...

By any chance do you know where this DB is located on a typical ubuntu 22.04.4 installation ? Just in case I need to delete DB file ... I am using jellyfin 10.9.2 btw

[TheDreadPirate]

/var/lib/jellyfin/data/library.db

[nothingveryobvious]

@pcm @TheDreadPirate

I am having the same issue, except mine is SQLite Error 6. In fact, many people are having this issue on GitHub:
- https://github.com/jellyfin/jellyfin/issues/11834
- https://github.com/jellyfin/jellyfin/issues/11839
- https://github.com/jellyfin/jellyfin/issues/11728
- https://github.com/jellyfin/jellyfin/issues/11589

Any suggestions? I even tried a fresh install of 10.9.3 and I still have this issue.

[lx77]

It might be an issue with files containing special characters like '&', ‘–' (hyphen) and others?
I didn't have an issue with special characters in the past, but now I'm having problems (not sure exactly when it occurred).
Characters like 'ä', 'ö' and 'ü' are working fine.

Folders which don't contain specific special characters (in folder or file name) seem to work fine.
But others seem to stop the library scan.

Update: Found a solution for my problem.
Maybe it helps here too.
https://forum.jellyfin.org/t-solved-issu...nce-update

[pcm]

It might be an issue with files containing special characters like '&', ‘–' (hyphen) and others?
I didn't have an issue with special characters in the past, but now I'm having problems (not sure exactly when it occurred).
Characters like 'ä', 'ö' and 'ü' are working fine.

Folders which don't contain specific special characters (in folder or file name) seem to work fine.
But others seem to stop the library scan.

Update: Found a solution for my problem.
Maybe it helps here too.
https://forum.jellyfin.org/t-solved-issu...nce-update

That's it! It was the '–' in the file names instead of the '-' that was causing this problem. Thanks a lot!

In case anyone else is facing this problem. Here's a quick oneliner linux CLI that I came up with to fix all unicode hyphens to standard (ascii) hyphens in all files and dirs. 

Code:

find . -name "*–*" -print | while IFS= read -r f ; do  mv -v "$f" "`echo \"$f\" | sed 's/–/-/g'`" ; done

[lx77]

That's it! It was the '–' in the file names instead of the '-' that was causing this problem. Thanks a lot!

In case anyone else is facing this problem. Here's a quick oneliner linux CLI that I came up with to fix all unicode hyphens to standard (ascii) hyphens in all files and dirs. 

Code:

find . -name "*–*" -print | while IFS= read -r f ; do  mv -v "$f" "`echo \"$f\" | sed 's/–/-/g'`" ; done

Hi, if those special characters are an issue, you probably have a wrong setting. Because with a correct installation they don't make problems.
You should check if you have a wrong env setting for 10.9.x (DOTNET_SYSTEM_GLOBALIZATION_INVARIANT set to true or 1). And if so, delete it.
https://github.com/jellyfin/jellyfin/iss...2105568202

Don't try to delete this variable with portainer if you're using synology docker (Container Manager), because it might reset the value. Took me a while to realize this. In Container Manager deleting the variable worked. https://github.com/portainer/portainer/issues/5813