+- Jellyfin Forum (https://forum.jellyfin.org)
+-- Forum: Support (https://forum.jellyfin.org/f-support)
+--- Forum: Troubleshooting (https://forum.jellyfin.org/f-troubleshooting)
+--- Thread: SOLVED: Changing an image triggers a full library scan (10.9.2) (/t-solved-changing-an-image-triggers-a-full-library-scan-10-9-2)
Pages:
1
2
Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-18
I'm doing a little housekeeping today, and changing a couple movie posters here and there to nicer versions (through the web UI), and I noticed every time I updated an image, it triggered a full movie library rescan, which seems strange and fairly intense behaviour, and one that I haven't noticed happening before the 10.9 update.
Being as I have a large collection and am pretty neurotic about images, I feel like this is likely to happen frequently, and I'm wondering if it's a normal/expected behaviour?
For reference, I'm using the standard docker image on a linux system.
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-18
Another (related) question: Has Jellyfin always triggered a rescan of the whole library when a new item is added? It seems like (as the drives in my NAS are a little loud, so it's pretty obvious that they're working hard) the server is working a *lot* harder when I add new content lately. I just added a single movie to my collection, and the logs show it having scanned a fairly significant number of files in my library as opposed to simply scanning the new file.
Similarly, Kodi is syncing hundreds of updates every time I update a single movie, which definitely wasn't happening before, and I assume is related.
As far as I can tell, everything is up to date (new Kodi addon this morning, new version of sync queue plugin yesterday, etc).
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-18
Continuing to look into this, it seems like it's 699 files each time (in a library of ~2800 movies), and ffprobe is looking at the movie files and the SRT files. I've had a look at the various movie files, and it seems like none of them have modified dates that are particularly recent, though it does appear that many of the subtitle files are fairly new, likely from a scheduled task for retrieving missing subs that ran on 05/15.
I had a look at the logs from before the 10.9 update, and not a single one even mentions ffprobe, where now I have thousands of lines like "[2024-05-18 08:53:01.331 -07:00] [INF] [19] MediaBrowser.MediaEncoding.Encoder.MediaEncoder: Starting "/usr/lib/jellyfin-ffmpeg/ffprobe" with args "-analyzeduration 200M -probesize 1G -i file:\"/data/movies/Seven.Samurai.1954.CRITERION.1080p.BluRay.x264.anoXmous_.mp4\" -threads 0 -v warning -print_format json -show_streams -show_chapters -show_format""
Is this a new feature of 10.9.x, where the logs are more verbose, or is it an indicator of an issue?
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-18
I'm starting to suspect it has SOMETHING to do with my directory structure, but I haven't been able to figure out what logic is causing it to get hung up on the specific files it's getting hung up on. For reference, I've been slowly trying to go from having my files loose in a /data/movies/ folder like "/data/movies/example.mp4" and switch to individual folders for each film, like "/data/movies/example (2024)/example.mp4", so there's currently a mix in my /data/movies folder.
From what I can tell from the log files, this only affects movies which are *loose*, as opposed to in their own individual folder. However, it does not seem to affect ALL movies that are loose in the /data/movies folder. It does not affect anything other than movie files and srt files, no jpegs or nfo files.
I'm not sure what this means. Does anybody else have this issue?
RE: Changing an image triggers a full library scan (10.9.2) - TheDreadPirate - 2024-05-18
I am pretty sure Jellyfin 10.8 also logged every time it called ffmpeg or ffprobe.
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-19
(2024-05-18, 10:44 PM)TheDreadPirate Wrote: I am pretty sure Jellyfin 10.8 also logged every time it called ffmpeg or ffprobe.
That's good to know, but I think it's calling it erroneously, in that case. When I look at the logfiles from my 10.8.9 backup, the word "ffprobe" is not contained within any of them, and I have a scheduled "Scan Media Libraries" job that fires at 12:00am every day. I assume that's maybe because I didn't add any new items during that period. What's happening since the 10.9 update is that ffprobe seems to be scanning several hundred files that are already in the library, which is then generating around 700 updates for Kodi to pointlessly sync each time. These files are the same each time (my library has around 2900 movies, and this only seems to affect around 700 files total, movies as well as subtitles), and are all videos that have been in my library for a very long time. As a result, each media scan takes a lot longer than the roughly one second it was taking before the update, and is definitely causing a great deal of hard drive usage, as well as slowing down the server as these scans are CPU intensive.
Beyond that, is it normal behaviour for the changing of a backdrop or poster image to trigger a full library scan? Did I just not notice the behaviour in previous versions because the scan was quick and uneventful?
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-19
I turned on Debug logging and it shows entries on all these scans that the ffprobe entry was triggered by a change in external subtitles, which is odd because I had a look at the files related to one of these movies and the modified dates on the subtitle files were several months old. Very strange that it's triggering on every scan.
At this point I'm going to consider the point moot, as I can no longer reproduce the issue of image changes triggering a rescan since doing a new docker pull this morning. If it wants to scan a bunch of old movies every time it does a full library scan I think that's fine, as long as really minor housekeeping doesn't continually trigger library rescans at this point.
Thanks for listening. Despite some weird bugs on this release, Jellyfin's great and the support team seems to be working TIRELESSLY and I appreciate the hell out of you guys.
Cheers!
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-20
Following up, I'm still fairly convinced that something's wrong when my Movie library is being scanned.
Today, I added one new album to my music library, and according to the logs, ffprobe checked the specific album I added, and nothing else. I added one new episode of a TV series, and ffprobe called the specific episode, and nothing else. Both scans were so quick as to be imperceptible.
I added a new movie, and got 4555 lines of ffprobe scanning already pre-existing files, none of which have been added recently or even modified recently. It doesn't make sense that Jellyfin would suddenly be endlessly re-scanning the same files.
RE: Changing an image triggers a full library scan (10.9.2) - TheDreadPirate - 2024-05-20
What plugins do you have installed?
RE: Changing an image triggers a full library scan (10.9.2) - kandykarter - 2024-05-20
The defaults, plus opensubtitles and kodi sync queue.
They're telling me on the issue tracker on github that this is likely because my films aren't in individual folders, so it sounds like I have an answer. It's still a little weird because it's definitely not triggering ffprobe on ALL the movies in that directory, and it wasn't happening in 10.8, but it looks like this is something I can fix by working on my folder structure.
Thanks for your help regardless!