LibraryManager.cs Memory Cache

[TrueCodePoet]

I have been struggling with an issue for over a year now.  
Despite having a server with 32GB of RAM, my server runs out of memory daily, Specifically the Jellyfin service consumes all available ram.  and I've noticed that I'm forced to restart the server service every day just to keep it running. 

After some digging into the source, I suspect the root of the problem lies in how the LibraryManager handles caching.
From what I gathered in the LibraryManager.cs source code, the application uses a ConcurrentDictionary to store every media item that's registered via the RegisterItem method. 

Once an item is added to this cache, it stays there for the lifetime of the server process because there's no expiration or eviction policy in place. 
Over time, especially as the media library grows (mine is well in the 10,000 range), this cache keeps accumulating items, which eventually leads to memory exhaustion.

I’ve also noticed that this issue becomes even more pronounced when using tools that scan the library for missing content or applications like jellyseerr (I actually run a separate server just for this now).

It seems that these operations increase the rate at which new items are cached, thereby accelerating the memory leak.
I believe that adding a better way to manage this cache—perhaps by introducing an eviction strategy such as a Time-To-Live (TTL) or Least Recently Used (LRU) policy—could make a big difference. 

It might also help to allow users to select the level of caching based on their environment and needs, which would go a long way toward scaling the application.

I have read a few posts in the help section where some think ever expanding cache is a good thing.  I am not in this camp.  Cache only makes sense when it is very targeted and should allow the application to respect the server and other potential applications.

Thanks for taking the time to read this. I hope this information is useful.

[TheDreadPirate]

What OS is Jellyfin installed on? I've only seen users have this problem with unRAID. And even on my unRAID test system I haven't been able to reproduce this problem.

[TrueCodePoet]

What OS is Jellyfin installed on?  I've only seen users have this problem with unRAID.  And even on my unRAID test system I haven't been able to reproduce this problem.

Two different windows machines. 

API calls      : 
     Windows 10 Pro Build 19045
     Processor Intel® Core™ i5-7300HQ CPU @ 2.50GHz, 2501 Mhz, 4 Core(s), 4 Logical Processor(s)
     Installed Physical Memory (RAM) 16.0 GB

Core Server : 
    Windows 11 Pro Build 26100  
    Processor AMD Ryzen AI 9 HX 370 w/ Radeon 890M, 2000 Mhz, 12 Core(s), 24 Logical Processor(s) 
    Installed Physical Memory (RAM) 32.0 GB
    This is a clean install.  Nothing else on server.

Six hours after reboot : 16g RAM usage for Jellyfin.Server

The Windows 10 ran an alternative Media server for years.  I purchased this new machine after having the ram issues.

Unless I am understanding the code wrong.  This Ram issue would be expected with large libraries.

I was looking at issues on GitHub and I also think this is related to this issue :  [Issue]: Library scan causes excessive RAM usage · Issue #11588 · jellyfin/jellyfin
This sounds like the same thing I am seeing.  if you interact with the library..  The more you interact the more the memory grows and never shrinks.  scanning is a good example.

[TrueCodePoet]

Oddly enough I searched this forum and not the GitHub forum. So wow this is not a new issue at all. people keep reporting memory issues, and it seems to me it is the gorilla in the room about how caching should work. I am very surprised at how this is not being taken seriously. I think I will fix this in my code base and run the server for a while and report back. This should be a strait forward fix. I will say I am eager for the new EF implementation for the database. The legacy database is a mess as I have seen the devs mention before.

[TrueCodePoet]

As mentioned, I pulled the Source code down.  I am still running v 10.10 so this is the .Net 8 build.  The LibraryManager.cs code is basically the same.

The subtle changes I have made are simple as a proof of concept.  Adding a TTL to the Cache in LibraryManager.

here is my implementation : 

Code:

        private const string ShortcutFileExtension = ".mblink";
        private static readonly TimeSpan DefaultTTL = TimeSpan.FromHours(2);
        private static readonly TimeSpan CleanupInterval = TimeSpan.FromHours(1);
        private readonly Timer _cleanupTimer;

Added this to the constructor:

Code:

            _cache = new ConcurrentDictionary<Guid, CacheEntry>();
            _cleanupTimer = new Timer(state => PurgeExpiredEntries(), null, CleanupInterval, CleanupInterval);

Change the model being stored in Cach to :

Code:

        private class CacheEntry
        {
            public CacheEntry(BaseItem item, DateTime expiration)
            {
                Item = item;
                Expiration = expiration;
            }
            public BaseItem Item { get; set; }
            public DateTime Expiration { get; set; }
        }

Added a Cleanup to purge Expired entries :

Code:

        private void PurgeExpiredEntries()
        {
            var purgeCount = 0;
            foreach (var kv in _cache)
            {
                if (kv.Value.Expiration <= DateTime.UtcNow)
                {
                    _cache.TryRemove(kv.Key, out _);
                    purgeCount++;
                }
            }
            _lastCleanup = DateTime.UtcNow;
            _logger.LogInformation("Purged {Count} expired entries from the library cache", purgeCount);
            _logger.LogInformation("Library cache contains {Count} entries", _cache.Count);

        }

An example use for get item : 

Code:

        
public BaseItem? GetItemById(Guid id)
        {
            if (id.IsEmpty())
            {
                throw new ArgumentException("Guid can't be empty", nameof(id));
            }
            if (_cache.TryGetValue(id, out var entry))
            {
                if (entry.Expiration <= DateTime.UtcNow)
                {
                    _cache.TryRemove(id, out _);
                }
                else
                {
                    if ((DateTime.UtcNow - _lastCleanup) > CleanupInterval)
                    {
                        PurgeExpiredEntries();
                    }
                    return entry.Item;
                }
            }
            var item = RetrieveItem(id);
            if (item is not null)
            {
                RegisterItem(item);
            }
            return item;
        }

I have also added some general logging in areas I felt needed a little boost.

So far my server is staying under 1 gig of ram.  but I am going to let it keep running.  I watch it bump when scans run and then settle back down a couple hours later.  I have noticed no real performance issues as I thought it could take a small hit.  But if you are using it.  then the cache is saturated.  If anything I think the overall performance is better especially on my resource starved backup server.

I will keep you updated as we could have other areas that could use a bump.

[TrueCodePoet]

Afte this runs for a while. I think I will update the TTL to be a sliding TTL so that frequently accessed items stay in cache longer.

[TrueCodePoet]

After letting the server run for the last 24 hours I have not had to restart the server which is amazing!  But I also found another issue.  

While I drastically reduced the Ram consumption.  I was still seeing a small uptick in ram over time.  Much slower than before but still ram was growing slowly. 

I watched the log files and noticed this entry a lot - item name and file path and then this:" will be refreshed".  I ran through the code and found this is part of the FileRefresher.cs and specifically the ProcessPathChanges method.  

This is due to having the Watch for changes in the library enabled.  I disabled this feature for all of my Larger libraries and holy cow has the performance increased and the ram stays under 600 meg.  So it looks like we have two issues.  The Item Cache and the File Watcher.  I am fine with manually or even scheduling the refresh on my media so I am leaving this off and I have no plans to rewrite this section as of yet.  

I am supper happy with the current performance with these changes.  I did add the sliding Expiration to the Item Cache and I like the performance a lot better. At some point I will look into why the File watcher is creating so many file refreshes, but I just don't have the time right now.  To many other projects.

Hope this helps.