Storing all metadata outside the media library

[OneNot]

I have Jellyfin installed in docker on a NAS. The NAS currently has two HDDs in a pool that runs the OS, Jellyfin and media library (+ some other light stuff). 
I recently got an NVMe drive, and I was thinking I'd move Jellyfin and all its metadata and images on it, keeping only the media itself in the library (and probably subtitles?) to hopefully improve performance a little. 
It's not a huge issue, but browsing my library does get a little sluggish and the images load slightly slow sometimes. Not too bad, but anyway... Also, I do sometimes get stutters when streaming media to my Nvidia Shield connected to my TV. I'm not sure if this would help with that, since the video would still be streamed from the HDDs, but with all other Jellyfin traffic moved to the NVMe, maybe a tiny bit? All this gets worse whenever a scheduled virus check is run, which again, might be slightly less of an issue if Jellyfin activity is moved to the NVMe...

As I understand it, if I uncheck "Save artwork into media folders", the artwork gets saved somewhere in the jellyfin config and/or cache directories instead. 
What about the NFO files? I see that I can disable the NFO file "metadata savers", but what does that mean exactly? Where does the metadata get stored if not in the NFO files?
Does it only get fetched in real time from the metadata providers as I open a movie page in the library? I'd hope not...
Does the metadata also get stored in the config/cache directories somewhere? If so, does it still work the same as with the NFO files in media libraries? As in, can I manually edit the metadata, lock it from being changed and otherwise use it as I did before with NFO files, but instead it's now stored somewhere else? 
What other limitations would this approach have?

Any info or advice would be appreciated. 
Thanks

[TheDreadPirate]

With images, if you uncheck the "Save artwork" setting, it will save images in /config/metadata. Unless you already have images in your media directory, it will continue to use them. Same applies for NFOs. If you turn off NFO saver, but you already have them, Jellyfin will continue to use them. I believe info in NFOs are written to the database, but I'm not entirely sure.

If you removed your NFOs and removed the images, you'd have to "Search for missing metadata" or "Replace all metadata" and Jellyfin will pull down images from TMDB/TVDB/etc., and store them in /config/metadata. It is not doing that in real time.

If you were to go this route, having Jellyfin store everything in /config, the UI should be much faster since all that data is now on a SSD instead of a HDD.

The only draw back to this approach over using NFOs or images in your media directory is preserving custom images or custom metadata if you were to lose your /config data. If you aren't customizing images and metadata, there aren't really any drawbacks, but there are a lot of advantages.

[He-who-is-tired]

This is exactly what I do. All media exists on the NAS and is accessible as read-only libraries to Jellyfin, any metadata exists within the Jellyfin docker files.

I've done this for years for a couple reasons:
- If I want to experiment with a different media system solution, all files can be made available to it as well. Prior to Jellyfin I had experimented with Plex and a couple of other pieces of software.

- Separating media to read-only access means that if any piece of software has a bug -- or there's a PEBKAC fault -- where it would say, delete an entire tree of media, well, it can't since that's all read-only. (Re)naming files, audio tags, etc. all can take place outside of a media software solution where I have a little more control and insight over things.

- TheDreadPirate's comments about efficiencies just makes me happier that I've done it this way!

I can always take backups of Jellyfin for safety's sake (and do). Media backups then can be managed separately. And, it is interesting to watch how different software packages manage the same set of files, insofar as how robust their own tagging/metadata handling goes, etc. Jellyfin, in my own opinion, is at least as good as any of the others I've tried, it's free, and it is under active development!

[OneNot]

Thank you both!

I'll probably do it at some point this week when I have time.
I'll try to remember to report back here with results.