It does look like my original install (Windows 10) was executed as the Admin (right-click "Run as administrator") and when I ran the "jellyfin_10.8.13_windows-x64.exe" to update I didn't do that and ran it as my user account.
I made backups of "C:\Program Files\Jellyfin", "C:\ProgramData\Jellyfin" and "C:\Users\UserName\AppData\Local\jellyfin". I uninstalled Jellyfin, then deleted all 3 of those file locations. After a reboot I installed Jellyfin as Admin (not as a service). I then had the "Program Files\Jellyfin" folder back but the "ProgramData\Jellyfin" folder was missing. Maybe the ProgramData gets created after Jellyfin runs and completes the setup for the first time?
I hadn't run Jellyfin at this point so copied my backup of the "ProgramData\Jellyfin" into ProgramData. Then I started Jellyfin and my libraries, settings, theme, etc all seem to be back. Yay!
The backups I made were after I screwed-up the update, but since I hadn't run the update as admin it didn't overwrite all my files. Also found out my Backblaze backups don't include Program Files or ProgramData, so will be creating a Windows 10 backup plan to create copies that Backblaze will grab from another drive. Also has me wondering about my Windows Subsystem for Linux and will make sure that's getting picked-up too!