What happen when you run the installer as admin? That ProgramData directory is where the database and configs should be. Does your user not have sufficient privileges to write to that folder?
Jellyfin 10.10.3 (Docker)
Ubuntu 24.04 LTS
Intel i3 12100
Intel Arc A380
OS drive - SK Hynix P41 1TB
Storage
3x WD Red Pro 6TB CMR in RAIDZ1 (JF Library)