What settings did you use? Can you share screenshots of Dashboard > Playback? CPU usage isn't unusual if the audio needs to be transcoded. As long as the video transcoding is being offloaded to the iGPU.
Jellyfin 10.10.3 (Docker)
Ubuntu 24.04 LTS
Intel i3 12100
Intel Arc A380
OS drive - SK Hynix P41 1TB
Storage
3x WD Red Pro 6TB CMR in RAIDZ1 (JF Library)