Seeking is very bad

[TheDreadPirate]

With that kind of CPU Power should be fine without a GPU.

Can you post jellyfin logs?

Or, try seeking in the client and then screenshot task manager. Either it's somewhere bottleneck't or we are missing something obvious.

What kind of files are you playing h264/h265? mkv? very big files?

8 broadwell cores isn't as powerful as you think for transcoding.  Encodes can only me split up to a certain point.  Usually around 8-12 threads and the returns on additional threads drops quickly.  So allocating more cores won't make too much of a difference.  Faster cores matters way more.  And broadwell on xeons are usually clocked slower than desktop chips.

IMO a GPU is almost a necessity, even with a modern CPU.  Having the GPU frees up the CPU to do other tasks that improve the Jellyfin UIs responsiveness.