Hardware under/overkill?

[Defunctusername]

Hi all, 

I'm planning to build a new MPC to server as a Jellyfin server/player and [Thing we don't talk about - TDP], I'm moving over to Linux and this will also serve as the first main step.

I'm currently running Jellyfin on a Windows machine (main/work PC), but want to move it to a dedicated unit in the lounge as I'm finding several playback errors via the app (Google TV) which aren't found natively. 

I generally only have one or two streams going at once, although when the extended family join us, it would be nice to have the capacity for 4-5. 4K would be idea, but 2x 4K and the rest at 1080p would more than likely suffice. 

I've currently got the following system:
OS: Win10 
CPU: AMD 5600X
GPU: GTX 1060oc
RAM: 32GB DDR4 3200Mhz
HDD: 1TB NVMe Gen4, various sized spinning rust.

As this build will be primarily media play only, will I notice a major difference in performance (loading, playback, streaming, etc) stepping over to the following?

OS: Linux Mint 22 Cinnamon
CPU: Intel i3-12100
GPU: Integrated UHD Graphics 730
RAM: 16GB DDR4 3600Mhz
HDD: Same as above.

Thanks in advance.