Playback is slow when server is accessed remotely

[yhtomit]

When my friends try to access my Jellyfin server playback seems to be much slower than it should be considering our connection speed. I have Gigabit upload and download and a friend I tested with also has Gigabit download. When watching with a video that has a bitrate of 8Mbps the video would constantly pause and need to wait wait for buffering. Just as a test I've also had them try downloading the video from my Jellyfin server and the download speed never passes 1MB/s. Considering both our connection speed, I would expect it to be much faster. Is there anything else I can try to improve this? I've also run multiple speedtests and my both my download and upload speed are as they should be.

Currently I have a pretty simple set up. Just a Windows 10 machine with Jellyfin installed, and just port forwarded.

Thanks in advance.