Can't access Jellyfin over the internet after replacing hardware

[gaming09]

Last night I replaced my Raspberry Pi 4 with a 5.
I simply moved the SD card containing my docker setup. 

Everything is working fine - all my containers, except for Jellyfin.
I can access it via the IP or DynamicDNS name on the LAN but over the internet it's not connecting. No settings have changed in docker or Jellyfin. 
The logs don't show any errors when trying to connect so I'm assuming that the connection isn't reaching Jellyfin. 

I have tried on different networks on different devices all have the same issue.
My Router is port forwarding 8096 to the IP of the pi, and like I said I have other Docker containers that are still accessible externally.

did you change the IP of the pi and then update your firewall rules or did you use the same address?