Remote IP filter

Is there anyway to restrict access of the jellyfin server to only the reverse proxy?
I am using Apache reverse proxy, I’ve added the remote server IP along with the domain but as soon as I hit save and refresh, the server is forbidden and inaccessible
I’ve added my home IP which solves the problem but how do I make it still accessible when I am on something like mobile data, the Apache reverse proxy seems to be forwarding my origin IP, how do I prevent that?