HAproxy on PFSense GUI Guide with screenshots Webhooks and all

[gaming09]

Edited: Added backend security update
For whatever reason this has been a lot of trial and error and via pfsense HA gui doesn't always align to HAproxy config documentation
For reverse proxy via HAproxy on PFSense this is what you need

Backend: (Example 1.bmp)
Your Jellyfin backend>Advanced Settings>Backend pass thru
These 3 lines as is
*** change JELLYFINLOCALIP with your local IP

Code:

http-request set-header X-Forwarded-Port %[dst_port]   
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server jellyfin(or your backend name) JELLYFINLOCALIP:8096
http-response set-header Access-Control-Allow-Origin https://YOURJFDOMAIN.com
http-response set-header Cache-Control "no-cache, no-store, must-revalidate, private"
http-response del-header Server

Frontend (Example 2.bmp)
Your Frontend>Default backend, access control lists and actions>Access Control lists>Table
***JF is my short-name to attach to the actions table to hit the backend for HA proxy change it to yours
***Change YOUR_PUBLIC_URL.com to your external URL

Code:

jf |  Custom ACL:  | hdr(host) -i YOUR_PUBLIC_URL.com
host_ws | Host starts with: | ws. 
hdr_connection_upgrade | Custom acl: | hdr(Connection) -i upgrade   
hdr_upgrade_websocket  | Custom acl: | hdr(Upgrade) -i websocket

I hope this helps someone else and saves them a few hours of logs (most of this was available on the jellyfin guide but it wasn't intuitive for the GUI
https://jellyfin.org/docs/general/networking/haproxy/

Maybe I have extra stuff in there, if I do let me know. As far as I can tell everything is working now without errors

[DarthVegan]

Nice writeup! Any chance you'd want to do it on OPNsense now?

[SKECHER9]

I am trying to understand the ssl/tls process. It is new to me and if I get something wrong I'd appreciate being corrected.
Question 1: If I'm reading this right, your guide tells HAproxy plugin on the PFsense router to pass-through the whole request of "https://YOURJFDOMAIN.com" to the JF server, and the JF server handles everything for the TLS handshake?
I'd seen people give examples of having the reverse proxy service caddy and nginx (no clue if haproxy has similar function) in order to force proper security.
Question 2: Are there more benefits beyond forcing a more secure TLS handshake with HAproxy handling the tls handshake, or would there be other security concerns on the back end to the JF server?

[TheDreadPirate]

I am trying to understand the ssl/tls process. It is new to me and if I get something wrong I'd appreciate being corrected.
Question 1: If I'm reading this right, your guide tells HAproxy plugin on the PFsense router to pass-through the whole request of "https://YOURJFDOMAIN.com" to the JF server, and the JF server handles everything for the TLS handshake?
I'd seen people give examples of having the reverse proxy service caddy and nginx (no clue if haproxy has similar function) in order to force proper security.

Reverse proxies, in general, handle the TLS handshake and pass the usually unencrypted request to the "upstream" server.

Question 2: Are there more benefits beyond forcing a more secure TLS handshake with HAproxy handling the tls handshake, or would there be other security concerns on the back end to the JF server?

Jellyfin is not hardened.  It makes no claims at being secure.

The reason for using a reverse proxy is because there are exploits and attacks that target the TLS handshake process.  Not necessarily Jellyfin itself.  And the majority of proper reverse proxies/http servers are hardened against these handshake attacks.  Additionally, Jellyfin provides no way to restrict what TLS/SSL protocols are allowed, what ciphers are allowed, or many many other TLS/SSL properties.  But nginx/haproxy/apache/caddy allow you to restrict, for example, the version of TLS to 1.2 or newer.  Or to only use ciphers that aren't vulnerable to attacks that weaken them.  Or ciphers that are resistant to hypothetical future quantum computers.

The wiki for TLS has a section about security that provides some examples of security issues that Jellyfin has no way to mitigate without a reverse proxy.

https://en.wikipedia.org/wiki/Transport_...y#Security

Additionally, I wrote a guide to configure Apache and Nginx to only use the strongest, most secure, available encryption ciphers and protocols.

https://forum.jellyfin.org/t-apache-ngin...ls-ciphers

[SKECHER9]

Ok got it, thanks TheDreadPirate. Was racking my brain on understanding this process, and was having trouble with the HAproxy setup. Got setup to enforce "modern" only TLS v1.3 and AEAD ciphers.
Since I found a solution to the setup I was struggling with for pfSense router ACME and HAProxy forwarding to my Jellyfin server, here is what walked me through.
This guide from Lawrence Systems on YouTube does a good job at explaining the setup. https://youtu.be/bU85dgHSb2E?si=J8C1Zo3N4RawYW1z
In their example though, they don't connect to the outside internet (he gives a simple explanation on what to switch to do so) and they use the self signed cert to encrypt back end for TrueNAS. I think it should be simple enough to set a self signed cert for back end with JF even if it's unnecessary.

Edit: After some testing with help from a friend. An important setting to check to enable logging for Fail2ban on Jellyfin's side is "forwardfor" under pfSense's HAProxy Frontend rules, whatever front end rule is listening outword, under "Advanced settings" "forwardfor" option creates an HTTP "X-Forwarded-For" header which contains the client's IP address. This is useful to let the final web server know what the client address was. (eg for statistics on domains)
--For others who may come back to read this setup, a plain English explanation "forwardfor" lets the jellyfin server on the backend know the external client's IP address. If "forwardfor" is not enabled all external traffic will appear to your Jellyfin server like the client traffic is coming from your router. So then the Jellyfin logs, and therefor the logs Fail2ban, would read (assuming you followed the setup on the Jellyfin docs) that your router failed access and may be blocked.

[cooljimy84]

In Opnsense this setting is in HAProxy under

Services: HAProxy: Settings
Virtual Services (then click the down arrow to the side and select)
Public Services

Last option under "HTTP(S) settings" is the "X-Forwarded-For header

Enable insertion of the X-Forwarded-For header to requests sent to servers."

[Icepick869]

I can't really tell if I did something stupid or if the guide is outdated due to updates to HAProxy and/or PFSense, but I could really use some help. (BTW, this was the most understandable guide I've found, so thank you!)

I followed the guide for the backend and frontend to set up the HAProxy, but it isn't getting through the PFSense to my jellyfin server. The server works and the DNS cuts back to my IP (I was getting a DNS rebinding warning until I changed the GUI port). It seems like the HAProxy isn't answering on 80 or 443 and I can't tell if the frontend is even getting the request.

Here are the frontend settings:
   
I've tried changing the port to 443 or 8920. No joy there.
   
The domainname used in the first value is the FQDN used in the DNS.
   
I've tried this with none as well since it wasn't mentioned in the guide, but that just grayed out the backend as not being used.

The backend actually seemed the most straightforward:
http-request set-header X-Forwarded-Port %[dst_port]   
http-request add-header X-Forwarded-Proto https if { ssl_fc }
server jellyfin 192.168.1.52:8096
http-response set-header Access-Control-Allow-Origin https://[FQDN as listed in DNS]
http-response set-header Cache-Control "no-cache, no-store, must-revalidate, private"
http-response del-header Server

Am I supposed to have added Firewall rules? This seems like the most likely mistake I could have made.

If there is any other info I've missed let me know.

[alleycat]

   

Did you open port 443 in Firewall rules for HAproxy?

[Icepick869]

Did you open port 443 in Firewall rules for HAproxy?

I had not, but now that I have it still isn't working. Thanks for the suggestion! It's the sort of thing I expect it to be, but apparently not quite the solution.

I just had a thought. I'm running VLANs, but don't see anything pointing to the correct VLAN for the server.

[alleycat]

Did you open port 443 in Firewall rules for HAproxy?

I had not, but now that I have it still isn't working. Thanks for the suggestion! It's the sort of thing I expect it to be, but apparently not quite the solution.

I just had a thought. I'm running VLANs, but don't see anything pointing to the correct VLAN for the server.

Instead of me posting a ton of screen shots here, I suggest watching Tom Lawrence's YT video on this, (you may have seen it already). I'm exposing three of my servers to the WAN with success including Jellyfin, but I had to watch it many times myself, lol.

https://youtu.be/gVOEdt-BHDY