Login

crashx · 2024-06-19, 03:30 PM

Hi all,

I have Jellyfin deployed successfully and now am exposing my server on the internet for family and friends. I want to harden it with Fail2Ban. My configuration is as follows.

Ngnix Proxy Mgr.
Docker container
192.168.1.108
Configuration is exactly like the JF guide
Takes connections in on port 80, forwards them to 8096 on the next machine (192.168.1.106)
Sets headers in Custom Locations

Jellyfin Server
Docker container (official)
192.168.1.106:8096
Network settings configured for Known Proxy

Fail2Ban
Docker container (crazy max)
192.168.1.106
Jail matches JF guide, chain is DOCKER-USER (and I have tried FORWARD as well)

Behavior
F2B detects IPs attempting to brute force the server and bans them. Makes expected updates to IPTables on the host (*.106). Does this by creating its own chain and adding IPs. However, the IP is never blocked and it appears that all packets are flowing to 0.0.0.0. For the life of me, I cannot figure out why. Does anyone have any insight?

Thanks!

IP TABLES OUTPUT (Note the packets next to 0.0.0.0; the IPs listed are via VPN, so no private info in this post):
Chain f2b-jellyfin (1 references)
pkts bytes target prot opt in out source destination
   0 0 REJECT 0 -- * * 84.247.59.144 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.127 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 85.203.15.105 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 85.203.15.103 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.9 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.50 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.49 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.45 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.43 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.39 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.38 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.29 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.217 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.21 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.20 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.18 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.17 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.143 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.124 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.123 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.118 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.112 0.0.0.0/0 reject-with icmp-port-unreachable
   0 0 REJECT 0 -- * * 84.247.59.111 0.0.0.0/0 reject-with icmp-port-unreachable
   345 563268 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0

TheDreadPirate · 2024-06-19, 03:54 PM

That's F2B output looks right. An example for my Nginx F2B rule.

Code:
Chain f2b-nginx-botsearch (1 references)

target    prot opt source              destination        

REJECT    0    --  103.110.127.136      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  46.249.102.187      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  162.240.66.3        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  47.93.161.187        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  185.244.36.221      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  84.54.51.37          0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  8.218.212.177        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  54.39.17.116        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  5.56.132.84          0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  45.156.129.125      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  45.148.10.174        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  45.128.232.174      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  39.105.120.190      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  31.220.1.83          0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  203.201.63.37        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  194.9.56.102        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  192.250.239.174      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  185.244.36.236      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  185.191.127.212      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  18.171.240.255      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  179.43.185.130      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  165.22.56.121        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  162.214.127.74      0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  146.59.158.41        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  139.196.25.23        0.0.0.0/0            reject-with icmp-port-unreachable

REJECT    0    --  103.74.116.72        0.0.0.0/0            reject-with icmp-port-unreachable

RETURN    0    --  0.0.0.0/0            0.0.0.0/0

(2024-06-19, 03:30 PM)crashx Wrote: However, the IP is never blocked and it appears that all packets are flowing to 0.0.0.0. For the life of me, I cannot figure out why. Does anyone have any insight?

I'm not sure what you mean by this. How are you able to tell "packets are flowing to 0.0.0.0"?

crashx

Thanks for the reply! If you run sudo iptables -xvnL, the leftmost column of output is packets (pkts). In the output I shared:

345 563268 RETURN 0 -- * * 0.0.0.0/0 0.0.0.0/0

345 packets skipped the ban rules and hit the return with source 0.0.0.0 which feels like it could be related to a container's address being forwarded instead of the right address.

here's jellyfin.local. Note that I updated the action to use the newer iptables call where all ports is specified as a argument.

[jellyfin]

backend = auto
enabled = true
port = 8096,80
protocol = all
filter = jellyfin
maxretry = 3
bantime = 86400
findtime = 43200
logpath = /jellyfin-logs/log*
action = iptables[type=allports, name=jellyfin, chain=DOCKER-USER]

TheDreadPirate · 2024-06-19, 04:26 PM

I believe this stack exchange post explains it.

https://unix.stackexchange.com/questions...urn-target

Quote:Packets traverse a chain until they hit ACCEPT, DROP, REJECT, or RETURN. They do not stop on a match unless that match contains a terminating action. In your example, a packet matching the first rule will be marked, but will then be examined (and possibly processed) by the second rule.

Purely for reference, here are the relevant sections from the man page:

A firewall rule specifies criteria for a packet and a target. If the packet does not match, the next rule in the chain is the examined; if it does match, then the next rule is specified by the value of the target, which can be the name of a user-defined chain or one of the special values ACCEPT, DROP [, REJECT], QUEUE or RETURN.

ACCEPT means to let the packet through.
DROP means to drop the packet on the floor, i.e. to discard it and not send any response
[REJECT is used to send back an error packet in response to the matched packet: otherwise it is equivalent to DROP so it is a terminating TARGET, ending rule traversal.]
QUEUE means to pass the packet to userspace.
RETURN means stop traversing this chain and resume at the next rule in the previous (calling) chain. If the end of a built-in chain is reached or a rule in a built-in chain with target RETURN is matched, the target specified by the chain policy determines the fate of the packet.
In response to your specific concern, I would say that your guide is misleading. Unless "associated action" is one of the five terminal actions, packets will continue to flow through the chain until they reach an implicit RETURN at the end.

crashx · 2024-06-19, 04:41 PM

Fair enough; so this implies that packets are hitting are hitting the chain and then not matching any of the rules. Any idea as to how I might diagnose what's happening?

TheDreadPirate · 2024-06-19, 04:50 PM

Other chians can send packets it received through the F2B chain, checking for matches. Return just sends the packet back to the originating chain. WITHOUT the return legitimate traffic will be sent through the F2B chain and will stop traversing. Resulting in it being dropped.

crashx · 2024-06-19, 04:57 PM

Got it. So traffic comes through Nginx, hits the JF box, traverses the f2b chain, and returns because it doesn't match anything. For the life of me, I cannot figure out why the traffic isn't matching the banned IPs that are clearly listed. I wonder if that has to do with the way that NPM is forwarding traffic....

TheDreadPirate · (This post was last modified: 2024-06-19, 05:18 PM by TheDreadPirate.)

I'm re-reading your question and I think I understand what you're asking now.

Check out this blog post. I think they detail the way to get the real IP of the requestor because, you are probably right, the traffic will have the Nginx container's IP with the requestor's IP in the x-forwardedfor header.

https://vaarlion.com/blog/how-to-use-fai...n-the-way/

crashx · 2024-06-19, 06:47 PM

Okay, cool, sounds like my thinking may be directionally correct! I've set up Nginx Proxy Manager just so that I have a frontend to configure the reverse proxy under the hood and I think the IPs are forwarding as expected. Here's a paste from my config which matches the JF docs.

# Proxy main Jellyfin traffic
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_headers_hash_max_size 2048;
proxy_headers_hash_bucket_size 128;

In the JF logs, the right IPs are being captured, so I am assuming that those headers are correct. Subsequently F2B is extracting those IPs from the logs and using them to set the IPTables on the host. But, at the host level I'm not getting a match when I refresh the JF pages to test if the ban worked.... So so strange.

TheDreadPirate · 2024-06-19, 06:59 PM

So Nginx is setting the requestor IP in the header, Jellyfin is reading the header, Fail2Ban is reading the header. But iptables is not reading the header.

I think that is where the blog I linked differs. Instead of blocking with iptables, they are configuring Nginx to silently disconnect the session from the "real IP".

From other posts I've read, iptables cannot read the header for https traffic due to the header being part of the encrypted content. If you want iptables blocking these requests instead of Nginx silently disconnecting, you'd have to setup fail2ban on the Nginx container and send logs to the Nginx container (with rsyslog) and setup the rules on the Nginx container.

Login

Username/Email:
Password:


Or login with a social network below

Login

Username/Email:
Password:


Or login with a social network below

Nginx Proxy Manager, Fail2Ban, and Jellyfin