firewalld masquerading on virtual interface

[paultimms]

I have a virtual server with two physical network interfaces, eth0 and eth1. eth0 has a 10.x.x.x address, eth1 has a public IP of 159.x.x.x. I just added a second IP address 5.x.x.x which is bound to virtual interface eth1:0.

On this server I have two versions of the same software, each running a web server. One is on port 80 and the other is on port 81. Port 80 is in the public zone, which is assigned to eth1. I have added a new "test" zone and assigned to interfaced eth1:0. What I would like to do is set up port forwarding just on this zone so that anything hitting 5.x.x.x on port 80 is forwarded to port 81. I have tried with masquerading on and off, but I keep hitting the web server on port 80 rather than the one on port 81.

Is what I'm trying to do possible with virtual network interfaces and firewalld?

Thanks

Paul

[aks]

Can't you just use the port forwarding of firewalld?

General syntax:
firewall-cmd [--zone=<zone>] --add-forward-port=port=<port>[-<port>]:proto=<protocol> { :toport=<port>[-<port>] | :toaddr=<address> | :toport=<port>[-<port>]:toaddr=<address> }
Source: https://fedoraproject.org/wiki/Firewalld?rd=FirewallD

[paultimms]

Thanks aks but it doesn't seem quite that simple, as eth1:0 is a virtual interface of eth1 and it seems that as eth1 is assigned to the public zone, this takes precedence over eth1:0 assigned to the test zone.

If I run ip route I get these entries relating to eth1 (some IPs replaced by letters):
5.a.b.c/29 dev eth1 proto kernel scope link src 5.a.b.d
159.e.f.g/29 dev eth1 proto kernel scope link src 159.e.f.h

Here are the results of --list-all for the public and test zones.

public (active)
target: default
icmp-block-inversion: no
interfaces: eth0 eth1
sources:
services: dhcpv6-client
ports: 81/tcp 80/tcp
protocols:
masquerade: no
forward-ports:
sourceports:
icmp-blocks: echo-reply
rich rules:

test(active)
target: default
icmp-block-inversion: no
interfaces: eth1:0
sources:
services:
ports:
protocols:
masquerade: yes
forward-ports: port=80:proto=tcp:toport=81:toaddr=10.x.y.z
sourceports:
icmp-blocks:
rich rules:

However if I browse to http://5.a.b.c I get the webserver running on port 80, not 81. If I browse to http://5.a.b.c:81 I get the webserver running on port 81. Maybe it's just not possible and I'll have to find another way around it.

[aks]

Well if firewalld doesn't work (some perfence for "real" interfaces apparently - I don't know), what about using redirect or URL re-write in httpd?

[jlehtone]

what about using redirect or URL re-write in httpd?

I was about to ask the same:

Why does the httpd listen 159.e.f.h:80 and 5.a.b.c:81, when it should be able to listen 159.e.f.h:80 and 5.a.b.c:80?
If it can separate between first two, then it should separate between the other two too.


The concept of alias interfaces (eth1:0) has looked archaic for a while.
The eth1 can have more than one IP address. (In IPv6 it will have more than one.)
Perhaps, for forcing separate zones aliases might make sense.


Out of interest, when you have added your rules, how did the netfilter rules did look like?
You can see them with:

Code: Select all

iptables -S
iptables -t nat -S
iptables -t mangle -S

(There is a lot of less essential. Anything with eth1 or eth1:0 are of particular interest.)

[paultimms]

I'm not using httpd, the web server is built into the ERP software I'm running. There are two versions of this software running on the server, one on port 80 and one on port 81. But I'd like both of them to be available to the outside world on port 80, just on different IP addresses.

Here are results from the iptables commands:
-A FORWARD_IN_ZONES -i eth1 -g FWDI_public
-A FORWARD_IN_ZONES -i eth1:0 -g FWDI_test
-A FORWARD_OUT_ZONES -o eth1 -g FWDO_public
-A FORWARD_OUT_ZONES -o eth1:0 -g FWDO_test
-A INPUT_ZONES -i eth1 -g IN_public
-A INPUT_ZONES -i eth1:0 -g IN_test

nat:
-A POSTROUTING_ZONES -o eth1 -g POST_public
-A POSTROUTING_ZONES -o eth1:0 -g POST_test
-A PREROUTING_ZONES -i eth1 -g PRE_public
-A PREROUTING_ZONES -i eth1:0 -g PRE_test
-A PRE_test_allow -p tcp -m mark --mark 0x64 -j DNAT --to-destination 159.e.f.g:81

mangle:
-A PREROUTING_ZONES -i eth1 -g PRE_public
-A PREROUTING_ZONES -i eth1:0 -g PRE_test
-A PRE_test_allow -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x64/0xffffffff

Note that this was after changing the destination IP in the forward-port rule to the 159 address rather than than 10 address.

[jlehtone]

The plot thickens ...

PRE_zim_allow is a typo? Should it not be PRE_test_allow?

Presuming so.
1. Incoming packages arriving via eth1:0 should get marked,
if they aim for port 80.
2. Marked packages are rerouted to 159.e.f.g:81, as they should.
3. What are the outputs of:

Code: Select all

iptables -S IN_public_allow
iptables -S IN_test_allow

How do the other devices on the physical network (and beyond) know that both subnets 159.x.x.x/y and 5.a.b.c/w exist here?

If you do listen eth1 or eth1:0 with tcpdump, do you see interesting details in the packets?

I gather the "ERP" can set a port, but not limit to an interface?


Presuming that the zone-based forwarding cannot separate traffic, one can try rich rules or direct instead.

Code: Select all

man firewalld.richlanguage
man firewalld.direct

The rich should allow matching (destination=5.a.b.c port=80) and only then forward-port.

[paultimms]

Thanks, yes "zim" is the real name of the zone, I was changing it to "test" on here but missed one.

I've been playing with rich rules, and have the following:

public (active)
target: default
icmp-block-inversion: no
interfaces: eth1
sources:
services: dhcpv6-client
ports: 4200/tcp 9580/tcp 4280/tcp 1245/tcp 81/tcp
protocols:
masquerade: no
forward-ports:
sourceports:
icmp-blocks: echo-reply
rich rules:
rule family="ipv4" destination address="159.x.x.x" port port="80" protocol="tcp" accept

test (active)
target: default
icmp-block-inversion: no
interfaces: eth1:0
sources:
services:
ports:
protocols:
masquerade: yes
forward-ports:
sourceports:
icmp-blocks:
rich rules:
rule family="ipv4" destination address="5.x.x.x" forward-port port="80" protocol="tcp" to-port="81"

As you can see, I removed port 80 from the list of ports in the public zone. I've checked http://159.x.x.x and it works. If I try http://5.x.x.x it times out. http://5.x.x.x:81 takes me to the web server on port 81.

The output of iptables -S IN_public_allow:
-N IN_public_allow
-A IN_public_allow -p tcp -m tcp --dport 4200 -m conntrack --ctstate NEW -j ACCEPT
-A IN_public_allow -p tcp -m tcp --dport 9580 -m conntrack --ctstate NEW -j ACCEPT
-A IN_public_allow -p tcp -m tcp --dport 4280 -m conntrack --ctstate NEW -j ACCEPT
-A IN_public_allow -p tcp -m tcp --dport 1245 -m conntrack --ctstate NEW -j ACCEPT
-A IN_public_allow -d 159.x.x.x/32 -p tcp -m tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
-A IN_public_allow -p tcp -m tcp --dport 81 -m conntrack --ctstate NEW -j ACCEPT

The output of iptables -S IN_test_allow:
-N IN_test_allow
-A IN_test_allow -m conntrack --ctstate NEW -m mark --mark 0x64 -j ACCEPT

From this it looks like the rich rule has not applied to the test zone?

I will investigate tcpdump.

[paultimms]

I solved it. It seems that firewalld zones cannot distinguish between the physical network interface and the virtual interfaces belonging to it. So, I took the rich rule from the test zone and applied it to the public zone, and that worked fine. Now, http://159.x.x.x goes to the web server on port 80 and http://5.x.x.x goes to the web server on port 81.

Here's the output of firewall-cmd --zone=public --list-all:

public (active)
target: default
icmp-block-inversion: no
interfaces: eth1
sources:
services: dhcpv6-client
ports: 4200/tcp 9580/tcp 4280/tcp 1245/tcp 81/tcp
protocols:
masquerade: no
forward-ports:
sourceports:
icmp-blocks: echo-reply
rich rules:
rule family="ipv4" destination address="159.x.x.x" port port="80" protocol="tcp" accept
rule family="ipv4" destination address="5.x.x.x" forward-port port="80" protocol="tcp" to-port="81"

Thanks for your help in pointing me in the right direction.