[SOLVED] Route traffic through private IP for only certain h

[IanB]

Hi

I am having a weird problem which I cant figure out - so I was hoping someone here could give me a hand.

First off the end goal is that a specific server in my network runs an IPSEC connection to another company and I want all other servers to route traffic for the IP on that network through this single server.

Server 1 in this example is the server that runs the IPSEC connection. (CentOS 6.6)
Server 2 in this example is an app server that would route traffic for only that specific IP through server 1. (CentOS 6.5)

Some IP's that will be used below:
Server 1

Code: Select all

Server 1 Public IP: x.x.x.x
Server 1 Public Broadcast: x.x.x.y
Server 1 Public Gateway: x.x.x.z
Server 1 Internal IP: 10.0.64.10/24

Server 2

Code: Select all

Server 2 Public IP: y.y.y.y
Server 2 Public Broadcast: y.y.y.z
Server 2 Public Gateway: y.y.y.a
Server 2 Internal IP: 10.0.64.150/24

Those servers have full connectivity between them internally (i.e. I can ping, ssh etc from one to the other without problem). They also both have full acceess to the internet and can be reached that way
----------

Server 1

Here is an ip a for that

Code: Select all

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:99:12:85 brd ff:ff:ff:ff:ff:ff
    inet x.x.x.x/28 brd x.x.x.y scope global eth0
    inet6 xxxx:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:99:12:8f brd ff:ff:ff:ff:ff:ff
    inet 10.0.64.10/24 brd 10.0.64.255 scope global eth1
    inet6 fe80::20c:29ff:fe99:128f/64 scope link
       valid_lft forever preferred_lft forever

Here is an ip route

Code: Select all

# ip route
x.x.x.y/28 dev eth0  proto kernel  scope link  src x.x.x.x
10.0.64.0/24 dev eth1  proto kernel  scope link  src 10.0.64.10
169.254.0.0/16 dev eth0  scope link  metric 1002
169.254.0.0/16 dev eth1  scope link  metric 1003
default via x.x.x.z dev eth0

Here is a sysctl -p

Code: Select all

# sysctl -p
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 1
net.ipv4.conf.all.rp_filter = 1
kernel.sysrq = 1
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 1

----------
Server 2

I've added a single test ip (8.8.8.8) to server two to test if it works before bringing IPSEC into the equation

Here is an ip a

Code: Select all

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:0c:29:15:8b:01 brd ff:ff:ff:ff:ff:ff
    inet y.y.y.y/29 brd y.y.y.z scope global eth0
    inet6 fe80::20c:29ff:fe15:8b01/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:0c:29:15:8b:0b brd ff:ff:ff:ff:ff:ff
    inet 10.0.64.150/24 brd 10.0.64.255 scope global eth1
    inet6 fe80::20c:29ff:fe15:8b0b/64 scope link
       valid_lft forever preferred_lft forever

Here is an ip route

Code: Select all

# ip route
8.8.8.8 via 10.0.64.10 dev eth1
y.y.y.z/29 dev eth0  proto kernel  scope link  src y.y.y.y
10.0.64.0/24 dev eth1  proto kernel  scope link  src 10.0.64.150
default via y.y.y.a dev eth0

----------

Now when I try do a ping from Server 2 -> 8.8.8.8 here are the tcpdumps from each server:

Server 2

If I tcpdump on eth0 i get no matches (so the route appears right!). eth1 gets matches:

Code: Select all

# tcpdump -vvv -i eth1 -n host 8.8.8.8
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
11:25:55.609902 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 1, length 64
11:25:56.609262 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 2, length 64

Server 1 (The hopeful gateway for 8.8.8.8)

On eth1 (Private)

Code: Select all

# tcpdump -vv -i eth1 -n host 8.8.8.8
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes

11:27:20.608766 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 86, length 64
11:27:21.608738 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 87, length 64

On eth0 (public)

Code: Select all

# tcpdump -vv -i eth0 -n host 8.8.8.8
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:29:04.608773 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 190, length 64
11:29:05.608800 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto ICMP (1), length 84)
    10.0.64.150 > 8.8.8.8: ICMP echo request, id 17999, seq 191, length 64

I've disabled the FW on both (as a test), made sure to not have any blocking rules on FORWARD traffic (as a separate test) and I just never get my traffic through from Server 2 to 8.8.8.8. I've also tried substituting 8.8.8.8 for another server that is reachable from both servers and the same thing happens.

I'm open to any suggestions - i'm super confused

Thanks in advance,
Ian

[aks]

I'm not sure what you're asking. What does

First off the end goal is that a specific server in my network runs an IPSEC connection to another company and I want all other servers to route traffic for the IP on that network through this single server.

mean?
Are you trying to get all servers on your network to pass all traffic to this ("IPSEC connected") router? If so, then add the router as the default gateway.
Or, are you try to pass some of the traffic to this ("IPSEC connected") router and other data via a different network path? If so, you could look at LARTC http://www.lartc.org/howto/. As a simple example of LARTC, say all hosts have the same (and working) destination to the internet, but I wanted to pass only SMTP traffic from 192.168.1.5/32 via a different network route to the Internet. Basically I need to split my routing into seperate tables (i.e.: the routing policy database). I would have the main (or default) table with the entry of whatever the existing default route is and then another table (let's call it mail), with a single entry of something like default via <the IP of where I want the SMTP traffic to go>. So if traffic matches the email table, it'll go via <the IP of where I want the SMTP traffic to go>, or in command terms something like ip rule add from 192.168.1.5 table email. Now I need to match on the transport layer (for the SMTP service). Here, we could use the firewall to "mark" the packets that are destined for port 25 (SMTP) and have all such marked traffic use the email table - the mark is called the forward-mark (or fwmark) and is tracked inside the kernel for the lifetime of the packet. So something like iptables --table mangle --append PREROUTING --protocol tcp --source-port 25 --jump MARK --set-mark 1 and then ip rule add fwmark 1 table email
The power of LARTC is truely amazing and one of my top reasons why Linux is simply the best OS ever (period).

[IanB]

Hi,

Sorry about that - in hindsight that was horribly worded

Once this is configured correctly then all traffic destined for 1 IP will have to go via the IPSEC tunnel (The App talks to a service that requires IPSEC connection). All other traffic can go via the Default Gateway. I'm going the test with 8.8.8.8 to just ensure that the gateway is working when routing to that server.

I've done it before without using LARTC - is that the only way that it can be done? I've done it before on another network using the same settings as now - only difference is the default gateway for the servers was the gateway - it wasnt conditional routing as with this setup. It was the same with two IFACs, the one ip_forwarding to the main eth0 to get to the internet.

I'll look into LARTC though - havent heard of it before today! So thanks for that!

Cheers
Ian

[IanB]

Hi,

This is resolved!

I was missing the following IPTables rule:

iptables -t nat -I POSTROUTING 1 -o eth0 --jump MASQUERADE

Works 100% now!

[aks]

Good to hear. Please mark this thread as solved.

Thanks.