two NICs, two gateways

Issues related to configuring your network
Post Reply
stuartk
Posts: 2
Joined: 2018/11/12 12:42:50

two NICs, two gateways

Post by stuartk » 2018/11/12 13:17:36

I would like to configure my box with (2) NICs, on two subnets, with two default gateways, such that responses to traffic entering via NIC 1 depart via NIC 1, ditto for NIC 2.

I have created (2) tables:
guru# cat /etc/iproute2/rt_tables
100 mgmt
200 production
guru#

Assigned a default route to each table:
guru# ip route show table mgmt
default via 10.1.100.1 dev enp3s0f0
guru# ip route show table production
default via 10.1.104.1 dev enp3s0f1
guru#

And written both incoming & outgoing rules for each table:
guru# ip rule show
0: from all lookup local
100: from 10.1.100.82 iif enp3s0f0 lookup mgmt
100: from all to 10.1.100.82 iif enp3s0f0 lookup mgmt
200: from 10.1.104.9 iif enp3s0f1 lookup production
200: from all to 10.1.104.9 iif enp3s0f1 lookup production
32766: from all lookup main
32767: from all lookup default
guru#

The addresses / subnets:
guru# ip addr show
[...]
2: enp3s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
[...]
inet 10.1.100.82/22 brd 10.1.103.255 scope global noprefixroute enp3s0f0
valid_lft forever preferred_lft forever
4: enp3s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
[...]
inet 10.1.104.9/22 brd 10.1.107.255 scope global noprefixroute enp3s0f1
valid_lft forever preferred_lft forever

However, at this point, the box can only return traffic sourced from locally attached hosts.

I'm fumbling something. Can anyone see what I am missing?

Errata:
guru# ip route show
10.1.100.0/22 dev enp3s0f0 proto kernel scope link src 10.1.100.82 metric 103
10.1.104.0/22 dev enp3s0f1 proto kernel scope link src 10.1.104.9 metric 102

guru#

CentOS Linux release 7.5.1804 (Core)

--sk
Last edited by stuartk on 2018/11/23 15:01:27, edited 1 time in total.

stuartk
Posts: 2
Joined: 2018/11/12 12:42:50

Re: two NICs, two gateways

Post by stuartk » 2018/11/23 13:54:20

Turns out that specifying the interface name (e.g. ' dev enp3s0f0') broke things. Here is the configuration which works:

cat /etc/iproute2/rt_tables
100 mgmt
200 production

cat rule-enp3s0f0
from 10.1.100.82/32 table mgmt priority 100
to 10.1.100.82/32 table mgmt priority 100

cat rule-enp3s0f1
from 10.1.104.9/32 table production priority 200
to 10.1.104.9/32 table production priority 200

cat route-enp3s0f0
default via 10.1.100.1 table mgmt

cat route-enp3s0f1
default via 10.1.104.1 table production


ip rule show
0: from all lookup local
100: from 10.1.100.82 lookup mgmt
100: from all to 10.1.100.82 lookup mgmt
200: from 10.1.104.9 lookup production
200: from all to 10.1.104.9 lookup production
32766: from all lookup main
32767: from all lookup default

ip route show
default via 10.1.104.1 dev enp3s0f1 proto static metric 102
default via 10.1.100.1 dev enp3s0f0 proto static metric 103
10.1.100.0/22 dev enp3s0f0 proto kernel scope link src 10.1.100.82 metric 103
10.1.104.0/22 dev enp3s0f1 proto kernel scope link src 10.1.104.9 metric 102

Finally, the behavior I found puzzling revolved around 'systemctl network restart' -- I thought I could change various config files, run 'systemctl restart network', and effectively test my changes. In fact, this didn't work well -- the output of 'ip rule show' become progressively weirder, and my test results inconsistent. I ended up resorting to rebooting between config file changes, in order to cleanly observe the results. In my experience, rebooting is rarely the right answer in *nix -- I remain skeptical about this.

--sk

Post Reply