Public IP routing with QEMU/KVM

[acrylic]

I rent a server from Soyoustart and purchased a /29 network.

I set this up with virt-manager as a routed virtual network on virbr2, I installed my first VM and it was fine.

I leave it a week or so and go to install my 2nd VM on the same domain, and it no longer works on either my new or old VM, but the gateway address on my Centos 7 host machine works just fine (on the same /29 subnet).

I have a sneaky suspiscion the problem is the server providers router not advertising the network as a /29, maybe as a /32?

In any case, I wanted to see if anyone could confirm my config is correct:

See below for host VM config (Centos 7):

Code: Select all

# virsh net-dumpxml publicIP
<network connections='1'>
  <name>publicIP</name>
  <uuid>58588826-b408-4686-ae5f-242dfb9f80a8</uuid>
  <forward dev='eth0' mode='route'>
    <interface dev='eth0'/>
  </forward>
  <bridge name='virbr2' stp='on' delay='0'/>
  <mac address='52:54:00:97:32:32'/>
  <domain name='publicIP'/>
  <ip address='217.182.112.113' netmask='255.255.255.248'>
    <dhcp>
      <range start='217.182.112.113' end='217.182.112.118'/>
    </dhcp>
  </ip>

Code: Select all

virbr2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 217.182.112.113  netmask 255.255.255.248  broadcast 217.182.112.119
        ether 52:54:00:97:32:32  txqueuelen 1000  (Ethernet)
        RX packets 7998870  bytes 2191613094 (2.0 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 12938484  bytes 8766288605 (8.1 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

See below for guest VM config (Centos 7):

Code: Select all

ens9: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 217.182.112.114  netmask 255.255.255.248  broadcast 217.182.112.119
        inet6 fe80::caed:a9a5:b4ff:bfb9  prefixlen 64  scopeid 0x20<link>
        ether 52:54:00:8e:34:d8  txqueuelen 1000  (Ethernet)
        RX packets 158  bytes 21007 (20.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 277  bytes 26729 (26.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Code: Select all

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         217.182.112.113 0.0.0.0         UG    100    0        0 ens9
0.0.0.0         192.168.100.1   0.0.0.0         UG    101    0        0 ens3
192.168.100.0   0.0.0.0         255.255.255.0   U     100    0        0 ens3
217.182.112.112 0.0.0.0         255.255.255.248 U     100    0        0 ens9

Note that the .113 address on virbr2 is reachable, but anything else on that subnet (set correctly via DHCP on my guest machines) does not work.

All and any help is appreciated.

**EDIT**

and yes I can ping the .113 address from the VM's, just not anything passed that address.

This is from 3 different VM's, a Centos 7, a Debain 7 and an Ubuntu 14.04.

[acrylic]

I use iptables, with firewalld masked - still the same thing with iptables disabled. I also use fail2ban, but I doubt that will stop routing to the guest VMs...

Really at a loss here, especially considering this used to work, but no longer does following no changes from me (aside from creating new VM's on the same virtual network).

[hunter86_bg]

Maybe you should create a pcap trace for the outbound interface and check what is going in and what is going out.

[jlehtone]

First a check:

Code: Select all

host# sysctl net.ipv4.ip_forward

That will tell whether your host routes at all.

Then (I must admit that I did never learn to read 'route' or 'ifconfig')
info about the host and vm in different format:

Code: Select all

host# ip -4 ad
host# ip ro
host# brctl show
host# nmcli d s
host# nmcli c s
host# chkconfig --list network
host# systemctl status NetworkManager
host# iptables -S
host# iptables -t nat -S
vm# ip -4 ad
vm# ip ro

You seem to have the vm connected to two subnets, with two default routes. Default, by definition, is one.

If someone from outside attempts connection to vm (217.182.112.114), will the host see any incoming packets?
If not, then check from service provider what they filter and what they advertise.

[acrylic]

First a check:

Code: Select all

host# sysctl net.ipv4.ip_forward

That will tell whether your host routes at all.

Then (I must admit that I did never learn to read 'route' or 'ifconfig')
info about the host and vm in different format:

Code: Select all

host# ip -4 ad
host# ip ro
host# brctl show
host# nmcli d s
host# nmcli c s
host# chkconfig --list network
host# systemctl status NetworkManager
host# iptables -S
host# iptables -t nat -S
vm# ip -4 ad
vm# ip ro

You seem to have the vm connected to two subnets, with two default routes. Default, by definition, is one.

If someone from outside attempts connection to vm (217.182.112.114), will the host see any incoming packets?
If not, then check from service provider what they filter and what they advertise.

Code: Select all

# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0
# sysctl -w net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1

Code: Select all

Pinging 217.182.112.114 with 32 bytes of data:
Reply from 217.182.112.114: bytes=32 time=18ms TTL=51

I am an idioooooooot

Thanks

[jlehtone]

Good. I presume you have read

Code: Select all

man sysctl.d

and have already created a *.conf file in /etc/sysctl.d/
with content

Code: Select all

net.ipv4.ip_forward = 1

to make the setting "permanent".

The kernel default is to not route. Nevertheless, I have routing enabled
on several machines, where I don't seem to explicitly set it on anywhere.

I do suspect that option 'masquerade' on active firewalld zone, or libvirtd
does automagically enable the routing in some cases. If so, they should have
enabled on your system too. Perhaps the way you started services ...

Anyway, adding the explicit config should do no harm here.