KVM VM Networking Issue

Issues related to configuring your network
etpoole60
Posts: 56
Joined: 2015/06/21 23:25:20

KVM VM Networking Issue

Post by etpoole60 » 2019/04/27 20:58:42

OK, I've rebuilt my machine using CentOS 7.6 and so far everything is work quite well. So now it's time to build my KVM virtual machines to run Apache 2.4, and other stuff.

Under CentOS 6 I used a bash script to execute virt-install to build the VM. I compared the CentOS 6 virt-install parameters to the CentOS 7 virt-install parameters and made the changes where necessary but I kept the 'network=bridge=br0' parameter. What I did was to take 2 of the 6 NICS and create bond0 that became br0. This worked perfectly under CentOS 6 but not so under CentOS 7. CentOS 7 wants to use virbr0. But virbr0 uses 192.168.122.0/24 but my entire network uses 192.168.1.0/24. And the IP address for the VM has to be 192.168.1.125 to satisfy DNS and various hosts files. Also my gateway is 192.168.1.1

What am I doing wrong? Why does this work under CentOS 6 but not CentOS 7?
Gene
Happily Retired
Gene Poole
Woodstock, Georgia

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: KVM VM Networking Issue

Post by TrevorH » 2019/04/27 21:30:42

If you want to use a bridge other than virbr0 then you have to set it up yourself. That's the same under CentOS 6 and 7. Both of them default to setting up virbr0 to be used with a NAT setup and if you don't want that then you need to disable the virbr0 network (/usr/bin/virsh net-autostart default --disable).
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: KVM VM Networking Issue

Post by hunter86_bg » 2019/04/28 05:02:02

By the way, you can always use virt-manager or cockpit to manage your VMs.
It will cover your needs untill getting in sync with RHEL/CentOS 7.

P.S.: Maybe you can wait 6-12 months more and just jump over to version 8.
Everything will be different compared to v6.

etpoole60
Posts: 56
Joined: 2015/06/21 23:25:20

Re: KVM VM Networking Issue

Post by etpoole60 » 2019/05/22 21:08:30

OK, Let me bring you up to date on this issue, yes it still exists!
I have not commented on this issue because I got side tracked by major hardware issues (due to bad weather I lost my cable modem; router for our home security; our personal router; a 24-port Netgear unmanaged switch; a 24-port D-Link unmanaged switch).
The Comcast Tech replaced the cable modem and the router for home security and assisted me with my new Netgear AC2300 router and Netgear ProSafe 24-port 1GB switch. And I replaced all of the patch cables (upgraded to cat6).
Everything worked perfectly (both wired and wireless) until I plugged my 3 servers (each server has 6 NICs) into the 24-port D-Link switch, then both wired and wireless stopped working. I unplugged 2 of the 3 server NICs. On the remaining server I unplugged all NICs except the NIC associated with the hostname. The network started working after I restarted my test machine (I have a HP laptop that has both wired and wireless connections). I started plugging in 1 NIC at a time and everything worked until I got to the last NIC and as soon as I plugged that one in the network stopped! Let me explain: This machine has 6 1 GB NICs; for KVM I use the second MB NIC and the forth NIC on the expansion card to create bond0 (mode 0) and use that to create br0 for the bridge (this worked perfectly on CentOS 6). But it appears that under CentOS 7 this isn't acceptable. So I removed the bond0 and just bridged the 2 physical NICs - but as soon as I plugged in the second cable the network stopped. So I rebooted the server and it still did not work (actually both wired and wireless stopped). I removed the bridge and plugged both NICs in (now all 6 NICs are plugged in) and everythis works great but I've not got a bridge for KVM. I can't use the default bridge because it's IP address is 192.168.122.0/24 but my network uses 192.168.1.0/24.

Is the solution to make a bridge with a single NIC and lose the 2GB bandwidth? I tried bridging and plugging into separate switches with no luck (all runs terminate at the Netgear switch witch is plugged into the Netgear router). A friend told me that it appears to be in some sort of loop but this loop didn't happen when I had my Netgear N600 router. Is there a question I can put to the router support?
TIA
Gene
Happily Retired
Gene Poole
Woodstock, Georgia

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: KVM VM Networking Issue

Post by hunter86_bg » 2019/05/25 16:51:58

The 24-port switch which is manageable,right?
The bridge on your Linux is a kind of a switch and thus it needs proper settings.
Provide the settings for the bridge and check if STP is enabled/didabled. Spanning tree protocol (STP) is supposed to prevent such loops.
Also check the settings for your switch and adjust your settings.

etpoole60
Posts: 56
Joined: 2015/06/21 23:25:20

Re: KVM VM Networking Issue

Post by etpoole60 » 2019/05/25 22:36:50

OK, I followed a suggestion where the Bridge is defined using virt-manager where the bridge uses a single physical NIC. If I need more bandwidth I'll add another Bridge. The VM was built using virt-install in a bash script. And everything is working just fine in the VM it can 'see' the entire LAN and WAN.

What I want to do now is to set up a DNS server for my LAN because the hosts file has grown beyond 60 entries. Does anyone know of a tutorial or installation/configuration guide that can show me how to accomplish building a DNS so I have a single update location as opposed to 32 hosts files to keep in sync?

TIA
Gene
Happily Retired
Gene Poole
Woodstock, Georgia

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: KVM VM Networking Issue

Post by hunter86_bg » 2019/05/26 08:37:02

You can always create your own bridge.
Just run

Code: Select all

brctl show virbr0
and get the settings, so you can have it as close as possible.

Are you planing to use Identify Management ?
If yes, you can use freeIPA for that.
Otherwise, you can use bind , unbount or even dnsmasq.

Edit:

I prefer using the libvirt's built in feature (which is actually dnsmasq) where you can assign an IP to a MAC address and this is resolvable by the clients :)
Also, you can live add/remove entries in the network without restarting libvirt.


Also, your VMs will have DHCP configured, so you don't have to update them all, nor set static IPs.

It should be like this:

Code: Select all

virsh net-update default add ip-dhcp-host    "<host mac='52:54:00:00:00:01' name='bob' ip='192.168.122.45' />"    --live --config
Then you can resolve 'bob' and 'bob.<whatever domain you used when created the network>' from any client .

etpoole60
Posts: 56
Joined: 2015/06/21 23:25:20

Re: KVM VM Networking Issue

Post by etpoole60 » 2019/05/28 23:08:47

First of all let me say to everyone - Thank you for all of your help.
I found a tutorial where the bridge was defined using virt-manager as opposed to doing everything via command line. One of the odd things about this tutorial was that you had to delete the existing virtual network configured for DHCP.
Everything is working fine (my VM built using virt-install in a bash script). What is missing is that the Bridge has only a single physical NIC. I'll attempt to add another physical NIC if I see I need more bandwidth.

It appears that my situation was caused by having a Bridge defined using 2 physical NICs (a network loop maybe?). Here's how I isolated the problem:
There are 6 NICs on the machine, 2 on the mother board, and 4 on a Intel network expansion card. Under CentOS 6 I took the second mother board NIC and the forth expansion card NIC and bound them as a single NIC which I used to create the Bridge (br0). This gave the Bridge a bandwidth of 2GB being used by 8 VMs.
To solve the issue I unplugged 5 of the 6 ethernet patch cables (the one left was the IP associated with the FQDN of the machine). Everything worked just fine from every machine. I plugged all the ethernet patch cables except the second half of the bridge. Everything worked just fine from every machine. When I plugged in that last NIC everything stopped working from every machine including wireless (I tested by entering ping -c 8 192.168.1.1). I deleted the bond0 definition and changed the 2 Bridge NICs to point to br1, and plugged the ethernet patch cable back in. I rebooted the machine so everything would start fresh. It still didn't work. So I removed all of the bond and bridge software (yum -y remove blah-blah-blah) and rebooted the machine.

When the machine came back up I found the tutorial mentioned above and followed it line by line. Everything worked (as I said above) so I defined a VM using KVM and did the test as was mentioned.
I am going to leave it like this and hope I don't need additional bandwidth.

TIA
Gene
Happily Retired
Gene Poole
Woodstock, Georgia

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: KVM VM Networking Issue

Post by TrevorH » 2019/05/29 08:31:13

What is missing is that the Bridge has only a single physical NIC. I'll attempt to add another physical NIC if I see I need more bandwidth.
That's not how it works. If you need to aggregate bandwidth then you don't use a bridge to do that, you need to use a bond or teaming and then that bond/team is the interface that needs to be part of the bridge. You can't just add a second interface to a bridge and hope that it will use all links (well you can hope buit it won't!).
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: KVM VM Networking Issue

Post by jlehtone » 2019/05/31 10:57:33

etpoole60 wrote:
2019/05/28 23:08:47
It appears that my situation was caused by having a Bridge defined using 2 physical NICs (a network loop maybe?).
If you have essentially taken two network switches and connected more than one cable between them, then yes, a loop and broadcast storm seems inevitable.


Even a bond/team may not automatically yield "more bandwidth". Some modes are strictly HA; one link in active use, other(s) standby as backup.
Furthermore, a mode that does use all wires simultaneously can still limit a connection to one wire; the additional bandwidth shows only if you have many devices connecting to many devices over the bond/team.

Post Reply