I've got one server here which has many roles in life from firewall to fileserver and pretty much everything else in between. In desperate hopes of TRYING to get some semblance of management I'm trying to break things up into VMs to aide in security as well as management sanity where I don't need to worry about one change bringing the whole house of cards crashing down upon my head if I thumb finger some update to the webserver and take the mailserver offline. We've all done it, I'm just admitting it! ;) LOL

My network topology is:
eth0 (Internet / DSL line - DHCP issues IP)
eth1 (Internal Network, runs a 10.x.x.x Class-A network which is highly segmented based on machines and users. Examples:
10.0.0.1 - The main Server and machine I'm having the issues with, also the firewall, mailserver, fileserver, webserver, mediaserver, CD-Replicator, dhcp, printserver, database, nameserver, ... ... ... (you get the idea, it's the life blood of the studio!)
10.0.0.X - Servers and workstations
10.0.1.X - Desktops
10.0.2.X - Laptops
10.0.6.X - Trusted Wireless
10.0.7.X - Open WAP (firewalls block all access to 10.x.x.x and only provides internet connectivity)
10.0.10.x - Intended landing space of the VMs
et cetera

Yes, I could do this all with a Cass-C 192.168 subnet..... but it's a LOT easier to tell somebody to map a drive to 10.0.0.1\{whatever} so that's what I went with. :)

Anyway, I'm using libvirt/KVM as my Hypervisor, and have been able to get NAT to work, but of course, no connectivity from the rest of the network TO the VMs is possible. So, I need to go with Either ROUTE or BRIDGE.

Method / Attempt 1:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ROUTE: I started with Route, as it seemed like the simplest and fastest method of getting this to work. No joy. I can't get the internal DHCP to even successfully issue an IP to the Guest. I see the request being received in /var/log/messages on the server, but the guest never gets the IP! I've tried setting up static IPs, which also don't work. No Route, either to ETH1 on the server or the world.

# route# brctl show default
bridge name bridge id STP enabled interfaces
virbr0 8000.fe52007a1f8c yes vnet0

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
71.123.1.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth1
10.0.0.0 * 255.0.0.0 U 0 0 0 eth1
default L101.VERIZON-DSL 0.0.0.0 UG 0 0 0 eth0

virsh # net-dumpxml default
<network>
<name>default</name>
<uuid>39c0a7f8-2f30-983d-041f-58ad4698f1ab</uuid>
<forward mode='route'/>
<bridge name='virbr0' stp='on' forwardDelay='0' />
<ip address='10.0.10.1' netmask='255.0.0.0'>
<dhcp>
<range start='10.0.10.100' end='10.0.10.254' />
<host mac='54:52:00:7a:1f:8c' name='fedora' ip='10.0.10.2' />
<host mac='54:52:00:67:d3:f3' name='win7-kvm' ip='10.0.10.3' />
</dhcp>
</ip>
</network>

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Never worked!!! So, that's when I cut bait after 2 days and went back to trying to get BRIDGE to work:
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BRIDGE: I've had even LESS luck here! Looking at the various online wiki's, forum posts, and how-to's they all more or less point to http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging and http://wiki.centos.org/HowTos/KVM neither of which show how to get a setup working for ETH1. I even went so far as to contact the author of the second article, who said he wasn't sure why my mods to his network setup script were not working for an ETH1 setup and that he's since moved on to using a different Hypervisor and was sorry but couldn't hep further. Unfortunately my various desperate Cro-Magnon pokes at the keyboard and Barbaric Yawps at the monitor failed to get the bridge functional, and if fact would render the network on the either server offline to all, even itself.

# cat ifcfg-br0 ifcfg-eth1
# Bridge for VM Network Connectivity
DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
BROADCAST=10.255.255.255
IPADDR=10.0.0.1
NETMASK=255.0.0.0
NETWORK=10.0.0.0
ONBOOT=yes

# cat ifcfg-eth1
# Broadcom Corporation NetXtreme BCM5754 Gigabit Ethernet PCI Express
DEVICE=eth1
#TYPE=Ethernet
TYPE=Bridge
BRIDGE=br0
#BOOTPROTO=none
#BROADCAST=10.255.255.255
HWADDR=00:18:f3:44:e0:55
#IPADDR=10.0.0.1
#NETMASK=255.0.0.0
#NETWORK=10.0.0.0
ONBOOT=yes

...and this is the bridge setup script I tried using:
# cat /etc/libvirt/qemu/networks/network-setup.sh
#!/bin/sh
PATH=$PATH:/usr/sbin:/sbin
sudo brctl addbr br0
sudo ifconfig eth1 0.0.0.0
sudo brctl addif br0 eth1
sudo ifconfig br0 10.0.10.1 netmask 255.0.0.0 up
sudo route add -net 10.0.10.0 netmask 255.0.0.0 br0
sudo route add default gw 10.0.10.1 br0
sudo tunctl -b -u root
sudo ifconfig tap0 up
sudo brctl addif br0 tap0
export SDL_VIDEO_X11_DGAMOUSE=0
sudo iptables -I INPUT -i br0 -j ACCEPT

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Never worked, knocked everything offline, and screwed the network up so much I had to REBOOT to get everything online again...
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

As I said, I got NAT working. Easy. They are issued DHCP IP addresses, can see ETH1 and the rest of the internal network as well as the world... but none of the machines on the internal network can access them... despite them all running on the same subnet (10.x.x.x with a netmask of 255.0.0.0). I -NEED- that to work, as it does me NO good to move functionality off of the baremetal server to the VMs, if the rest of the internal network are unable to access them! I also couldn't care LESS which method I go with (Bridge or Route) so long as it WORKS. :( I realize I'm probably doing something wrong here, but I've no clue what, and I'm unable to find ANY source of info which would work on my server... or at least that I can make heads or tails of and figure out how to modify.