[Xen] Networking with bonding.

Issues related to configuring your network
Post Reply
StinGer
Posts: 46
Joined: 2008/11/13 01:24:14
Location: Universe > Milky Way > Solar System > Earth > France > Montrouge
Contact:

[Xen] Networking with bonding.

Post by StinGer » 2011/12/26 19:49:44

Hi everyone !

It's been quite a long time since I last came here... Everything was doing right in fact, up until I wanted to change something. :-D

So... I have a server running CentOS 5.7, up to date etc... I have 3 NICs that I bonded together in mode 4 (802.3ad) and I have the correct config on the other part, on my switch. On top of that bond0, I created a VLAN with ID 5. My server is used for several stuff, but mainly samba 3 as PDC and routing to the Internet. With a regular kernel, it's working perfectly (CentOS rocks !).

For several reasons, I need to run a WinXP virtual machine. So I decided to move towards xen. I installed all the stuff required for xen. So now, I am running a xen kernel 2.6.18-274.7.1.el5xen. The 2.6.18-274.12.1.el5 is installed but I have not rebooted yet. Anyway, I do not believe the update would solve my problems. What I would like is to keep my bond0 as the physical NIC with xen.

I read a lot everywhere and I came to the conclusion I should end with the following configuration. My bond0 device should be part of a bridge (let's call it xenbr0), with IP 0.0.0.0 (to receive everything). Then, I should create a bridge, let's call it xenbr0, and add vif0.0 to xenbr0. Finally, I should put my desired addresses for dom0 on veth0 (created by xen). And finally, fire up all the stuff. All right, that's the theory. The fact is : this is not working. I cannot reach anything in my network with this config', and nothing on the network can reach the server.

If I do use the scripts, they give the bridge (xenbr0, which is in fact renamed bond0 because bond0 is called pbond0... whatever...) the IP I want my dom0 to have. In that config', the server is reachable through the bridge. This could be all right, but if I add a vifU.0 to that bridge to let my domU get in the network, the domU does not connect to the network. The domU is completely isolated.

So my question is simple : how should I proceed to get what I want ? I guess the answer is a bit more complicated. :-P

I must add I tried to work with the MACs : give an appropriate MAC to the vifn.0 ifaces (one like 02:... as it is said on the xen wiki), the same to the vethn, not carrying at all about MACs... No way.

I hope someone could help me. Thanks in advance.

Cheers !

StinGer
Posts: 46
Joined: 2008/11/13 01:24:14
Location: Universe > Milky Way > Solar System > Earth > France > Montrouge
Contact:

Re: [Xen] Networking with bonding.

Post by StinGer » 2012/02/20 10:53:08

BUMP

No one ? :cry:

pschaff
Retired Moderator
Posts: 18276
Joined: 2006/12/13 20:15:34
Location: Tidewater, Virginia, North America
Contact:

[Xen] Networking with bonding.

Post by pschaff » 2012/02/20 18:21:08

[url=http://www.centos.org/modules/newbb/viewtopic.php?topic_id=28726&forum=54]Readme First[/url]:

[quote][b][size=120]12. What to do if no one answers[/size][/b]

Please wait for at least 24 hours. The volunteers on this forum live all over the world. If it's day time where you live, it's going to be night time somewhere else and the person able to answer your question may be sleeping, so give everyone a chance to read your question. Be sure that you have refreshed the page if you left it open - new messages do not cause an auto-refresh. Also, be sure to check your Notification Method - at the bottom of each page when logged in - as it defaults to "Private Message", not email as many people seem to expect.

If after 24 hours you haven't received any answers, then you may [b]bump your thread by posting more information. By more information, we mean what you have tried during the last 24 hours to fix the problem. You have been trying to fix your problem, haven't you and not just waiting for someone else to fix it for you?[/b]
[/quote]

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: [Xen] Networking with bonding.

Post by TrevorH » 2012/02/20 21:53:28

Sorry, I didn't see this one until now.

What works for me is to leave everything as default and edit /etc/xen/xend-config.sxp and find the line about network-script and change it to

[code]
(network-script network-bridge-bonding)
[/code]

That invokes the xen provided network script that handles bonding and just worked for me. I didn't have to make any other changes at all so I would back out anything else that you already tried.

StinGer
Posts: 46
Joined: 2008/11/13 01:24:14
Location: Universe > Milky Way > Solar System > Earth > France > Montrouge
Contact:

Re: [Xen] Networking with bonding.

Post by StinGer » 2012/03/07 10:48:14

@pschaff: hmm... My first message may not reflect it, but I've been on this problem for 3 months now (not had time to focus on it full time however) and did not post anything. When in the end I had no other choice, I posted my message and explained all the manipulations I did and ideas I tried (or at least, the ones I remembered). The point of my message was to call for new ideas. When I saw my post was far away in the pages with no replies, I felt a bump would let other people read it and maybe propose ideas. I'm not really the kind of person who waits for solution to fall from above on their own... I know this is your job to make people read and respect the rules, but that one was a bit harsh. 8-)

[quote]
TrevorH wrote:
Sorry, I didn't see this one until now.

What works for me is to leave everything as default and edit /etc/xen/xend-config.sxp and find the line about network-script and change it to

[code]
(network-script network-bridge-bonding)
[/code]

That invokes the xen provided network script that handles bonding and just worked for me. I didn't have to make any other changes at all so I would back out anything else that you already tried.[/quote]

Yeah, this is what I meant when I said "If I do use the scripts". I should have been clearer. ;-) In fact, I pretty much tried all the scripts except the NAT ones. With no success.

But I have a great news. I finally got it working, by hand. Will check how to make it work correctly with the scripts now.


Here is what I did. First, I was in a mixed setup, as I said before, receiving packets to dom0 through the bridge: awful and disgusting. :evil: My 3 NICs were bonded together into pbond0. I may not have mention it before, but there are 2 different types of NICs: 2 Intel 82576 10/100/1000 and 1 Intel 82574L 10/100/1000.

1. Turned off the bridge (bond0 in my case), the vifs, the veths and pbond0 [b]but not the ethX[/b].
2. Deleted all my VLANs.
3. Set the MAC address of veth0 with the "real" MAC address of one of the ethX (bonding causes the 3 NICs enslaved to use the same MAC address).
4. Deleted the bridge.
5. Recreated the bridge (named bond0 hereafter).
6. [b]Set the MAC address of pbond0, the bridge and vif0.0 to FE:FF:FF:FF:FF:FF.[/b] This also sets the MAC address of the ethX.
7. Added pbond0 (the bond) and vif0.0 to the bridge (bond0).
8. Set my desired IP addresses on veth0.
9. Turned on the bridge (bond0), pbond0 (the bond), vif0.0 and veth0.
10. Added my VLANs to veth0.

Then, as I'm doing firewalling, I flushed my iptables rules and modified according to my new configuration. In particular, I allowed all traffic through the bridge using physdev. here is what you have to do (taken from the Xen networking Wiki):

[code]iptables -t filter -A INPUT -m physdev --physdev-is-in -j ACCEPT
iptables -t filter -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
iptables -t filter -A OUTPUT -m physdev --physdev-is-out -j ACCEPT[/code]
Finally, I updated some config files and restarted the needing services (dhcpd, samba, named etc...).

Then, with (vif-script vif-bridge), any new virtual machine created is correctly added to the bridge and works flawlessly.

I guess the problem came from incorrect MAC addresses. More preciseley, I think the FE:FF... address was not correctly forwarded to the ethX interfaces. But that's just my opinion.

Hope this will be handful for some people. ;-)

StinGer
Posts: 46
Joined: 2008/11/13 01:24:14
Location: Universe > Milky Way > Solar System > Earth > France > Montrouge
Contact:

Re: [Xen] Networking with bonding.

Post by StinGer » 2012/03/07 12:40:47

I was looking for an edit button to mark my thread as solved but I'm blind and I can't find any...

Post Reply