Only 1 NIC used in the bond

[JohnPreston78]

Hi everyone,

I have 1 server running the latest CentOS 6 version

Code: Select all

uname -a
Linux nc-9.emea 2.6.32-504.16.2.el6.x86_64 #1 SMP Wed Apr 22 06:48:29 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

The NICs are very standard :

Code: Select all

02:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
02:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)

My Switch is a Netgear GS748T and FlowControl (802.3aX) is enabled on the machine.

I have configured my 2 NICs to be bonded into a single bond0

Code: Select all

$> cat /etc/sysconfig/network-scripts/ifcfg-bond0

[root@nc-9 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
USRCTL=none
NOZEROCONF=yes

IPADDR=109.104.120.12
GATEWAY=109.104.120.1
NETMASK=255.255.255.192

DNS1=192.168.1.100
DNS2=8.8.8.8

MTU=9000
BONDING_OPTS="mode=4 lacp_rate=fast miimon=80 ad_select=1"

[root@nc-9 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
ONBOOT=yes
SLAVE=yes
MASTER=bond0
BOOTPROTO=none
MTU=9000

[root@nc-9 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
ONBOOT=yes
SLAVE=yes
MASTER=bond0
BOOTPROTO=none
MTU=9000

Doing some iperf, I see that I reach 1Gbps which is 1 link's max speed. I have Zabbix monitoring all my hosts and I can see that indeed, traffic goes over only 1 link.
Checking the proc/sys config I get:

Code: Select all

[root@nc-9 network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 80
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Aggregator selection policy (ad_select): bandwidth
Active Aggregator Info:
        Aggregator ID: 7
        Number of ports: 1
        Actor Key: 17
        Partner Key: 1
        Partner Mac Address: 00:00:00:00:00:00

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e8:9a:8f:74:11:ac
Aggregator ID: 7
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: e8:9a:8f:74:11:ad
Aggregator ID: 8
Slave queue ID: 0

What I do not understand is that the number of ports is 1 and the MAC address is 00:00:00:00:00:00
Has anyone any idea why ? What could I have missed so far ?

If you need more details from my system, feel free to ask

[TrevorH]

LACP requires set up on the switch(es) that it talks to. Is your switch configured to allow LACP on these two ports?

[JohnPreston78]

Well it is enabled definitely from the switch perspective.
That said, the UI doesn't tell much about the port's configuration itself. That's where I miss my CISCO CLIs .

I will try later adding a second switch (which makes more sense anyway) to the environment and see how this goes.
Reading this : http://brasstacksblog.typepad.com/brass ... 00000.html it sounds to give a bit of insight as to how to configure the port on the Switch's side.
The "Active" mode rang a bell so I activated in the SW to bound ports into a LAG in LACP mode. So far, it sounds to have helped.
I get the HWAddr set properly, the bond uses 2 NICs.

BUT

On the L3 everything works just fine (icmp works, routing passes). I configured the ports and the config of another server B which's BOND came up too with 2 NICs etc (same LAG as the first server). Problem is : communication between those 2 machines on L7 is not happening.
Packets are sent properly, but they are not going back. From another server C which has only 1 NIC the communication happens both ways just fine.

I gonna go to the DC in a couple hours to have my eyes on it in case I blow up the entire network .
Keep you posted.