CentOS 7 Kickstart using LACP and team driver

General support questions
Post Reply
babelr
Posts: 3
Joined: 2018/08/17 15:09:26

CentOS 7 Kickstart using LACP and team driver

Post by babelr » 2018/08/17 17:26:06

Hi everyone!

I'm trying to kickstart a bunch of Servers with CentOS 7.5.1804. Each server has two network connections to a switch and I have to use LACP port channeling.
The kickstart configurations are stored on a local web server, so I have to establish a network connection with kernel parameters in order to download the kickstart file. Since the switch will suspend the network ports when no LACP is detected I have to do channeling from the very beginning.

This post helped me a lot establishing the connection and run the kickstart installation successfully.

However, it uses the bonding driver. I'd like to setup the team driver instead.

Unfortunately, I did not find any documentation about using the team driver in LACP mode at boot time. The (Red Hat Documentation does not tell me how to set the teaming mode to LACP. I can only define it's slave members like so:

Code: Select all

team=master:slaves
So I ended up configuring the bonding driver at boot time and the team driver within the kickstart file like so (y.y.y.y = gateway)

Code: Select all

linuxefi /images/pxeboot/vmlinuz ip=x.x.x.x::y.y.y.y:24:servername:bond0:none nameserver=z.z.z.z bond=bond0:eno1,eno2:mode=802.3ad,miimon=100,lacp_rate=slow,updelay=200,downdelay=200,xmit_hash_policy=layer3+4 inst.ks=http://location.to/kickstart.cfg
And inside the kickstart file:

Code: Select all

network  --bootproto=static --device=team0 --gateway=y.y.y.y --ip=x.x.x.x --nameserver=z.z.z.z --netmask=255.255.255.0 --noipv6 --activate --teamslaves="eno1,eno2" --teamconfig="{\"runner\": {\"name\": \"lacp\",\"tx_hash\": [\"eth\",\"ipv4\"],\"active\": true,\"fast_rate\": false,\"agg_select_policy\": \"lacp_prio\"},\"link_watch\": {\"name\": \"ethtool\"}}"
This somehow works, until setup is done and the server is rebooted. After that nmcli tells me, that there is still the bond0 connection from the boot manager and the new team0 connection, which both claim the same slave interfaces.

Code: Select all

# nmcli c show
NAME                   UUID                                  TYPE      DEVICE
Team connection team0  15a9cee2-619e-4e84-abd3-f5b48e25c45a  team      team0
bond0                  6eb04786-315a-493a-918e-9b9c96a36d76  bond      bond0
eno1                   72edcb1b-16cd-4f5c-9487-1a92f288348e  ethernet  eno1
team0 slave 2          38be5e24-e964-40ef-96c1-e653d2066a14  ethernet  eno2
eno2                   9537f9e3-227d-4fe0-9597-447d6c438e00  ethernet  --
team0 slave 1          7bc5b3a0-37c0-41fa-8b24-cabc4a222085  ethernet  --
I have to manually delete the bond0 connection in order to gain network access. Deleting the connection within the kickstart %post section won't work. It's gonna reappear when the server is rebooted

What's the correct syntax to fire up LACP teaming with kernel parameters? Is it even supported (yet)?
Shouldn't the bond0 interface be deleted, after anaconda downloads the kickstart file and establishes the new teaming network connection?

Any help appreciated!

Cheers
Rene
There are 10 types of people in the world. Whose who understand binary, and those who don't

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: CentOS 7 Kickstart using LACP and team driver

Post by hunter86_bg » 2018/08/20 04:27:00

Why do you think that the switch will kill your connection if you don't use LACP?

In real example it will be like a dead NIC and the server got rebooted and should work.
Test without the LACP and then use your team setttings in the kickstart.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: CentOS 7 Kickstart using LACP and team driver

Post by TrevorH » 2018/08/20 11:03:15

On my Cisco switches, if the switch is configured for LACP then you cannot use the individual connections standalone, they will not pass any traffic. You have to have the server side configured to use LACP at the same time.

I am not aware that the team driver can be configured via kernel command line options, unlike the bonding one, which can. The list of options that can be passed to the CentOS 7 installer are at https://access.redhat.com/documentation ... tions.html and that does have team=master:slave. Personally, I tried the team driver and found it distinctly lacking and buggy and went back to good old bonding which actually works.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

babelr
Posts: 3
Joined: 2018/08/17 15:09:26

Re: CentOS 7 Kickstart using LACP and team driver

Post by babelr » 2018/08/20 11:33:59

hunter86_bg wrote:
2018/08/20 04:27:00
Why do you think that the switch will kill your connection if you don't use LACP?

In real example it will be like a dead NIC and the server got rebooted and should work.
Test without the LACP and then use your team setttings in the kickstart.
Well, our switches (Cisco Nexus VPC Pairs) are configured with the "lacp suspend-individual" command, so the connection will be disabled if there is no LACP present. Of course I could disable that feature and use just one interface for the setup, but that's not where I want to go. Besides, going that way still leaves a configured eno1 interface (Generated by dracut initrd) in addition to a team interface with two slave interfaces (generated by parse-kickstart)
I'm actually interested in why there is still a eno1/bond0 interface after the setup is complete or how to configure a team (with LACP) at boot time with kernel parameters.
There are 10 types of people in the world. Whose who understand binary, and those who don't

babelr
Posts: 3
Joined: 2018/08/17 15:09:26

Re: CentOS 7 Kickstart using LACP and team driver

Post by babelr » 2018/08/21 12:29:37

I found two ways to produce a clean network configuration with kickstart files.
One way is to include the kickstart files into the boot media and skip the initial network connection.
The other way is to disable the NetworkManager, delete the automatically created ifcfg files and recreate them by hand.
That's how I solved it in my kickstart file:

Code: Select all

%post
# disable network manager
systemctl disable NetworkManager
mv /etc/sysconfig/network-scripts/ifcfg-lo /etc/sysconfig/network-scripts/lo.ifcfg
rm -rf /etc/sysconfig/network-scripts/ifcfg-*
mv /etc/sysconfig/network-scripts/lo.ifcfg /etc/sysconfig/network-scripts/ifcfg-lo

# Create static Network configuration scripts
cat > /etc/sysconfig/network-scripts/ifcfg-team0 <<EOF
# Generated by Kickstart script
NAME="team0"
DEVICE="team0"
DEVICETYPE="Team"
TEAM_CONFIG='{"runner": {"name": "lacp","tx_hash": ["ipv3","ipv4"],"active": true,"fast_rate": false,"agg_select_policy": "lacp_prio"},"link_watch": {"name": "ethtool"}}'
ONBOOT="yes"
NM_CONTROLLED="no"
BOOTPROTO="static"
IPADDR="x.x.x.x"
GATEWAY="y.y.y.y"
NETMASK="255.255.255.0"
DNS1="z.z.z.z"
IPV6INIT="no"
EOF

cat > /etc/sysconfig/network-scripts/ifcfg-eno1 <<EOF
# Generated by Kickstart script
NAME="eno1"
DEVICE="en01"
TEAM_MASTER="team0"
DEVICETYPE="TeamPort"
ONBOOT="yes"
NM_CONTROLLED="no"
HWADDR="$(ethtool -P eno1 | awk ' {print $3 }; ')"
EOF

cat > /etc/sysconfig/network-scripts/ifcfg-eno2 <<EOF
# Generated by Kickstart script
NAME="eno2"
DEVICE="eno2"
TEAM_MASTER="team0"
DEVICETYPE="TeamPort"
ONBOOT="yes"
NM_CONTROLLED="no"
HWADDR="$(ethtool -P eno2 | awk ' {print $3 }; ')"
EOF

%end
There are 10 types of people in the world. Whose who understand binary, and those who don't

Tuxwielder
Posts: 4
Joined: 2014/01/14 16:52:16

Re: CentOS 7 Kickstart using LACP and team driver

Post by Tuxwielder » 2018/10/08 16:24:44

Same issue here :)

Looks like there is no simple way to use the right runner in CentOS/dracut. This warning tells a lot:

"Team master team0 specified, but no /etc/teamd/team0.conf present. Using activebackup."

Creating the file in a %pre-section in the kickstart-configuration file isn't working for me. I have no idea how you are supposed to create this in the dracut/anaconda root-environment. So I adapted the initrd as follows :)

Create a working directory:
# mkdir /root/jr

Copy the initrd.img file you want to adapt:
# cp /tftpboot/centos7/initrd.img /root

Unpack the initrd.img file to the working directory:
# cd /root/jr
# xzcat ../initrd.img | cpio -idmv


Make your adaptations, in this case edit the right dracut-script and replace "activebackup" with "lacp":
# vi ./usr/lib/dracut/hooks/cmdline/96-parse-team.sh

Repack the initrd.img:
# find . | cpio -o -c | xz --format=lzma > ../initrd_lacp.img

Now you should be able to PXEboot using the new image, and get further with the installation (at least it works for me ;) ). There should be an easier way though... If anyone knows how to generate the teamd-configuration in a more friendly way, then please let me in on it :)

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: CentOS 7 Kickstart using LACP and team driver

Post by TrevorH » 2018/10/08 17:14:41

Did you pass the team options via the kernel command line?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Tuxwielder
Posts: 4
Joined: 2014/01/14 16:52:16

Re: CentOS 7 Kickstart using LACP and team driver

Post by Tuxwielder » 2018/10/08 19:41:53

Yes, I use the following in the pxelinux.cfg:

...team=team0:eno1,eno2 ip=team0:dhcp...

The docs don't mention any other options than that though, so no way to specify the runner...

Kickstart configured as follows:
...
network --device=team0 --activate --bootproto=dhcp --teamslaves="eno1,eno2" --teamconfig="{\"runner\": {\"name\": \"lacp\"}}"
...

Post Reply