I have four computers n0-n3 interconnected with GbE (n0 is frontend) and the "compute" nodes n1-n3 are now also being interconnected with Infiniband (n0 has no IB adapter). DHCP is very convenient for kickstart install (n1-n3 have the same kickstart file, no need to customize network-scripts) and I'd like to also assign IPoIB addresses to "ib1"-"ib3" using DHCP. Is it possible to do that over the GbE switch? n0 has no IB HCA.
That is, I'd like to have
Code: Select all
# ypcat hosts |sort |uniq
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
150.XX.XX.XX frontend frontend.dom.es
192.168.1.10 n0 nodo0 nodo0.cluster
192.168.1.11 n1 nodo1 nodo1.cluster
192.168.1.12 n2 nodo2 nodo2.cluster
192.168.1.13 n3 nodo3 nodo3.cluster
192.168.2.11 ib1 iband1 iband1.cluster
192.168.2.12 ib2 iband2 iband2.cluster
192.168.2.13 ib3 iband3 iband3.cluster
Code: Select all
# ip addr
1: lo: ...
2: enp17s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0a:5e:XX:XX:XX brd ff:ff:ff:ff:ff:ff
inet 150.XX.XX.XX/23 brd 150.XX.XX.255 scope global enp17s0
valid_lft forever preferred_lft forever
3: enp13s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:1d:7d:XX.XX.XX brd ff:ff:ff:ff:ff:ff
inet 192.168.1.10/24 brd 192.168.1.255 scope global enp13s0
valid_lft forever preferred_lft forever
Code: Select all
# ip addr
1: lo: ...
2: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:24:8c:XX:XX:XX brd ff:ff:ff:ff:ff:ff
inet 192.168.1.11/24 brd 192.168.1.255 scope global dynamic enp6s0
valid_lft 32919sec preferred_lft 32919sec
3: qlib0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 65520 qdisc pfifo_fast state DOWN qlen 256
link/infiniband 80:00:00:03:fe:80:00:00:00:00:00:00:00:11:75:00:00:XX:XX:XX brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
Code: Select all
# cat ifcfg-qlib0
DEVICE=qlib0
TYPE=InfiniBand
ONBOOT=yes
HWADDR=80:00:00:03:fe:80:00:00:00:00:00:00:00:11:75:00:00:XX:XX:XX
BOOTPROTO=dhcp
#BOOTPROTO=none
#IPADDR=192.168.2.11
#PREFIX=24
#NETWORK=192.168.2.0
#BROADCAST=192.168.2.255
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
MTU=65520
CONNECTED_MODE=yes
NAME=ipoib0
https://access.redhat.com/documentation ... erver.html
...would work for me (this was my translation)...
Code: Select all
subnet 192.168.1.0 netmask 255.255.255.0 {
option routers n0;
...
filename "/kickstart/ks.cfg";
}
subnet 192.168.2.0 netmask 255.255.255.0 {
}
host n1 {
hardware ethernet 00:24:8C:XX:XX:XX;
option dhcp-client-identifier=80:00:00:03:fe:80:00:00:00:00:00:00:00:11:75:00:00:XX:XX:XX;
fixed-address n1,ib1;
}
https://access.redhat.com/documentation ... _line.html
but getting the IPoIB address from DHCPd in n0 (via GbE) would allow me to have all the Ethers and HCAs in a single file (dhcpd.conf) and the same kickstart file for all n1-n3 (except a systemctl enable opensm in n1).
I have been reading man dhcpd[.conf], dhclient, dhcrelay, and learning the parameters used by dhclient on CentOS. Reading files /var/lib/NetworkManager/dhclient-*.conf I feel rather "intimidated" (as when reading Procedure 21.1.3 in
https://access.redhat.com/documentation ... setup.html
which by the way I have used for PXE kickstart), and I thought to myself better not to meddle in the affairs of wizards.
I think I want a dhcrelay _before_ the DHCPDISCOVER goes out the HCA, and maybe it can be done with a mix of NetworkManager and dhclient -sf magic in /var/lib/NetworkManager/*conf and/or /usr/libexec/nm-dhcp-helper. Is it possible? Or maybe I have misunderstood it all and it simply can't be done...
Thanks in advance for any ideas