Four ghost interfaces

[Bed]

Hi all

I have a problem on a server Fujitsu CX400 with a daughter card 4x eth ports Intel X722.

The card is installed and properly detected from the bios and OS (Centos v. 7.6.18.10) with its 4 interfaces (eno2-5; eno1 is the embedded interface, that is ok)

[bed@cx400-centos-1 ~]# ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.1.2.133 netmask 255.255.255.0 broadcast 10.1.2.255
inet6 fe80::e003:41ce:bff9:1373 prefixlen 64 scopeid 0x20<link>
ether 8c:0f:6f:7e:04:e8 txqueuelen 1000 (Ethernet)
RX packets 476 bytes 51403 (50.1 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 749 bytes 172328 (168.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x91a00000-91a7ffff

eno2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 8c:0f:6f:7e:04:ee txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno3: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 8c:0f:6f:7e:04:ef txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno4: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 8c:0f:6f:7e:04:f0 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eno5: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 8c:0f:6f:7e:04:f1 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 1922 bytes 272785 (266.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1922 bytes 272785 (266.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

I have noticed that with a network cable plugged into one of the interfaces no one leds are blinking...
Centos is updated.

If I try to configure/activate with nmtui is displayed an alert message, this for all 4 interfaces:

"Could not activate connection: Connection 'Wired connection 1' is not available on the device eno2 at this time"

If I try to configure temporarly with:

sudo ifconfig eno2 10.0.0.1 255.255.255.0

apaprentrly the intefaces appear configured:

eno2: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 10.0.0.1 netmask 255.255.255.0 broadcast 10.0.0.255
ether 8c:0f:6f:7e:04:ee txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

From the same server it respond at ping:

[bed@cx400-centos-1 ~]# ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.063 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.050 ms


but there is no connection really, and no leds are blinking...

Any suggestions?

Thanks for any advice
Bed

[jlehtone]

First a note: commands 'ifconfig' and 'route' are ancient.

For getting (same&more) information, there are alternatives:

Code: Select all

ethtool eno2
ethtool -i eno2
lspci -nn | grep -i net
ip li
ip -s li
ip -4 ad
ip ro
nmcli
nmcli d s
nmcli c s

I assume that you have checked that the cable is ok and that the other device (behind cable) is ok?

Pinging self shows nothing, because those packets never visit the physical NIC.

[TrevorH]

And what is the output from lspci -nn | grep -i net so we can see the hardware involved.

[Bed]

First a note: commands 'ifconfig' and 'route' are ancient.
...

And what is the output from lspci -nn | grep -i net so we can see the hardware involved.

First, thanks all for reply

Wiring and connections are ok, the server is part (master node) of a small cluster connected with a switch to the other slave nodes: slave nodes interfaces are on a dedicated VLAN without communication problems: one of the four "ghost" interfaces should be the downlink to the switch/slave nodes.

I am surprised that no Led lights up when I plug in the network cable...

And these are various results:

Code: Select all

[bed@cx400-centos-1 ~]# ethtool -i eno2
driver: i40e
version: 2.3.2-k
firmware-version: 3.33 0x80000fc0 1.1693.0
expansion-rom-version:
bus-info: 0000:1a:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[bed@cx400-centos-1 ~]# lspci -nn | grep -i net
03:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
1a:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 08)
1a:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 08)
1a:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 08)
1a:00.3 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 1GbE [8086:37d1] (rev 08)
[bed@cx400-centos-1 ~]# ip li
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:e8 brd ff:ff:ff:ff:ff:ff
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ee brd ff:ff:ff:ff:ff:ff
4: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ef brd ff:ff:ff:ff:ff:ff
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f0 brd ff:ff:ff:ff:ff:ff
6: eno5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f1 brd ff:ff:ff:ff:ff:ff
[bed@cx400-centos-1 ~]# ip -s li
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    RX: bytes  packets  errors  dropped overrun mcast
    74872711   518670   0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    74872711   518670   0       0       0       0
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:e8 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    8011039    57297    0       0       0       63885
    TX: bytes  packets  errors  dropped carrier collsns
    8615822    83530    0       0       0       0
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ee brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    0          0        0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    0          0        0       0       0       0
4: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ef brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    0          0        0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    0          0        0       0       0       0
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f0 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    0          0        0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    0          0        0       0       0       0
6: eno5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f1 brd ff:ff:ff:ff:ff:ff
    RX: bytes  packets  errors  dropped overrun mcast
    0          0        0       0       0       0
    TX: bytes  packets  errors  dropped carrier collsns
    0          0        0       0       0       0
[bed@cx400-centos-1 ~]# ip -4 li
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:e8 brd ff:ff:ff:ff:ff:ff
3: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ee brd ff:ff:ff:ff:ff:ff
4: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:ef brd ff:ff:ff:ff:ff:ff
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f0 brd ff:ff:ff:ff:ff:ff
6: eno5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether 8c:0f:6f:7e:04:f1 brd ff:ff:ff:ff:ff:ff
[bed@cx400-centos-1 ~]# ip ro
default via 10.1.118.254 dev eno1 proto static metric 100
10.1.118.0/24 dev eno1 proto kernel scope link src 10.1.118.231 metric 100
[bed@cx400-centos-1 ~]# nmcli
eno1: connected to eno1
        "Intel I210"
        ethernet (igb), 8C:0F:6F:7E:04:E8, hw, mtu 1500
        ip4 default
        inet4 10.1.118.231/24
        route4 10.1.118.0/24
        route4 0.0.0.0/0
        inet6 fe80::e003:41ce:bff9:1373/64
        route6 fe80::/64
        route6 ff00::/8

eno2: unavailable
        "Intel X722"
        ethernet (i40e), 8C:0F:6F:7E:04:EE, hw, port 8c0f6f7e04ee, mtu 1500

eno3: unavailable
        "Intel X722"
        ethernet (i40e), 8C:0F:6F:7E:04:EF, hw, port 8c0f6f7e04ef, mtu 1500

eno4: unavailable
        "Intel X722"
        ethernet (i40e), 8C:0F:6F:7E:04:F0, hw, port 8c0f6f7e04f0, mtu 1500

eno5: unavailable
        "Intel X722"
        ethernet (i40e), 8C:0F:6F:7E:04:F1, hw, port 8c0f6f7e04f1, mtu 1500

lo: unmanaged
        "lo"
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

DNS configuration:
        servers: 10.1.1.5 10.1.1.6
        interface: eno1

Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.

Consult nmcli(1) and nmcli-examples(5) manual pages for complete usage details.
[bed@cx400-centos-1 ~]# nmcli d s
DEVICE  TYPE      STATE        CONNECTION
eno1    ethernet  connected    eno1
eno2    ethernet  unavailable  --
eno3    ethernet  unavailable  --
eno4    ethernet  unavailable  --
eno5    ethernet  unavailable  --
lo      loopback  unmanaged    --
[bed@cx400-centos-1 ~]# nmcli c s
NAME                UUID                                  TYPE      DEVICE
eno1                d24d4f91-50cd-478b-af40-a96ab0235d96  ethernet  eno1
Wired connection 1  37899a9e-bc49-3c3f-8329-e7a1af6b5c69  ethernet  --
Wired connection 2  eb50350f-5773-3412-b96a-663c99ce0a77  ethernet  --
Wired connection 3  7c7b9728-93ca-3477-b997-af1adab2af28  ethernet  --
Wired connection 4  4b0e8ce9-edff-36e2-a4e0-e3237b3e1e10  ethernet  --
[bed@cx400-centos-1 ~]#
[bed@cx400-centos-1 ~]# ethtool eno2
Settings for eno2:
        Supported ports: [ ]
        Supported link modes:   1000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  1000baseT/Full
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: g
        Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: no
[bed@cx400-centos-1 ~]#

"link detetected: no" this and other similar answers worry me: interfaces are detected but...? :-\


B.

[TrevorH]

There are bucketloads of fixes to the i40e module in the changelog for the 7.7 kernel-3.10.0-1062.el7.x86_64 so I'm going to suggest that you enable the CR repo and yum --enablerepo=cr update and get yourself onto 7.7 and retest. The rpm changelog for kernel versions after the 7.6 one ends at line 8882 in the changelog so...

Code: Select all

$ rpm -q --changelog kernel-$(uname -r) | head -8882 | grep -c i40e
97