Poor performance with HP NC552SFP 10GbE NIC

Issues related to configuring your network
Post Reply
Diglet
Posts: 6
Joined: 2018/06/09 21:02:12

Poor performance with HP NC552SFP 10GbE NIC

Post by Diglet » 2018/06/09 21:05:28

I have CentOS 7.5 running on two HP DL360-G7 servers, each with 96GB RAM. The BIOS and all the firmware are up-to-date on the servers. I recently installed one NC552SFP 10GbE fiber NIC in each server. The NIC's firmware is also up-to-date. I then configured IP on the new NIC interface and deactivated the old interfaces. Finally, I disconnected the no-longer-in-use 1Gb copper connections so everything is definitely running over the 10GbE fiber connection.

The data transfer rates between the two servers is about the same as it was with the 1Gb copper connections; about 102MB/s. Obviously there is a performance problem here. The port status on the switch as well as both servers show 10Gb full-duplex.

Any ideas why this is happening and how to correct it? Any help is appreciated. I've searched the internet high and low and cannot find a solution.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Poor performance with HP NC552SFP 10GbE NIC

Post by TrevorH » 2018/06/09 22:56:27

What is the output from ethtool $interface (where $interface is the name of the right interface) ?

What tool are you using to measure the throughput and to generate that traffic?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Diglet
Posts: 6
Joined: 2018/06/09 21:02:12

Re: Poor performance with HP NC552SFP 10GbE NIC

Post by Diglet » 2018/06/12 18:19:00

TrevorH wrote:
2018/06/09 22:56:27
What is the output from ethtool $interface (where $interface is the name of the right interface) ?

What tool are you using to measure the throughput and to generate that traffic?
I created a 10GB ramdisk on each server and transferred an 8GB file between the ramdisks using scp. I did this to eliminate the possibility of slow disk I/O affecting results. I calculated the overall throughput simply by dividing 8192MB by the number of seconds it took to complete the transfer.

Here is the ethtool output:

# ethtool ens2f0
Settings for ens2f0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
10000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: g
Wake-on: g
Current message level: 0x00002000 (8192)
hw
Link detected: yes

# ethtool -i ens2f0
driver: be2net
version: 11.4.0.0r
firmware-version: 11.1.183.23
expansion-rom-version:
bus-info: 0000:09:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: no
supports-priv-flags: yes

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Poor performance with HP NC552SFP 10GbE NIC

Post by TrevorH » 2018/06/12 19:48:13

Yeah, scp is not a good test tool for this. It uses encryption and is limited to a single core for that so the bottleneck in that case becomes "how fast can you encrypt the data".

Try installing and using iperf3 (from EPEL) or configuring e.g an httpd server to serve the files you want to transfer and use curl/wget to pull the data,
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Diglet
Posts: 6
Joined: 2018/06/09 21:02:12

Re: Poor performance with HP NC552SFP 10GbE NIC

Post by Diglet » 2018/06/12 23:16:57

TrevorH wrote:
2018/06/12 19:48:13
Yeah, scp is not a good test tool for this. It uses encryption and is limited to a single core for that so the bottleneck in that case becomes "how fast can you encrypt the data".

Try installing and using iperf3 (from EPEL) or configuring e.g an httpd server to serve the files you want to transfer and use curl/wget to pull the data,
Well, that fixed the "problem". LOL. scp is definitely NOT suitable for network performance testing.

I performed the exact same file transfer as I described previously (ramdisk to ramdisk) but with httpd and wget. wget reported a transfer rate of 279MB/s which is much faster than the scp transfer, but I suspect slower than what can be achieved on a 10 GbE network. So, what is the preferred/recommended or most reliable means to performance test data transfers between servers? And, what is a reasonable expectation I should have of my new 10 GbE network?

And thank you for your help.

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: Poor performance with HP NC552SFP 10GbE NIC

Post by hunter86_bg » 2018/06/18 04:14:33

The 'iperf3' , as mentioned by Trevor, is your swiss army knife.

Post Reply