[Solved] Only 100 M/bit network adapter Centos vm's on hyper-v

Issues related to configuring your network
Post Reply
crashdog
Posts: 38
Joined: 2015/07/11 16:11:43
Location: Schaffhausen,CH
Contact:

[Solved] Only 100 M/bit network adapter Centos vm's on hyper-v

Post by crashdog » 2017/09/12 07:03:25

Hello,
I have an odd issue with x11 forwarding. I've been googling and searching in the forum for quite a bit but could not find any solution yet.

In my home LAN I have one Mac book pro (VMWare) and one HP Proliant DL380 G8 (Hyper-V 2012 core). Both systems host Centos 7 VMs. When I forward x11 from my Mac book I get good response speeds (just like working locally) any were in the LAN. But when I forward x11 from the Proliant system the x11 windows are extremely slow. Too slow to work with. One thing is that when I connect to the Proliant based Centos systems over the hyper-v console I get fast x11, also pinging the systems give reply <1ms. So it's only when forwarding x11 over ssh that it's slow.

I'm forwarding the x11 windows on my mac using ssh -X -C using Xquarz and also to windows 8.1 using putty with x11 forwarding enabled and Xming.
Another thing is that I installed Xrdp on one of the Proliant based Centos with the same result as when trying to export x11 directly, slow connection.

I'm really unsure on how to further investigate this problem. Are there any x11 analysing programms that could check where the bottleneck is ?

One difference is that the mac book is connected to a simple Netgear switch GS108 that goes directly to the fritzbox router. The Proliant system is connected to a Cisco SG-200-26. When looking at the admin webpage of the Cisco router I see that the proliant network ports are connected with 1000 Mbit/s.

Is there any further information that I could give to clarify the issue ? Any one experianced something simmilar ?

*EDIT" changed the Title because it does not only affect x11 but shows to be a generall network device problem.
Some further information that I found : https://social.technet.microsoft.com/Fo ... rverhyperv
https://social.technet.microsoft.com/Fo ... rverhyperv

So basically the question boils down to: how to correctly configure a 1GBit virtual network adapter in Centos 7.3 when hosted by Hyper-V .

Regards,

Gery
Last edited by crashdog on 2017/09/23 12:42:08, edited 3 times in total.

crashdog
Posts: 38
Joined: 2015/07/11 16:11:43
Location: Schaffhausen,CH
Contact:

Re: Slow x11 forwarding

Post by crashdog » 2017/09/16 07:20:34

It's weekend and I have some time to try to get to the bottom of this problem. After googling some more I found the ethtool that I didn't know of until now. First thing is that the ethtool gives me an emtpy output on those systems that have a slow x11 connection over tcp.
example :

Code: Select all

> ethtool eth0
Settings for eth0:
        Supported ports: [ ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: Unknown!
        Duplex: Unknown! (255)
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
*EDIT* While the driver info is :

Code: Select all

[gery@webserver1 network-scripts]$ sudo lspci
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (AGP disabled) (rev 03)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 01)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 02)
00:08.0 VGA compatible controller: Microsoft Corporation Hyper-V virtual VGA
(So I don't have an ethernet adapter ? So where does eth0 come from ? How can it connect to the intra/internet ?)

*EDIT"

Code: Select all

$ /etc/rc.d/init.d/network status
Configured devices:
lo eth0
Currently active devices:
lo eth0 virbr0
Another piece of information :

Code: Select all

netstat -i
Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500   422052      0      0 0         83231      0      0      0 BMRU
lo       65536    19874      0      0 0         19874      0      0      0 LRU
virbr0    1500        0      0      0 0             0      0      0      0 BMU

Code: Select all

 nmcli d
DEVICE      TYPE      STATE      CONNECTION
virbr0      bridge    connected  virbr0
eth0        ethernet  unmanaged  --
lo          loopback  unmanaged  --
virbr0-nic  tun       unmanaged  --
where as the systems on the mac give an output like :

Code: Select all

Settings for eno16777736:
Supported ports: [ TP ]
Supported link modes:   10baseT/Half 10baseT/Full 
                        100baseT/Half 100baseT/Full 
                        1000baseT/Full 
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes:  10baseT/Half 10baseT/Full 
                        100baseT/Half 100baseT/Full 
                        1000baseT/Full 
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
MDI-X: off (auto)
Cannot get wake-on-lan settings: Operation not permitted
Current message level: 0x00000007 (7)
      drv probe link
Link detected: yes
*Edit* The driver info is :

Code: Select all

00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:00.0 USB controller: VMware USB1.1 UHCI Controller
02:01.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
02:02.0 Multimedia audio controller: Ensoniq ES1371/ES1373 / Creative Labs CT2518 (rev 02)
02:03.0 USB controller: VMware USB2 EHCI Controller
But not sure what the means yet...

*EDIT* Thought: when downloading I get 11.5 mb/s so it's 100 m/bit not 1G... but I still don't see how I can force it to settle for 1G. Obvoiusly the network adapter physicly is conected with 1G from my Cisco router's output : 16 GE16 1000M-copper Up 1000M Full

Gery

crashdog
Posts: 38
Joined: 2015/07/11 16:11:43
Location: Schaffhausen,CH
Contact:

Re: Only 100 M/bit network adapter Centos vm's on hyper-v

Post by crashdog » 2017/09/16 14:42:54

I tried to install a fresh centos system on the hyper-v with the result that it now shows 10000Mb/s with the ethtool. I also updated one of the older systems to the latest release which now also show 10000Mb/s with the ethtool. But practically when transfering data I still only get 100Mb/s... :?:

Code: Select all

[gery@svserver1 ~]$ ethtool eth0
Settings for eth0:
        Supported ports: [ ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
        Link detected: yes
Updated system: CentOS Linux release 7.4.1708 (Core)

Code: Select all

[gery@localhost ~]$ ethtool eth0
Settings for eth0:
        Supported ports: [ ]
        Supported link modes:   Not reported
        Supported pause frame use: No
        Supports auto-negotiation: No
        Advertised link modes:  Not reported
        Advertised pause frame use: No
        Advertised auto-negotiation: No
        Speed: 10000Mb/s
        Duplex: Full
        Port: Other
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
Cannot get wake-on-lan settings: Operation not permitted
        Link detected: yes
freshly installed:CentOS Linux release 7.4.1708 (Core)

Code: Select all

Uploading mqadv_dev80_linux_x86-64.tar.gz to /home/gery/mqadv_dev80_linux_x86-64.tar.gz
mqadv_dev80_linux_x86-64.tar.gz                                                                                                              100%  552MB  12.5MB/s   00:44
SFTP test from 192.168.178.61 -> 192.168.178.45
*EDIT": Ok I realized that using sftp is not a good idea for a performance test : viewtopic.php?f=16&t=42908
So I installed iperf3 on centos and iperd.exe from iperf.fr on windows.
Also having read somewhere (which I can't find anymore) that using "Gen 1" for vm's on hyper-v can cause Centos (or Linux in general) to use a legacy network driver which will max out at 100Mb/s.
Will post my findings on iperf later.

Gery

crashdog
Posts: 38
Joined: 2015/07/11 16:11:43
Location: Schaffhausen,CH
Contact:

Re: Only 100 M/bit network adapter Centos vm's on hyper-v

Post by crashdog » 2017/09/17 07:12:24

Case 1:
System 1 Client :192.168.178.41 : System 2 Server 192.168.178.61 : Both systems are Centos VMs on the same HP Proliant system using the same hyper-v virtual switch:

Code: Select all

 iperf3 -c 192.168.178.61
Connecting to host 192.168.178.61, port 5201
[  4] local 192.168.178.41 port 35060 connected to 192.168.178.61 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   342 MBytes  2.86 Gbits/sec    7   1.87 MBytes
[  4]   1.00-2.00   sec   451 MBytes  3.79 Gbits/sec    0   2.03 MBytes
[  4]   2.00-3.00   sec   444 MBytes  3.72 Gbits/sec    0   2.16 MBytes
[  4]   3.00-4.00   sec   451 MBytes  3.79 Gbits/sec    0   2.29 MBytes
[  4]   4.00-5.00   sec   379 MBytes  3.18 Gbits/sec    0   2.37 MBytes
[  4]   5.00-6.00   sec   378 MBytes  3.17 Gbits/sec  294   1.23 MBytes
[  4]   6.00-7.00   sec   424 MBytes  3.55 Gbits/sec    0   1.31 MBytes
[  4]   7.00-8.00   sec   415 MBytes  3.48 Gbits/sec    0   1.37 MBytes
[  4]   8.00-9.00   sec   386 MBytes  3.24 Gbits/sec   34   1.03 MBytes
[  4]   9.00-10.00  sec   370 MBytes  3.10 Gbits/sec    0   1.14 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  3.94 GBytes  3.39 Gbits/sec  335             sender
[  4]   0.00-10.00  sec  3.94 GBytes  3.39 Gbits/sec                  receiver

iperf Done.

Case 2:
System 1 Client 192.168.178.41 : System 2 Server 192.168.178.26 : Second system is on it's own windows based Hardware System. The two systems are connected by wire which goes from the HP Proliant to the cisco switch and over the Netgear 108GS to the Windows system:

Code: Select all

 iperf3 -c 192.168.178.26
Connecting to host 192.168.178.26, port 5201
[  4] local 192.168.178.61 port 34026 connected to 192.168.178.26 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   133 KBytes  1.09 Mbits/sec    0   17.0 KBytes
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    0   17.0 KBytes
[  4]   2.00-3.00   sec  60.8 KBytes   498 Kbits/sec    0   17.0 KBytes
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0   14.1 KBytes
[  4]   4.00-5.00   sec  66.5 KBytes   544 Kbits/sec    0   14.1 KBytes
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   14.1 KBytes
[  4]   6.00-7.00   sec  60.8 KBytes   499 Kbits/sec    0   14.1 KBytes
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   14.1 KBytes
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   14.1 KBytes
[  4]   9.00-10.00  sec  66.5 KBytes   545 Kbits/sec    0   14.1 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   387 KBytes   317 Kbits/sec    0             sender
[  4]   0.00-10.00  sec   269 KBytes   220 Kbits/sec                  receiver
Case 3:
The same two sytems as case 2 but server client switched:

Code: Select all

iperf3.exe -c 192.168.178.61
Connecting to host 192.168.178.61, port 5201
[  4] local 192.168.178.26 port 56719 connected to 192.168.178.61 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   256 KBytes  2.09 Mbits/sec
[  4]   1.00-2.00   sec   768 KBytes  6.29 Mbits/sec
[  4]   2.00-3.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   3.00-4.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   4.00-5.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   5.00-6.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   6.00-7.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   7.00-8.00   sec  1.00 MBytes  8.39 Mbits/sec
[  4]   8.00-9.00   sec  1.25 MBytes  10.5 Mbits/sec
[  4]   9.00-10.00  sec  1.25 MBytes  10.5 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-10.00  sec  10.8 MBytes  9.02 Mbits/sec                  sender
[  4]   0.00-10.00  sec  10.8 MBytes  9.02 Mbits/sec                  receiver

iperf Done.
Case 4:
System 1 Client 192.168.178.35 Hyper-V console (directly connected to the ethernet adapter) System 2: Server 192.168.178.26 Windows system on how hardware :

Code: Select all

-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 192.168.178.35, port 64689
[  5] local 192.168.178.26 port 5201 connected to 192.168.178.35 port 64690
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec  3.38 MBytes  28.3 Mbits/sec
[  5]   1.00-2.00   sec  1.62 MBytes  13.6 Mbits/sec
[  5]   2.00-3.00   sec  1.12 MBytes  9.44 Mbits/sec
[  5]   3.00-4.00   sec  1.38 MBytes  11.5 Mbits/sec
[  5]   4.00-5.00   sec  1.62 MBytes  13.6 Mbits/sec
[  5]   5.00-6.00   sec  1.38 MBytes  11.5 Mbits/sec
[  5]   6.00-7.00   sec  1.75 MBytes  14.7 Mbits/sec
[  5]   7.00-8.00   sec  1.25 MBytes  10.5 Mbits/sec
[  5]   8.00-9.00   sec  3.38 MBytes  28.3 Mbits/sec
[  5]   9.00-10.00  sec  2.03 MBytes  17.0 Mbits/sec
[  5]  10.00-10.07  sec   229 KBytes  26.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.07  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.07  sec  19.1 MBytes  15.9 Mbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

crashdog
Posts: 38
Joined: 2015/07/11 16:11:43
Location: Schaffhausen,CH
Contact:

Re: [Solved] Only 100 M/bit network adapter Centos vm's on hyper-v

Post by crashdog » 2017/09/23 13:03:03

Ok, this problem is sort of solved. The solution was to update the "Windows Server 2012 R2" driver. Since Hyper-V Core is basically the same as running Hyper-v on Windows Server 2012 the driver is compatible. Just for the record the driver can be found here : http://h20564.www2.hpe.com/hpsc/swd/pub ... nvOid=4168

The reason I only call this sort of solved is that there are still a lot that I don't understand in this respect.

Like:
Why does the nic not appear in the list when lspci is used ?
Why does it appear to autonegotiate the network speed when ethtool tells me that autoneg is off ? (different network speeds on same nic when connected to different servers).
How can I for sure tell which NIC driver is used ?
Why is it so hard to find information on Centos related network setup ?

This was a painfull experiance (not blaming anyone or anything). It appears that network speed related problems are common but never easy to resolve.
The available tools don't help too much. It appears that solving these kind of problems need a big portion of luck.

Regards,
Gery

Post Reply