curl/wget/yum no-longer work

General support questions
Post Reply
darreng
Posts: 3
Joined: 2015/06/22 14:54:18

curl/wget/yum no-longer work

Post by darreng » 2015/06/22 15:10:54

Hi,

New poster, but long time Redhat user, but I'm a bit stumped by this one...

On a Centos 6.6 box I manage, it has suddenly stopped taking yum updates. From yum.log, the last one was:
Jun 05 01:25:10 Updated: openssl-1.0.1e-30.el6.9.x86_64

Any further runs of yum give a mirror too slow response.

During investigations, I've found that curl and wget no-longer work for non-SSL connections either:

[root@tumbleweed ~]# curl www.centos.org
^C (I cancelled after 60 seconds)
[root@tumbleweed ~]# wget http://www.centos.org/index.html
--2015-06-22 16:00:53-- http://www.centos.org/index.html
Resolving www.centos.org... 85.12.30.227, 2a02:2498:1:3d:0:b0ff:fe43:ad29
Connecting to www.centos.org|85.12.30.227|:80... connected.
HTTP request sent, awaiting response... ^C (I cancelled after 60 seconds)

As wget seems to be connecting OK, I'm assuming this isn't a downstream or local firewall issue. I did temporarily disable iptables to make sure as well, but still no response.

The strange thing is that I can connect to port 80 manually with telnet and get a response:

[root@tumbleweed ~]# telnet www.centos.org 80
Trying 85.12.30.227...
Connected to www.centos.org.
Escape character is '^]'.
get
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/0.8.55</center>
</body>
</html>
Connection closed by foreign host.
[root@tumbleweed ~]#

Checking my IP address online, I can see it is the same as the one allocated to the WAN of the edge router (i.e. before LAN NAT), so I don't think its a proxy.
The same issue happens with all websites, I checked just in case there is a transparent proxy in place and a bad copy is cached.

The server (HP DL360 g5) has been power cycled and had a new cable onto a new switch port, just in case there was an intermittent problem there, but no change.

Anyone got any other ideas?

Cheers

Darren


(before anyone mentions it, I've just logged in as root for the purpose of documenting this issue, normally use sudo!)

User avatar
avij
Retired Moderator
Posts: 3046
Joined: 2010/12/01 19:25:52
Location: Helsinki, Finland
Contact:

Re: curl/wget/yum no-longer work

Post by avij » 2015/06/22 16:22:51

You could try your telnet command with a valid HTTP request. I think there is still a chance that there is some sort of a misbehaving transparent proxy somewhere.

Code: Select all

$ telnet www.centos.org 80
Trying 85.12.30.227...
Connected to www.centos.org.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.centos.org

HTTP/1.1 200 OK
Server: nginx/0.8.55
Date: Mon, 22 Jun 2015 16:19:55 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Last-Modified: Fri, 19 Jun 2015 12:30:57 GMT
ETag: "20142-4224-518de18892240"
Accept-Ranges: bytes
Content-Length: 16932

<!DOCTYPE html>
<html>
...
How about https? The equivalent 'telnet' command for https would be openssl:

Code: Select all

$ openssl s_client -connect www.centos.org:443
CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify return:1
depth=0 C = US, ST = North Carolina, L = Raleigh, O = Red Hat Inc., CN = centos.org
verify return:1
---
Certificate chain
 0 s:/C=US/ST=North Carolina/L=Raleigh/O=Red Hat Inc./CN=centos.org
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
subject=/C=US/ST=North Carolina/L=Raleigh/O=Red Hat Inc./CN=centos.org
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
---
No client certificate CA names sent
Server Temp Key: DH, 1024 bits
---
SSL handshake has read 3480 bytes and written 439 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : DHE-RSA-AES256-GCM-SHA384
    Session-ID: FD31BBE6C317DC9BD9BE48CE7B1FE820FE86AE3368D225A85346F1D9DA4AD5E7
    Session-ID-ctx:
    Master-Key: 801EC06B31CA176EF0E89DA4AB812CD0D9005C257FBFE868B727471668327D6DBCDE2A12BB9ADCA0CD7FBADA2E8D4A17
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
...
    Start Time: 1434989961
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
GET / HTTP/1.1
Host: www.centos.org

HTTP/1.1 200 OK
...

darreng
Posts: 3
Joined: 2015/06/22 14:54:18

Re: curl/wget/yum no-longer work

Post by darreng » 2015/06/22 16:39:04

Thanks for your reply.

Unfortunately both time out:

Code: Select all

[root@tumbleweed ~]# telnet www.centos.org 80
Trying 85.12.30.227...
Connected to www.centos.org.
Escape character is '^]'.
GET / HTTP/1.1
^]
telnet> quit
Connection closed.
[root@tumbleweed ~]# openssl s_client -connect www.centos.org:443
CONNECTED(00000003)
^C   
[root@tumbleweed ~]#
Whilst I can't rule out a downstream transparent proxy, as this is a BT Business Broadband ADSL service, I would expect other people to calling out an issue.

User avatar
avij
Retired Moderator
Posts: 3046
Joined: 2010/12/01 19:25:52
Location: Helsinki, Finland
Contact:

Re: curl/wget/yum no-longer work

Post by avij » 2015/06/22 17:24:15

I'm wondering why you were able to get a response when you submitted an invalid request. I get the impression that the possible transparent proxy didn't understand your request, and passed the request directly to the web server. Valid requests would be processed by the transparent proxy.

Please note that your telnet test wasn't entirely valid. You should be doing

$ telnet www.centos.org 80
GET / HTTP/1.1
Host: www.centos.org

(blank line after Host header)
to make a valid HTTP/1.1 request.

darreng
Posts: 3
Joined: 2015/06/22 14:54:18

Re: curl/wget/yum no-longer work

Post by darreng » 2015/06/26 22:06:07

hi,

just to close this issue off - after much debugging, it seems my ISP had made a change at their end and cURL was getting upset with fragmented packets.
PC web browser worked fine though.

Ultimate fix was to apply mss-clamping on the edge router to use a smaller packet size and prevent the issue - i guess this is why the smaller failure response worked but larger requests failed.

Thanks for your help :)

Post Reply