[RESOLVED] RHEL6.0 --> 6.1 update breaks LDAP?

A 5 star hangout for overworked and underpaid system admins.
r_hartman
Posts: 706
Joined: 2009/03/23 15:08:11
Location: Netherlands
Contact:

[RESOLVED] RHEL6.0 --> 6.1 update breaks LDAP?

Post by r_hartman » 2011/06/14 11:32:29

Hi all,

Just curious whether anyone playing with RHEL6 using OpenLDAP has experienced that LDAP stopped working after updating RHEL6.0 to RHEL6.1?
I'm not familiar with the bugzilla procedures and I'm not even sure this is a bug, as there's a lot of info on changes made in LDAP in RHEL6.1 but as far as I understand that info updating should be transparent. And I still have memories of TUV breaking LDAP before, when updating RHEL5.4. to RHEL5.5, which (also?) was not a bug, but a change in configuration...

After the update I could not log in anymore, except with local users. /var/log/messages has lots and lots of these:

[code]Jun 6 12:47:26 hostR6 nslcd[28193]: [8b4567] failed to bind to LDAP server ldaps://ldapserver1/: Can't contact LDAP server
Jun 6 12:47:26 hostR6 nslcd[28193]: [7b23c6] failed to bind to LDAP server ldaps://ldapserver1/: Can't contact LDAP server: Operation now in progress
Jun 6 12:47:26 hostR6 nslcd[28193]: [8b4567] failed to bind to LDAP server ldaps://ldapserver2/: Can't contact LDAP server: Operation now in progress
Jun 6 12:47:26 hostR6 nslcd[28193]: [7b23c6] failed to bind to LDAP server ldaps://ldapserver2/: Can't contact LDAP server: Operation now in progress
Jun 6 12:47:26 hostR6 nslcd[28193]: [7b23c6] failed to bind to LDAP server ldaps://ldapserver3/: Can't contact LDAP server: Operation now in progress
Jun 6 12:47:26 hostR6 nslcd[28193]: [7b23c6] no available LDAP server found, sleeping 1 seconds[/code]
After reading and reading more, not finding anything conclusive, but seeing hints on nss-sysinit, I decided to downgrade nss and openldap again:
[code]# yum downgrade nss nss-softokn* nss-sysinit openldap
Loaded plugins: rhnplugin
Setting up Downgrade Process
No Match for available package: nss-softokn-devel-3.12.7-1.1.el6.i686
No Match for available package: nss-softokn-devel-3.12.7-1.1.el6.x86_64
No Match for available package: nss-softokn-freebl-devel-3.12.9-3.el6.i686
No Match for available package: nss-softokn-freebl-devel-3.12.9-3.el6.x86_64
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.12.8-3.el6_0 will be a downgrade
---> Package nss.x86_64 0:3.12.9-9.el6 will be erased
---> Package nss-softokn.x86_64 0:3.12.8-1.el6_0 will be a downgrade
---> Package nss-softokn.x86_64 0:3.12.9-3.el6 will be erased
---> Package nss-softokn-freebl.i686 0:3.12.8-1.el6_0 will be a downgrade
---> Package nss-softokn-freebl.x86_64 0:3.12.8-1.el6_0 will be a downgrade
---> Package nss-softokn-freebl.i686 0:3.12.9-3.el6 will be erased
---> Package nss-softokn-freebl.x86_64 0:3.12.9-3.el6 will be erased
---> Package nss-sysinit.x86_64 0:3.12.8-3.el6_0 will be a downgrade
---> Package nss-sysinit.x86_64 0:3.12.9-9.el6 will be erased
---> Package openldap.x86_64 0:2.4.19-15.el6_0.2 will be a downgrade
---> Package openldap.x86_64 0:2.4.23-15.el6 will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================================================
Downgrading:
nss x86_64 3.12.8-3.el6_0 rhel-x86_64-server-6 749 k
nss-softokn x86_64 3.12.8-1.el6_0 rhel-x86_64-server-6 166 k
nss-softokn-freebl i686 3.12.8-1.el6_0 rhel-x86_64-server-6 109 k
nss-softokn-freebl x86_64 3.12.8-1.el6_0 rhel-x86_64-server-6 115 k
nss-sysinit x86_64 3.12.8-3.el6_0 rhel-x86_64-server-6 26 k
openldap x86_64 2.4.19-15.el6_0.2 rhel-x86_64-server-6 231 k

Transaction Summary
=================================================================================================================================================================================
Downgrade 6 Package(s)

Total download size: 1.4 M
Is this ok [y/N]: y
Downloading Packages:
(1/6): nss-3.12.8-3.el6_0.x86_64.rpm | 749 kB 00:00
(2/6): nss-softokn-3.12.8-1.el6_0.x86_64.rpm | 166 kB 00:00
(3/6): nss-softokn-freebl-3.12.8-1.el6_0.i686.rpm | 109 kB 00:00
(4/6): nss-softokn-freebl-3.12.8-1.el6_0.x86_64.rpm | 115 kB 00:00
(5/6): nss-sysinit-3.12.8-3.el6_0.x86_64.rpm | 26 kB 00:00
(6/6): openldap-2.4.19-15.el6_0.2.x86_64.rpm | 231 kB 00:00
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 46 kB/s | 1.4 MB 00:30
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : nss-softokn-freebl-3.12.8-1.el6_0.x86_64 1/12
Installing : nss-softokn-3.12.8-1.el6_0.x86_64 2/12
Installing : nss-sysinit-3.12.8-3.el6_0.x86_64 3/12
warning: /etc/pki/nssdb/pkcs11.txt created as /etc/pki/nssdb/pkcs11.txt.rpmnew
Installing : nss-3.12.8-3.el6_0.x86_64 4/12
Installing : openldap-2.4.19-15.el6_0.2.x86_64 5/12
warning: /etc/openldap/ldap.conf created as /etc/openldap/ldap.conf.rpmnew
Installing : nss-softokn-freebl-3.12.8-1.el6_0.i686 6/12
Cleanup : openldap-2.4.23-15.el6.x86_64 7/12
Cleanup : nss-softokn-freebl-3.12.9-3.el6 8/12
Cleanup : nss-sysinit-3.12.9-9.el6.x86_64 9/12
Cleanup : nss-3.12.9-9.el6.x86_64 10/12
Cleanup : nss-softokn-3.12.9-3.el6.x86_64 11/12
Cleanup : nss-softokn-freebl-3.12.9-3.el6 12/12

Removed:
nss.x86_64 0:3.12.9-9.el6 nss-softokn.x86_64 0:3.12.9-3.el6 nss-softokn-freebl.i686 0:3.12.9-3.el6 nss-softokn-freebl.x86_64 0:3.12.9-3.el6
nss-sysinit.x86_64 0:3.12.9-9.el6 openldap.x86_64 0:2.4.23-15.el6

Installed:
nss.x86_64 0:3.12.8-3.el6_0 nss-softokn.x86_64 0:3.12.8-1.el6_0 nss-softokn-freebl.i686 0:3.12.8-1.el6_0 nss-softokn-freebl.x86_64 0:3.12.8-1.el6_0
nss-sysinit.x86_64 0:3.12.8-3.el6_0 openldap.x86_64 0:2.4.19-15.el6_0.2

Complete![/code]
and, after a reboot, everything worked like a charm again.

If I now try an update again, the following packages will be updated:
[code]# yum update
Loaded plugins: rhnplugin
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package nss.x86_64 0:3.12.8-3.el6_0 will be updated
---> Package nss.x86_64 0:3.12.9-9.el6 will be an update
---> Package nss-softokn.x86_64 0:3.12.8-1.el6_0 will be updated
---> Package nss-softokn.x86_64 0:3.12.9-3.el6 will be an update
---> Package nss-softokn-freebl.i686 0:3.12.8-1.el6_0 will be updated
---> Package nss-softokn-freebl.x86_64 0:3.12.8-1.el6_0 will be updated
---> Package nss-softokn-freebl.i686 0:3.12.9-3.el6 will be an update
---> Package nss-softokn-freebl.x86_64 0:3.12.9-3.el6 will be an update
---> Package nss-sysinit.x86_64 0:3.12.8-3.el6_0 will be updated
---> Package nss-sysinit.x86_64 0:3.12.9-9.el6 will be an update
---> Package openldap.x86_64 0:2.4.19-15.el6_0.2 will be updated
---> Package openldap.x86_64 0:2.4.23-15.el6 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Updating:
nss x86_64 3.12.9-9.el6 rhel-x86_64-server-6 766 k
nss-softokn x86_64 3.12.9-3.el6 rhel-x86_64-server-6 170 k
nss-softokn-freebl i686 3.12.9-3.el6 rhel-x86_64-server-6 115 k
nss-softokn-freebl x86_64 3.12.9-3.el6 rhel-x86_64-server-6 122 k
nss-sysinit x86_64 3.12.9-9.el6 rhel-x86_64-server-6 28 k
openldap x86_64 2.4.23-15.el6 rhel-x86_64-server-6 255 k

Transaction Summary
================================================================================
Upgrade 6 Package(s)

Total download size: 1.4 M
Is this ok [y/N]:
Exiting on user Command
Complete![/code]
I have not tried re-updating.

Any thoughts, clues, config changes?
Should I report this as a bug?

User avatar
AlanBartlett
Forum Moderator
Posts: 9319
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by AlanBartlett » 2011/06/14 20:37:00

[quote]
Should I report this as a bug?
[/quote]
If you have carefully checked and found that there is no reference to any feature change in either the [url=http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/6.1_Release_Notes/index.html]RHEL 6.1 Release Notes[/url] or the [url=http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/6.1_Technical_Notes/index.html]RHEL 6.1 Technical Notes[/url] and if what you have shown us is reproducible, then yes, a [url=https://bugzilla.redhat.com/frontpage.cgi]bug report[/url] via [i]Red Hat[/i]'s bug tracker is in order.

As I am not an [i]LDAP[/i] user, I am unable to make any other comment.

I wonder if [b]Scott[/b] has noticed anything?

scottro
Forum Moderator
Posts: 2383
Joined: 2007/09/03 21:18:09
Location: NYC
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by scottro » 2011/06/15 00:06:36

No, I haven't been running LDAP on 6.x servers yet. I do know that RH tends to make changes and not document them, as they did in either the 5.5 or 5.6 update. (See another thread somewhere on these fora).

r_hartman
Posts: 706
Joined: 2009/03/23 15:08:11
Location: Netherlands
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by r_hartman » 2011/06/15 05:16:15

[b]Alan[/b], your first links points, like the last, to bugzilla, not the [url=http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/6.1_Release_Notes/index.html]release notes[/url] :-D

I checked those already, plus a couple of Google hits related to debian. The only remarks on LDAP are referring to some bugfixes and info that the crypto backend was changed from OpenSSL to Mozilla NSS.[quote]The transition should be seamless. OpenLDAP with Mozilla NSS can use all of the PEM cert and key files, and other TLS configuration, that OpenLDAP with OpenSSL used.[/quote]
I like the phrasing "[i]should be[/i]" :-?
There's also some info on LDAP improvements in SSSD, but I did not have to downgrade SSSD to fix it.

I don't use explicit SSL (TLS) but SSL only (implicit, through 'ssl on') so I can block port 389 in my firewall and force port 636 to be used for encrypted traffic only. Given the quoted statement that should not be a problem. And this is only a client. The break in 5.5 concerned slapd configuration changes, server side. I haven't tried opening up port 389, as I don't need that for all my other (RHEL 4 and 5, CentOS 5, Ubuntu 10.04) clients, and the RHEL6 box is only a testbox, not needing it for 6.0.

If 6.1 should require port389 to start TLS, effectively abandoning implicit SSL support, it would break TUV's policy to not require high-impact changes in the Enterprise environment with point-release updates, especially as there's nothing specific in the docs, so my approach for now is that implicit SSL will still be supported, even while it's clearly a connectivity thing.

Looks like I'll need to update again and verify, and probably downgrade again and create a bugzilla account, then.

Oh well...

Thanks for responding, [b]Alan[/b] and [b]Scottro[/b]. When I have new info I'll update this thread, one never knows who's going to run into similar issues.

EDIT:
Updating the packages again and restarting nslcd recreates the problem. Downgrading again and restarting nslcd fixes it.
tcpdump shows it still tries to connect on port ldaps, and even gets replies from the ldap server. But it still says it cannot connect.
nslcd is the local LDAP name service daemon and is provided by the nss-pam-ldapd package, which incidentally is not part of the package-set causing the issue.
I'll go create a bugzilla account.

r_hartman
Posts: 706
Joined: 2009/03/23 15:08:11
Location: Netherlands
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by r_hartman » 2011/06/15 08:06:04

Bug report created with TUV: https://bugzilla.redhat.com/show_bug.cgi?id=713371
Also logged a support case.

scottro
Forum Moderator
Posts: 2383
Joined: 2007/09/03 21:18:09
Location: NYC
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by scottro » 2011/06/15 12:32:37

Thanks. As I maintain my own LDAP page, I've added myself as a cc to the bug.

(My page has nothing to do with RH, but it's the reason Alan wondered if I might have some input. It's at http://home.roadrunner.com/~computertaijutsu/ldap.html but I think it's written for someone who knows a lot less than you do.)

User avatar
AlanBartlett
Forum Moderator
Posts: 9319
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by AlanBartlett » 2011/06/15 17:21:20

[quote]
[b]Alan[/b], your first links points, like the last, to bugzilla, not the release notes :-D
[/quote]
[img]http://www.centos.toracat.org/ajb/tmp/doh.gif[/img] [i]D'oh![/i] Thanks for mentioning it, [b]René[/b]. I've now corrected the link.

r_hartman
Posts: 706
Joined: 2009/03/23 15:08:11
Location: Netherlands
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by r_hartman » 2011/06/16 05:56:08

Well, like before with the server-end change, it appears to be a config change.
I was asked to provide some info in order to verify my bug was or wasn't a duplicate from
https://bugzilla.redhat.com/show_bug.cgi?id=713525
which was solved by taking out the 'tls_cacertdir' reference from /etc/openldap/ldap.conf (sic, I expect that should've been /etc/openldap/pam_ldap.conf).

I didn't have that reference in there, but I did have it in /etc/nslcd.conf. Once I took that out (after updating again, of course) the problem had gone.
Full details in the bugreport: https://bugzilla.redhat.com/show_bug.cgi?id=713371

@[b]scottro[/b]
LDAP continues to be a box of worms, as documentation is both sparse and confusing, and one is pretty much left to his own devices to concoct some working setup.
I set it up with RHEL5.2 and found that I needed to change the nis_schema, adding the 'host' attribute, in order to implement something as basic as discriminating between servers which users should be allowed ssh access to (using 'pam_check_host_attr yes').

There probably are other ways, and my setup is pretty basic, only aimed at providing common ssh and http (subversion) access, so I wouldn't think of myself as an LDAP guru; setting up a complex LDAP directory would probably cause me some nightmares and some more grey hairs. :-D

I thought I had my documentation (LDAPS on RHEL5.2) put up on my website, but it appears I have not. I will do so shortly and put up the link here. I may also try and find time to add some comments on the 5.4-to-5.5 server-side config change, the changes for RHEL6.0 and this little 6.0-to-6.1 bugger. I might start with just putting up the 5.2 docs, though.

I've looked at your pages before, and I must say I was impressed; unfortunately I only found them quite a while (through this forum) after I had done my little setup. I actually might have done some things differently if I had found them first; it would definitely have saved me a lot of time in trial-and-error.

r_hartman
Posts: 706
Joined: 2009/03/23 15:08:11
Location: Netherlands
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by r_hartman » 2011/06/16 10:02:23

Oops! Slightly premature, it would seem. Sorry.

As I had other pressing matters I was satisfied that LDAP users could now be resolved was an indication that authentication would work (based on past exprerience).
Unfortunately, that is not the case. I now get[code]Jun 16 11:26:08 hostR6 sshd[16413]: pam_ldap: reconnecting to LDAP server...
Jun 16 11:26:08 hostR6 sshd[16413]: pam_ldap: ldap_simple_bind Can't contact LDAP server
Jun 16 11:26:08 hostR6 sshd[16414]: fatal: Access denied for user ldapuser by PAM account configuration[/code]

Looks like the downgrade/upgrade only concerned the LDAP resolution. From root I can now do "su - ldapuser" without problems, I just can't establish an ssh session for that user. This definitely worked under RHEL6.0

More details in the bugreport, again.

scottro
Forum Moderator
Posts: 2383
Joined: 2007/09/03 21:18:09
Location: NYC
Contact:

Re: RHEL6.0 --> 6.1 update breaks LDAP?

Post by scottro » 2011/06/16 12:34:09

Thanks for working on this with them. I have to add a link to the report to my page (and thank you for your kind words about it.)

As you've probably seen, Alan and I have added ourselves to be cc'd on the report.

Post Reply