"Libgcrypt warning: MD5 used - FIPS mode inactivated" after

[a coder]

Steps to reproduce:

Enable openSSH FIPS 140-2 module using these instructions.

1) edit /etc/sysconfig/prelink and set PRELINKING=NO. Issue prelink -u -a at a prompt.
2) yum install dracut-fips
3) dracut -f
4) add "fips=1" and "boot=/dev/sda3" to kernel line of grub.conf. df /boot revealed the correct boot partion.
5) ensure /etc/ssh/sshd_config is configured with:

Code: Select all

Protocol 2
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
Macs hmac-sha1,hmac-sha2-256,hmac-sha2-512

After rebooting, I confirmed that FIPS mode is enabled by using

Code: Select all

openssl md5 somefile (fails)

and

Code: Select all

openssl sha1 somefile (succeeds)

Also:

Code: Select all

$ cat /proc/sys/crypto/fips_enabled
1

Finally, knowing that FIPS is enabled, I attempted to connect to a remote SQL Server instance with a config that worked prior to enabling FIPS:

Code: Select all

[mybox ~]# tsql -S egServer80 -U myusername
Password:
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
Error 20002 (severity 9):
    Adaptive Server connection failed
There was a problem connecting to the server

I checked the log files and find this:

Code: Select all

tsql: Libgcrypt warning: MD5 used - FIPS mode inactivated

Enabling debug in freetds yielded this additional error:

Code: Select all

14:56:46.617196 3577 (net.c:1366):'''handshake failed: GnuTLS internal error.

Additional Information:
Backing out the FIPS module (removing fips=1 from grub.conf) and rebooting sets things back to normal (I was able to tsql into my SQL Server instance again).

I can reproduce the same libgcrypt/tsql problem without enabling FIPS 140-2 module in grub, by creating an empty file /etc/gcrypt/fips_enabled. Removing this file sets the system back to normal, and tsql works again.

CentOS version 6.7
libgcrypt version 1.4.5
freetds version 0.91
openssl version 1.0.1e


Why (or how) is enabling FIPS in grub causing `libgcrypt` to fail on this one machine?

[a coder]

Can anyone take a crack at this?

[drk]

Off hand I'd say it is your remote server that is the problem.

[a coder]

Off hand I'd say it is your remote server that is the problem.

A second server running CentOS 6.7 with FIPS enabled (and confirmed working) does not have this problem. I don't think it's the remote server. It's not an easily reproduceable problem - I've installed CentOS 6.7 in a VM, enabled FIPS, and am also able to use libgcrypt without issue.

I have a feeling there is a configuration issue, but am not sure what to look for. The error log shows the same libgcrypt warning associated with tsql, httpd and others.

Is there a less known config file for libgcrypt? It's trying to use MD5 when FIPS is enabled. If I can force it not to use MD5 (SHA instead), I'd be in business.