"decryption failed or bad record mac" between apache and Java

Question

I have a Java program (running inside a tomcat container) which uses RSA SSL-J implementation for SSL, and an apache webserver which is configured for SSL using mod_ssl/openssl

When the Java program tried to open a HttpsUrlConnection to the apache server, errors with javax.net.ssl.SSLException: Fatal Alert received: Bad Record Mac

(The exception stacktrace is not very helpful since sslj.jar is obfuscated)

The issue is not intermittent. It happens every time.

This is from the mod_ssl logs on apache after I set LogLevel to debug :

[Mon Jul 30 22:00:25 2012] [debug] ssl_engine_kernel.c(1884): OpenSSL: Write: SSLv3 read certificate verify A
[Mon Jul 30 22:00:25 2012] [debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read certificate verify A`
[Mon Jul 30 22:00:25 2012] [debug] ssl_engine_kernel.c(1903): OpenSSL: Exit: error in SSLv3 read certificate verify A
[Mon Jul 30 22:00:25 2012] [info] [client 172.16.195.208] %%CryptoAuditEntry:: SSL library error 1 in handshake (server <HOSTNAME>:9005)
[Mon Jul 30 22:00:25 2012] [info] %%CryptoAuditEntry:: SSL Library Error: 336130329 error:1408F119:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac
[Mon Jul 30 22:00:25 2012] [info] [client <IPADDRESS>] %%CryptoAuditEntry:: Connection closed to child 4 with abortive shutdown (server <HOSTNAME>:9005)

This is the java side code :

                SSLContext sc = SSLContext.getInstance("TLS");
                sc.init(null /*keyManagers*/, trustAllCerts,
                          new java.security.SecureRandom());
                HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
                HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();


                // Don't check the hostname against the certificate name
                conn.setHostnameVerifier(new HostnameVerifier() {
                        public boolean verify(String urlHostname,
                            SSLSession session) {
                            return true;
                        }
                    });
                conn.setDoInput(true);
                conn.setDoOutput(true);
                conn.setUseCaches(false);
                conn.setRequestProperty("METHOD", "POST");
                conn.setRequestProperty("Authorization", "Basic " +
                    credentials);
                conn.setRequestProperty("Content-Type", "application/pkcs10");
                conn.setReadTimeout(8000);
                conn.connect();

Another interesting fact is that I can use openssl to connect to server using this command, without any problems.

openssl s_client -connect HOST:PORT

Any pointers?

Answer 1

I was seeing this error from certain java clients only. Adjusting the SSLProtocol in the apache's virtual host config to only allow TLS allowed the clients to connect.