LWP::UserAgent for XML POST to HTTPS server not working

Question

OK... let's see if this is any clearer.

I'm doing a POST of an XML doc via LWP::UserAgent to a remote third-party server.

They previously sent me their .crt files, which I installed on my server, and verified their address and the fact that they exist, through CPanel and via Putty command line "locate" calls. The certs are there.

I ensured that I have LWP::Protocol::https and all other necessary Perl mods installed, including Mozilla::CA, Crypt::SSLeay, and others.

When I run the script, I get the following error:

An Error Occurred

500 Can't connect to shortURL:port (certificate verify failed) 500 Can't connect to shortURL:port (certificate verify failed) Content-Type: text/plain Client-Date: Wed, 25 Jan 2012 17:53:36 GMT Client-Warning: Internal response Can't connect to shortURL:port (certificate verify failed) LWP::Protocol::https::Socket: SSL connect attempt failed with unknown errorerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed at /usr/lib/perl5/site_perl/5.8.8/LWP/Protocol/http.pm line 51.

Here is the calling code:

sub ConsumerInfo {
my $cid = shift;

my $response = undef;
my $sendXML = &Create_ConsumerInfo_Request($cid);
if ($sendXML) {
    &DoXMLUpload($sendXML);

    my $browser = LWP::UserAgent->new(agent => 'site_perl',
        ssl_opts => {
            verify_hostname => 1,
            SSL_ca_path => '/home/<mysite>/ssl/certs/',
        },);
    $browser->credentials('shortURL:port','realm','username'=>'password');
    $response = $browser->request(POST 'longURL',
        Content_Type => 'text/xml',
        Content => $sendXML);
    print "Content-type:text/html\n\n";
    print $response->error_as_HTML unless $response->is_success;
    print $response->as_string;
} else {
    &ErrorMsg("No XML Code Was Found.");
    exit;   
}
# ===============================================================
# Need to insert parser in here to convert this into an array.
# ===============================================================
return $response;
}

Any thoughts? I'm running out of ideas here. :(

Answer 1

rm -f /home/site/ssl/certs/*

wget --no-check-certificate \
             -O /home/site/ssl/certs/DigiCertHighAssuranceCA-3.crt.pem \
    https://www.digicert.com/CACerts/DigiCertHighAssuranceCA-3.crt

wget --no-check-certificate \
              -O /home/site/ssl/certs/DigiCertHighAssuranceEVRootCA.crt.pem \
    https://www.digicert.com/testroot/DigiCertHighAssuranceEVRootCA.crt

c_rehash /home/site/ssl/certs/

echo -n | openssl s_client -connect previewtest.clverify.com:443 -CApath /home/site/ssl/certs

Third last line says now Verify return code: 0 (ok).

echo -n | HTTPS_CA_DIR=/home/site/ssl/certs lwp-request -m POST https://previewtest.clverify.com/webservice/exec

Response goes now past the crypto step and returns a Tomcat error message.

Perl code with LWP and SSL_ca_path as shown works, too, at least the crypto part.

---

Broad hint: sit down with the OpenSSL docs and perhaps a relevant book or two and do a thorough RTFM to learn the underlying concepts. If you just go ahead and implement this without understanding what it really does, then one day someone MITMs your transmissions and you don't even notice that something is wrong, and then it will be your head on the line.