I am battling a little problem with OpenSSL mutually authenticated DTLS client handshake which I should be able to solve for both OpenSSL 1.0 and 1.1.
In a nutshell the issue is that if node and peer certificates are properly loaded when the handshake is initiated the handshake is successful and DTLS session works. But in case the handshake is initiated before the certificates are loaded I cannot seem to find a way to get the handshake completed once the certificates are loaded which would also seem to prohibit renewing the certificates, atleast if trying to use the same SSL context.
I tried replacing X509_STORE for the SSL context when loading the certificates and calling SSL_clear or SSL_renegotiate when the peer verify fails but it would seem that SSL3_MT_CERTIFICATE nor peer verify do not reoccur once they have failed.
Any suggestions where to look next or is it even possible to get this working using the same SSL context?
Ok I identified the reason for this problem. After certificate verificatio failure the client and server do not succeed with secure renegotiation as described in RFC 5746.