OpenSSL 1.1.1 PSK TLS1.3 - no suitable signature algorithm error with TLS_256_GCM_SHA384 ciphersuite

Question

I'm running some tests in windows with OpenSSL 1.1.1l 24 Aug 2021, using s_client and s_server for both PSK TLS_128_GCM_SHA256 and TLS_256_GCM_SHA384.

For PSK TLS_128_GCM_SHA256 i'm able to stablish the connection sucessfully:

SERVER:

$openssl s_server -nocert -psk 1234567890ABCDEF -tls1_3 -ciphersuites TLS_AES_128_GCM_SHA256

CLIENT

openssl s_client -psk 1234567890ABCDEF -tls1_3 -ciphersuites  TLS_AES_128_GCM_SHA256

However, I'm not able to successfully stablish a connection for the ciphersuite TLS_AES_256_GCM_384

SERVER

$openssl s_server -psk 63ef2024b1de6417f856fab7005d38f6df70b6c5e97c220060e2ea122c4fdd054555827ab229457c366b2dd4817ff38b -ciphersuites TLS_AES_256_GCM_SHA384 -nocert -tls1_3

Using default temp DH parameters
ACCEPT
ERROR
30508:error:14201076:SSL routines:tls_choose_sigalg:no suitable signature algorithm:ssl\t1_lib.c:2760:
shutting down SSL
CONNECTION CLOSED

CLIENT

$openssl s_client -psk 63ef2024b1de6417f856fab7005d38f6df70b6c5e97c220060e2ea122c4fdd054555827ab229457c366b2dd4817ff38b -ciphersuites TLS_AES_256_GCM_SHA384 -tls1_3
CONNECTED(000001C4)
32968:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl\record\rec_layer_s3.c:1544:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 291 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

What am i missing? Thanks!

Answer 1

A PSK in TLSv1.3 must be associated with a hash. By default s_server and s_client will use SHA256. The TLS_AES_256_GCM_SHA384 ciphersuite uses SHA384 and is therefore not compatible with the SHA256 associated with the PSK.

Unfortunately there isn't an option to tell s_client/s_server to use a different hash with the PSK. However there is a work around. You can instead use the -psk_session option which takes a session file as an argument. You can construct a session file using a regular (non-PSK) connection like this:

$ openssl s_server -ciphersuites TLS_AES_256_GCM_SHA384 -tls1_3 -cert cert.pem -key key.pem
$ openssl s_client -ciphersuites TLS_AES_256_GCM_SHA384 -tls1_3 -sess_out sess.pem

This saves the key from the connection in the session file and associates it with the hash for the ciphersuite (SHA384). Then you can use the session file as the PSK like this:

$ openssl s_server -ciphersuites TLS_AES_256_GCM_SHA384 -tls1_3 -psk_session sess.pem -nocert
$ openssl s_client -ciphersuites TLS_AES_256_GCM_SHA384 -tls1_3 -psk_session sess.pem