Signing SHAKE256 hash with RSA signature with further validation

Question

I need to hash input data with SHAKE256 (found SeriesOne.CORE package for this) and then generate a digital signature for it based on RSA algorithm.

The problem is that the default RSACryptoServiceProvider does not support any of the SHA3 hashing functions. Methods like RSACryptoServiceProvider.SignHash(), RSACryptoServiceProvider.VerifyData() and RSACryptoServiceProvider.VerifyHash() require hashing algorithm specification. Is there any workaround or maybe I miss something?

Answer 1

.NET currently does not natively support SHA-3 (and thus also not the SHA3 variant SHAKE256), see e.g. here.

Therefore, the native C# methods for signing/verifying do not work either (even the SignHash() method throws a runtime exception if the already hashed data is passed, the digest is specified via the OID, and RSASSA-PKCS1-v1_5 is used, i.e. if no explicit hashing would be required at all: The specified OID (2.16.840.1.101.3.4.2.12) does not represent a known hash algorithm).

Ultimately, therefore, not only the hashing, but for the entire signing process a third-party provider must be used. One possibility is BouncyCastle (as suggested in Maarten Bodewes' comment). For .NET6 BouncyCastle.NetCore should be applied.

For RSASSA-PKCS1-v1_5 the RsaDigestSigner class is to be used with the following in mind:

• The internal mapping of the RSADigestSigner class does not take the SHAKE digests into account, so the RSADigestSigner(Digest digest) ctor will not work and the RSADigestSigner(Digest digest, ASN1ObjectIdentifier digestOid) ctor must be used, i.e. the OID (2.16.840.1.101.3.4.2.12) must be explicitly specified. This can be done either with new DerObjectIdentifier("2.16.840.1.101.3.4.2.12") or with NistObjectIdentifiers.IdShake256. The DER encoding of the DigestInfo value is thus 3031300D060960864801650304020C05000420 (which differs from the SHA56 value only in the 15th byte with 0x0c instead of 0x01, s. RFC8017).
• The SHAKE digests have a variable output length. In the implementation below, a fixed output length of 32 bytes is applied for SHAKE256.

A possible implementation is:

using Org.BouncyCastle.Asn1.Nist;
using Org.BouncyCastle.Crypto.Digests;
using Org.BouncyCastle.Crypto.Signers;
...
RsaDigestSigner signer = new RsaDigestSigner(new ShakeDigest(256), NistObjectIdentifiers.IdShake256);
signer.Init(true, privateKeyParameter);
signer.BlockUpdate(dataToSign, 0, dataToSign.Length);
byte[] signature = signer.GenerateSignature();
...

For RSASSA-PSS the PssSigner class is to be used. The following implementation applies SHAKE256 for PSS and MGF1 digest, and as salt length 32 bytes (but there are also constructors to set the parameters explicitly):

using Org.BouncyCastle.Crypto.Engines;
using Org.BouncyCastle.Crypto.Digests;
using Org.BouncyCastle.Crypto.Signers;
...
PssSigner pssSigner = new PssSigner(new RsaEngine(), new ShakeDigest(256));
//PssSigner pssSigner = new PssSigner(new RsaEngine(), new ShakeDigest(256), new ShakeDigest(256), 32); // works also
pssSigner.Init(true, privateKeyParameter);
pssSigner.BlockUpdate(dataToSign, 0, dataToSign.Length);
byte[] pssSignature = pssSigner.GenerateSignature();