PKCS11Exception: CKR_MECHANISM_INVALID How to integrate Java and PKCS11 and GCP Cloud KMS?

Question

I'm trying to use GCP Cloud KMS for establishing mTLS. For this purpose I'm preparing PoC of integration Java + JCE + PKCS11 + Cloud KMS integration lib + GCP Cloud KMS. Stack:

• OpenJDK 1.8
• SunPKCS11 Provider
• https://github.com/GoogleCloudPlatform/kms-integrations
• GCP Cloud KMS

As a result I have an exception:

Private key:  SunPKCS11-KmsHSM RSA private key, 3072 bitstoken object, sensitive, unextractable)
I20220812 18:55:30.725561    12 logging.cc:137] returning 0x70 from C_DecryptInit due to status INVALID_ARGUMENT: at crypter_ops.cc:33: mechanism 0x1 is not valid for operation decrypt [type.googleapis.com/kmsp11.StatusDetails='CK_RV=0x70']
I20220812 18:55:30.726071    12 logging.cc:137] returning 0x70 from C_DecryptInit due to status INVALID_ARGUMENT: at crypter_ops.cc:33: mechanism 0x1 is not valid for operation decrypt [type.googleapis.com/kmsp11.StatusDetails='CK_RV=0x70']
java.security.InvalidKeyException: init() failed
    at sun.security.pkcs11.P11RSACipher.implInit(P11RSACipher.java:239)
    at sun.security.pkcs11.P11RSACipher.engineInit(P11RSACipher.java:168)
    at javax.crypto.Cipher.implInit(Cipher.java:805)
    at javax.crypto.Cipher.chooseProvider(Cipher.java:867)
    at javax.crypto.Cipher.init(Cipher.java:1252)
    at javax.crypto.Cipher.init(Cipher.java:1189)
    at engsec.cmds.EncryptKMS.run(EncryptKMS.java:73)
    at engsec.App.main(App.java:34)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_MECHANISM_INVALID
    at sun.security.pkcs11.wrapper.PKCS11.C_DecryptInit(Native Method)
    at sun.security.pkcs11.P11RSACipher.initialize(P11RSACipher.java:323)
    at sun.security.pkcs11.P11RSACipher.implInit(P11RSACipher.java:237)
    ... 7 more

The code is below:

            KeyStore ks = KeyStore.getInstance("PKCS11");
            ks.load(null, null);

            Key privateKey = ks.getKey(
                    PRIVATE_KEY_ALIAS,
                    null
            );

            Cipher cipher2 = Cipher.getInstance("RSA");
            cipher2.init(
                    Cipher.DECRYPT_MODE,
                    privateKey
            );
            byte[] clearText = cipher2.doFinal(cipherBytes);

            System.out.println("Decrypted value: ", new String(clearText));

How to fix the exception? Is it possible to integrate Java Ecosystem and Cloud KMS for establishing mTLS? Thank you!

Answer 1

Assuming that the key at PRIVATE_KEY_ALIAS refers to a signing key in Cloud KMS (that is, a key with the ASYMMETRIC_SIGN purpose), then it cannot be used for decryption.