Encrypt message for Web Push API in Java

Question

I'm trying to create a server capable of sending push messages using the Push API: https://developer.mozilla.org/en-US/docs/Web/API/Push_API

I've got the client side working but now I want to be able to send messages with a payload from a Java server.

I saw the nodejs web-push example (https://www.npmjs.com/package/web-push) but I couldn't translate that correctly to Java.

I tried following the example to use the DH key exchange found here: http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#DH2Ex

With the help of sheltond below I was able to figure out some code that should be working but isn't.

When I post the encrypted message to the Push service, I get back the expected 201 status code but the push never reaches Firefox. If I remove the payload and headers and simply send a POST request to the same URL the message successfully arrives in Firefox with no data. I suspect it may have something to do with the way I'm encrypting the data with Cipher.getInstance("AES/GCM/NoPadding");

This is the code I'm using currently:

try {
    final byte[] alicePubKeyEnc = Util.fromBase64("BASE_64_PUBLIC_KEY_FROM_PUSH_SUBSCRIPTION");
    KeyPairGenerator kpg = KeyPairGenerator.getInstance("EC");
    ECGenParameterSpec kpgparams = new ECGenParameterSpec("secp256r1");
    kpg.initialize(kpgparams);

    ECParameterSpec params = ((ECPublicKey) kpg.generateKeyPair().getPublic()).getParams();
    final ECPublicKey alicePubKey = fromUncompressedPoint(alicePubKeyEnc, params);
    KeyPairGenerator bobKpairGen = KeyPairGenerator.getInstance("EC");
    bobKpairGen.initialize(params);

    KeyPair bobKpair = bobKpairGen.generateKeyPair();
    KeyAgreement bobKeyAgree = KeyAgreement.getInstance("ECDH");
    bobKeyAgree.init(bobKpair.getPrivate());


    byte[] bobPubKeyEnc = toUncompressedPoint((ECPublicKey) bobKpair.getPublic());


    bobKeyAgree.doPhase(alicePubKey, true);
    Cipher bobCipher = Cipher.getInstance("AES/GCM/NoPadding");
    SecretKey bobDesKey = bobKeyAgree.generateSecret("AES");
    byte[] saltBytes = new byte[16];
    new SecureRandom().nextBytes(saltBytes);
    Mac extract = Mac.getInstance("HmacSHA256");
    extract.init(new SecretKeySpec(saltBytes, "HmacSHA256"));
    final byte[] prk = extract.doFinal(bobDesKey.getEncoded());

    // Expand
    Mac expand = Mac.getInstance("HmacSHA256");
    expand.init(new SecretKeySpec(prk, "HmacSHA256"));
    String info = "Content-Encoding: aesgcm128";
    expand.update(info.getBytes(StandardCharsets.US_ASCII));
    expand.update((byte) 1);
    final byte[] key_bytes = expand.doFinal();

    // Use the result
    SecretKeySpec key = new SecretKeySpec(key_bytes, 0, 16, "AES");
    bobCipher.init(Cipher.ENCRYPT_MODE, key);

    byte[] cleartext = "{\"this\":\"is a test that is supposed to be working but it is not\"}".getBytes();
    byte[] ciphertext = bobCipher.doFinal(cleartext);

    URL url = new URL("PUSH_ENDPOINT_URL");
    HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
    urlConnection.setRequestMethod("POST");
    urlConnection.setRequestProperty("Content-Length", ciphertext.length + "");
    urlConnection.setRequestProperty("Content-Type", "application/octet-stream");
    urlConnection.setRequestProperty("Encryption-Key", "keyid=p256dh;dh=" + Util.toBase64UrlSafe(bobPubKeyEnc));
    urlConnection.setRequestProperty("Encryption", "keyid=p256dh;salt=" + Util.toBase64UrlSafe(saltBytes));
    urlConnection.setRequestProperty("Content-Encoding", "aesgcm128");
    urlConnection.setDoInput(true);
    urlConnection.setDoOutput(true);
    final OutputStream outputStream = urlConnection.getOutputStream();
    outputStream.write(ciphertext);
    outputStream.flush();
    outputStream.close();
    if (urlConnection.getResponseCode() == 201) {
        String result = Util.readStream(urlConnection.getInputStream());
        Log.v("PUSH", "OK: " + result);
    } else {
        InputStream errorStream = urlConnection.getErrorStream();
        String error = Util.readStream(errorStream);
        Log.v("PUSH", "Not OK: " + error);
    }
} catch (Exception e) {
    Log.v("PUSH", "Not OK: " + e.toString());
}

where "BASE_64_PUBLIC_KEY_FROM_PUSH_SUBSCRIPTION" is the key the Push API subscription method in the browser provided and "PUSH_ENDPOINT_URL" is the push endpoint the browser provided.

If I get values (ciphertext, base64 bobPubKeyEnc and salt) from a successful nodejs web-push request and hard-code them in Java, it works. If I use the code above with dynamic values it does not work.

I did notice that the ciphertext that worked in the nodejs implementation is always 1 byte bigger then the Java ciphertext with the code above. The example I used here always produces a 81 byte cipher text but in nodejs it's always 82 bytes for example. Does this give us a clue on what might be wrong?

How do I correctly encrypt the payload so that it reaches Firefox?

Thanks in advance for any help

Answer 1

Able to receive notifications after changing code as per https://jrconlin.github.io/WebPushDataTestPage/

Find the modified code below :

 


import com.sun.org.apache.xerces.internal.impl.dv.util.Base64;
import java.io.BufferedInputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.math.BigInteger;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.security.KeyFactory;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.SecureRandom;
import java.security.Security;
import java.security.interfaces.ECPublicKey;
import java.security.spec.ECFieldFp;
import java.security.spec.ECParameterSpec;
import java.security.spec.ECPoint;
import java.security.spec.ECPublicKeySpec;
import java.security.spec.EllipticCurve;
import java.util.Arrays;
import javax.crypto.Cipher;
import javax.crypto.KeyAgreement;
import javax.crypto.Mac;
import javax.crypto.SecretKey;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import org.bouncycastle.jce.provider.BouncyCastleProvider;


public class WebPushEncryption {

    private static final byte UNCOMPRESSED_POINT_INDICATOR = 0x04;
    private static final ECParameterSpec params = new ECParameterSpec(
            new EllipticCurve(new ECFieldFp(new BigInteger(
                                    "FFFFFFFF00000001000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFF",
                                    16)), new BigInteger(
                            "FFFFFFFF00000001000000000000000000000000FFFFFFFFFFFFFFFFFFFFFFFC",
                            16), new BigInteger(
                            "5AC635D8AA3A93E7B3EBBD55769886BC651D06B0CC53B0F63BCE3C3E27D2604B",
                            16)), new ECPoint(new BigInteger(
                            "6B17D1F2E12C4247F8BCE6E563A440F277037D812DEB33A0F4A13945D898C296",
                            16), new BigInteger(
                            "4FE342E2FE1A7F9B8EE7EB4A7C0F9E162BCE33576B315ECECBB6406837BF51F5",
                            16)), new BigInteger(
                    "FFFFFFFF00000000FFFFFFFFFFFFFFFFBCE6FAADA7179E84F3B9CAC2FC632551",
                    16), 1);

    public static void main(String[] args) throws Exception {
        Security.addProvider(new BouncyCastleProvider());
        String endpoint = "https://updates.push.services.mozilla.com/push/v1/xxx";
        final byte[] alicePubKeyEnc = Base64.decode("base64 encoded public key ");

        KeyPairGenerator keyGen = KeyPairGenerator.getInstance("ECDH", "BC");
        keyGen.initialize(params);

        KeyPair bobKpair = keyGen.generateKeyPair();
        PrivateKey localPrivateKey = bobKpair.getPrivate();
        PublicKey localpublickey = bobKpair.getPublic();

        final ECPublicKey remoteKey = fromUncompressedPoint(alicePubKeyEnc, params);

        KeyAgreement bobKeyAgree = KeyAgreement.getInstance("ECDH", "BC");
        bobKeyAgree.init(localPrivateKey);

        byte[] bobPubKeyEnc = toUncompressedPoint((ECPublicKey) bobKpair.getPublic());

        bobKeyAgree.doPhase(remoteKey, true);

        SecretKey bobDesKey = bobKeyAgree.generateSecret("AES");

        byte[] saltBytes = new byte[16];
        new SecureRandom().nextBytes(saltBytes);

        Mac extract = Mac.getInstance("HmacSHA256", "BC");
        extract.init(new SecretKeySpec(saltBytes, "HmacSHA256"));
        final byte[] prk = extract.doFinal(bobDesKey.getEncoded());

        // Expand
        Mac expand = Mac.getInstance("HmacSHA256", "BC");
        expand.init(new SecretKeySpec(prk, "HmacSHA256"));

        //aes algorithm
        String info = "Content-Encoding: aesgcm128";
        expand.update(info.getBytes(StandardCharsets.US_ASCII));
        expand.update((byte) 1);
        final byte[] key_bytes = expand.doFinal();
        byte[] key_bytes16 = Arrays.copyOf(key_bytes, 16);
        SecretKeySpec key = new SecretKeySpec(key_bytes16, 0, 16, "AES-GCM");

        //nonce
        expand.reset();
        expand.init(new SecretKeySpec(prk, "HmacSHA256"));
        String nonceinfo = "Content-Encoding: nonce";
        expand.update(nonceinfo.getBytes(StandardCharsets.US_ASCII));
        expand.update((byte) 1);
        final byte[] nonce_bytes = expand.doFinal();
        byte[] nonce_bytes12 = Arrays.copyOf(nonce_bytes, 12);

        Cipher bobCipher = Cipher.getInstance("AES/GCM/NoPadding", "BC");

        byte[] iv = generateNonce(nonce_bytes12, 0);

        bobCipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));

        byte[] cleartext = ("{\n"
                + "      \"message\" : \"great match41eeee!\",\n"
                + "      \"title\" : \"Portugal vs. Denmark4255\",\n"
                + "      \"icon\" : \"http://icons.iconarchive.com/icons/artdesigner/tweet-my-web/256/single-bird-icon.png\",\n"
                + "   \"tag\" : \"testtag1\",\n"
                + "   \"url\" : \"http://www.yahoo.com\"\n"
                + "    }").getBytes();

        byte[] cc = new byte[cleartext.length + 1];
        cc[0] = 0;

        for (int i = 0; i < cleartext.length; i++) {
            cc[i + 1] = cleartext[i];
        }

        cleartext = cc;
        byte[] ciphertext = bobCipher.doFinal(cleartext);

        URL url = new URL(endpoint);
        HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
        urlConnection.setRequestMethod("POST");
        urlConnection.setRequestProperty("Content-Length", ciphertext.length + "");
        urlConnection.setRequestProperty("Content-Type", "application/octet-stream");
        urlConnection.setRequestProperty("encryption-key", "keyid=p256dh;dh=" + Base64.encode(bobPubKeyEnc));
        urlConnection.setRequestProperty("encryption", "keyid=p256dh;salt=" + Base64.encode(saltBytes));
        urlConnection.setRequestProperty("content-encoding", "aesgcm128");
        urlConnection.setRequestProperty("ttl", "60");
        urlConnection.setDoInput(true);
        urlConnection.setDoOutput(true);
        final OutputStream outputStream = urlConnection.getOutputStream();
        outputStream.write(ciphertext);
        outputStream.flush();
        outputStream.close();
        if (urlConnection.getResponseCode() == 201) {
            String result = readStream(urlConnection.getInputStream());
            System.out.println("PUSH OK: " + result);
        } else {
            InputStream errorStream = urlConnection.getErrorStream();
            String error = readStream(errorStream);
            System.out.println("PUSH" + "Not OK: " + error);
        }
    }

    static byte[] generateNonce(byte[] base, int index) {
        byte[] nonce = Arrays.copyOfRange(base, 0, 12);

        for (int i = 0; i < 6; ++i) {
            nonce[nonce.length - 1 - i] ^= (byte) ((index / Math.pow(256, i))) & (0xff);
        }
        return nonce;
    }

    private static String readStream(InputStream errorStream) throws Exception {
        BufferedInputStream bs = new BufferedInputStream(errorStream);
        int i = 0;
        byte[] b = new byte[1024];
        StringBuilder sb = new StringBuilder();
        while ((i = bs.read(b)) != -1) {
            sb.append(new String(b, 0, i));
        }

        return sb.toString();
    }

    public static ECPublicKey fromUncompressedPoint(
            final byte[] uncompressedPoint, final ECParameterSpec params)
            throws Exception {

        int offset = 0;
        if (uncompressedPoint[offset++] != UNCOMPRESSED_POINT_INDICATOR) {
            throw new IllegalArgumentException(
                    "Invalid uncompressedPoint encoding, no uncompressed point indicator");
        }

        int keySizeBytes = (params.getOrder().bitLength() + Byte.SIZE - 1)
                / Byte.SIZE;

        if (uncompressedPoint.length != 1 + 2 * keySizeBytes) {
            throw new IllegalArgumentException(
                    "Invalid uncompressedPoint encoding, not the correct size");
        }

        final BigInteger x = new BigInteger(1, Arrays.copyOfRange(
                uncompressedPoint, offset, offset + keySizeBytes));
        offset += keySizeBytes;
        final BigInteger y = new BigInteger(1, Arrays.copyOfRange(
                uncompressedPoint, offset, offset + keySizeBytes));
        final ECPoint w = new ECPoint(x, y);
        final ECPublicKeySpec ecPublicKeySpec = new ECPublicKeySpec(w, params);
        final KeyFactory keyFactory = KeyFactory.getInstance("EC");
        return (ECPublicKey) keyFactory.generatePublic(ecPublicKeySpec);
    }

    public static byte[] toUncompressedPoint(final ECPublicKey publicKey) {

        int keySizeBytes = (publicKey.getParams().getOrder().bitLength() + Byte.SIZE - 1)
                / Byte.SIZE;

        final byte[] uncompressedPoint = new byte[1 + 2 * keySizeBytes];
        int offset = 0;
        uncompressedPoint[offset++] = 0x04;

        final byte[] x = publicKey.getW().getAffineX().toByteArray();
        if (x.length <= keySizeBytes) {
            System.arraycopy(x, 0, uncompressedPoint, offset + keySizeBytes
                    - x.length, x.length);
        } else if (x.length == keySizeBytes + 1 && x[0] == 0) {
            System.arraycopy(x, 1, uncompressedPoint, offset, keySizeBytes);
        } else {
            throw new IllegalStateException("x value is too large");
        }
        offset += keySizeBytes;

        final byte[] y = publicKey.getW().getAffineY().toByteArray();
        if (y.length <= keySizeBytes) {
            System.arraycopy(y, 0, uncompressedPoint, offset + keySizeBytes
                    - y.length, y.length);
        } else if (y.length == keySizeBytes + 1 && y[0] == 0) {
            System.arraycopy(y, 1, uncompressedPoint, offset, keySizeBytes);
        } else {
            throw new IllegalStateException("y value is too large");
        }

        return uncompressedPoint;
    }
}

Answer 2

Have a look at the answer from Maarten Bodewes in this question.

He gives Java source for encoding/decoding from the X9.62 uncompressed format into an ECPublicKey, which I think should be suitable for what you're trying to do.

== Update 1 ==

The spec says "User Agents that enforce encryption MUST expose an elliptic curve Diffie-Hellman share on the P-256 curve".

The P-256 curve is a standard curve approved by NIST for use in US government encryption applications. The definition, parameter values and rationale for choosing this particular curve (along with a few others) are given here.

There is support for this curve in the standard library using the name "secp256r1", but for reasons that I haven't been able to fully work out (I think it's to do with the separation of cryptography providers from the JDK itself), you seem to have to jump through some very inefficient hoops to get one of these ECParameterSpec values from this name:

KeyPairGenerator kpg = KeyPairGenerator.getInstance("EC");
ECGenParameterSpec kpgparams = new ECGenParameterSpec("secp256r1");
kpg.initialize(kpgparams);
ECParameterSpec params = ((ECPublicKey) kpg.generateKeyPair().getPublic()).getParams();

This is pretty heavyweight because it actually generates a keypair using the named ECGenParameterSpec object, then extracts the ECParameterSpec from it. You should then be able to use this to decode (I'd recommend caching this value somewhere to avoid having to do this key-generation frequently).

Alternatively, you can just take the numbers from page 8 of the NIST document and plug them in directly to the ECParameterSpec constructor.

There is some code here which looks like it does exactly that (around line 124). That code is Apache licensed. I haven't used that code myself, but it looks like the constants match what's in the NIST document.

== Update 2 ==

The actual encryption key is derived from the salt (randomly generated) and the shared secret (agreed by the DH key exchange), using the HMAC-based key derivation function (HKDF) described in section 3.2 of Encrypted Content-Encoding for HTTP.

That document references RFC 5869 and specifies the use of SHA-256 as the hash used in the HKDF.

This RFC describes a two stage process: Extract and Expand. The Extract phase is defined as:

PRK = HMAC-Hash(salt, IKM)

In the case of web-push, this should be an HMAC-SHA-256 operation, the salt value should be the "saltBytes" value that you already have, and as far as I can see the IKM value should be the shared secret (the webpush document just says "These values are used to calculate the content encryption key" without specifically stating that the shared secret is the IKM).

The Expand phase takes the value produced by the Extract phase plus an 'info' value, and repeatedly HMACs them until it has produced enough key data for the encryption algorithm that you're using (the output of each HMAC is fed into the next one - see the RFC for details).

In this case, the algorithm is AEAD_AES_128_GCM which requires a 128-bit key, which is smaller than the output of SHA-256, so you only need to do one hash in the Expand stage.

The 'info' value in this case has to be "Content-Encoding: aesgcm128" (specified in Encrypted Content-Encoding for HTTP), so the operation that you need is:

HMAC-SHA-256(PRK, "Content-Encoding: aesgcm128" | 0x01)

where the '|' is concatenation. You then take the first 16 bytes of the result, and that should be the encryption key.

In Java terms, that would look something like:

// Extract
Mac extract = Mac.getInstance("HmacSHA256");
extract.init(new SecretKeySpec(saltBytes, "HmacSHA256"));
final byte[] prk = extract.doFinal(bobDesKey.getEncoded());

// Expand
Mac expand = Mac.getInstance("HmacSHA256");
expand.init(new SecretKeySpec(prk, "HmacSHA256"));
String info = "Content-Encoding: aesgcm128";
expand.update(info.getBytes(StandardCharsets.US_ASCII));
expand.update((byte)1);
final byte[] key_bytes = expand.doFinal();

// Use the result
SecretKeySpec key = new SecretKeySpec(key_bytes, 0, 16, "AES");
bobCipher.init(Cipher.ENCRYPT_MODE, key);

For reference, here's a link to the part of the BouncyCastle library that does this stuff.

Finally, I just noticed this part in the webpush document:

Public keys, such as are encoded into the "dh" parameter, MUST be in the form of an uncompressed point

so it looks like you will need to use something like this:

byte[] bobPubKeyEnc = toUncompressedPoint((ECPublicKey)bobKpair.getPublic());

instead of using the standard getEncoded() method.

== Update 3 ==

First, I should point out that there is a more recent draft of the spec for http content encryption than the one that I have previous linked to: draft-ietf-httpbis-encryption-encoding-00. People who want to use this system should make sure that they are using the latest available draft of the spec - this is work in progress and seems to be changing slightly every few months.

Second, in section 2 of that document, it specifies that some padding must be added to the plaintext before encryption (and removed after decryption).

This would account for the one byte difference in length between what you mentioned that you're getting and what the Node.js example produces.

The document says:

Each record contains between 1 and 256 octets of padding, inserted into a record before the enciphered content. Padding consists of a length byte, followed that number of zero-valued octets. A receiver MUST fail to decrypt if any padding octet other than the first is non-zero, or a record has more padding than the record size can accommodate.

So I think what you need to do is to push a single '0' byte into the cipher before your plaintext. You could add more padding than that - I couldn't see anything that specified that the padding must be the minimum amount possible, but a single '0' byte is the simplest (anyone reading this who is trying to decode these messages from the other end should make sure that they support any legal amount of padding).

In general for http content encryption, the mechanism is a bit more complicated than that (since you have to split up the input into records and add padding to each one), but the webpush spec says that the encrypted message must fit into a single record, so you don't need to worry about that.

Note the following text in the webpush encryption spec:

Note that a push service is not required to support more than 4096 octets of payload body, which equates to 4080 octets of cleartext

The 4080 octets of cleartext here includes the 1 byte of padding, so there effectively seems to be a limit of 4079 bytes. You can specify a larger record size using the "rs" parameter in the "Encryption" header, but according to the text quoted above, the recipient isn't required to support that.

One warning: some of the code that I've seen to do this seems to be changing to using 2 bytes of padding, presumably as a result of some proposed spec change, but I haven't been able to track down where this is coming from. At the moment 1 byte of padding should be ok, but if this stops working in the future, you may need to go to 2 bytes - as I mentioned above this spec is a work in progress and browser support is experimental right now.

Answer 3

See https://datatracker.ietf.org/doc/html/draft-ietf-webpush-encryption-01#section-5 and https://w3c.github.io/push-api/#widl-PushSubscription-getKey-ArrayBuffer-PushEncryptionKeyName-name (point 4).

The key is encoded using the uncompressed format defined in ANSI X9.62, so you can't use x509EncodedKeySpec.

You could use BouncyCastle, that should support the X9.62 encoding.

Answer 4

The solution of santosh kumar works with one modification:

I added a 1-byte cipher padding right before defining the cleartext byte[].

Cipher bobCipher = Cipher.getInstance("AES/GCM/NoPadding", "BC");
byte[] iv = generateNonce(nonce_bytes12, 0);
bobCipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));

// adding firefox padding:
bobCipher.update(new byte[1]);

byte[] cleartext = {...};