Group enrollment in Azure Device Provisoning Service (DPS) not working through REST API: unauthorized 401002

Question

I am trying to support DPS for my ESP32 firmware through HTTPS REST API using SAS.

My device registration ID is: xx-xx-8c4b14149ff4

I took the group enrollment from DPS primary key to generate the symmetric key from the registration ID.

I created the related SAS from that and forged the request, but the server returns "Unauthorized" with error code being 401002.

Here is my request (a curl version is provided for handiness ):

curl -L -i -X PUT \
-H 'Content-Type: application/json' \
-H 'Content-Encoding: utf-8' \
-H 'Authorization: SharedAccessSignature sr=0neXXXXXX22%2Fregistrations%xx-xx-8c4b14149ff4&sig=XXXXXXXXXXXXXXXXXXX%3D&skn=registration&se=1651482003' \
-d '{"registrationId": "xx-xx-8c4b14149ff4"}' \
https://global.azure-devices-provisioning.net/0neXXXXXX22/registrations/xx-xx-8c4b14149ff4/register?api-version=2021-06-01

Note that I replaced secret information with "xx".

The response body is as follows:

{
    "errorCode": 401002,
    "trackingId": "9fecada7-4e51-455e-9392-68522654a64a",
    "message": "Unauthorized",
    "timestampUtc": "2022-05-02T08:04:03.5761437Z"
}

Is there anything I must tweak to use the HTTPS REST API from the portal?

What should I look at beside the information themselves (which I already double-checked)?

References:

• About SAS: https://learn.microsoft.com/en-us/azure/iot-dps/how-to-control-access
• About the REST API: https://learn.microsoft.com/en-us/rest/api/iot-dps/device/runtime-registration/register-device#provisioningserviceerrordetails

Answer 1

Actually my signature code had a small bug. I fixed it and now it works.

Here is the (working) code for signing:

#include <mbedtls/md.h>         // mbed tls lib used to sign SHA-256

#include <base64.hpp>           // Densaugeo Base64 version 1.2.0 or 1.2.1


/// Returns the SHA-256 signature of [dataToSign] with the key [enrollmentPrimaryKey]
/// params[in]: dataToSign The data to sign (for our purpose, it is the registration ID (or the device ID if it is different)
/// params[in]: enrollmentPrimaryKey The group enrollment primary key.
/// returns The SHA-256 base-64 signature to present to DPS.
/// Note: I use mbed to SHA-256 sign.
String Sha256Sign(String dataToSign, String enrollmentPrimaryKey){
  /// Length of the dataToSign string
  const unsigned dataToSignLength = dataToSign.length();
  /// Buffer to hold the dataToSign as a char[] buffer from String.
  char dataToSignChar[dataToSignLength + 1];
  /// String to c-style string (char[])
  dataToSign.toCharArray(dataToSignChar, dataToSignLength + 1);

  /// The binary decoded key (from the base 64 definition)
  unsigned char decodedPSK[32];

  /// Encrypted binary signature
  unsigned char encryptedSignature[32];

  /// Base 64 encoded signature
  unsigned char encodedSignature[100];
  
  Serial.printf("Sha256Sign(): Registration Id to sign is: (%d bytes) %s\n", dataToSignLength, dataToSignChar);
  Serial.printf("Sha256Sign(): DPS group enrollment primary key is: (%d bytes) %s\n", enrollmentPrimaryKey.length(), enrollmentPrimaryKey.c_str());


  // Need to base64 decode the Preshared key and the length
  const unsigned base64DecodedDeviceLength = decode_base64((unsigned char*)enrollmentPrimaryKey.c_str(), decodedPSK);
  Serial.printf("Sha256Sign(): Decoded primary key is: (%d bytes) ", base64DecodedDeviceLength);

  for(int i= 0; i<base64DecodedDeviceLength; i++) {
    Serial.printf("%02x ", (int)decodedPSK[i]);
  }
  Serial.println();
  
  // Use mbed to sign
  mbedtls_md_type_t mdType = MBEDTLS_MD_SHA256;
  mbedtls_md_context_t hmacKeyContext;    

  mbedtls_md_init(&hmacKeyContext);
  mbedtls_md_setup(&hmacKeyContext, mbedtls_md_info_from_type(mdType), 1);
  mbedtls_md_hmac_starts(&hmacKeyContext, (const unsigned char *) decodedPSK, base64DecodedDeviceLength);
  mbedtls_md_hmac_update(&hmacKeyContext, (const unsigned char *) dataToSignChar, dataToSignLength);
  mbedtls_md_hmac_finish(&hmacKeyContext, encryptedSignature);
  mbedtls_md_free(&hmacKeyContext);
  
  Serial.print("Sha256Sign(): Computed hash is: ");

  for(int i= 0; i<sizeof(encryptedSignature); i++) {
    Serial.printf("%02x ", (int)encryptedSignature[i]);
  }
  Serial.println();
  // base64 decode the HMAC to a char
  encode_base64(encryptedSignature, sizeof(encryptedSignature), encodedSignature);

  Serial.printf("Sha256Sign(): Computed hash as base64: %s\n", encodedSignature);

  // creating the real SAS Token
  return String((char*)encodedSignature);
}

See it there: How to generate a symmetric key in C or C++ the same way this script does?

Answer 2

Can you provide a code sample of how you are generating the individual key? Did you try the samples at https://learn.microsoft.com/en-us/azure/iot-dps/how-to-legacy-device-symm-key?tabs=linux%22%20%5Cl%20%22derive-a-device-key#derive-a-device-key ?

Here's an example using C#

        private string GenerateDeviceSas(string enrollmentGroupName, string dpsIdScope, string registrationId, string key)
        {
            if(String.IsNullOrEmpty(dpsIdScope) || String.IsNullOrEmpty(registrationId) || String.IsNullOrEmpty(key))
            {
                Console.WriteLine("Error: Missing required values in settings.json");
                return null;
            }

            if (String.IsNullOrEmpty(enrollmentGroupName))
            {
                // Generate Device API SAS key for individual enrollment
                return GenerateSasToken($"{dpsIdScope}/registrations/{registrationId}", key, "registration");
            }
            else
            {

                // Generate derived Device API SAS key for group enrollment
                // See https://learn.microsoft.com/en-us/azure/iot-dps/how-to-legacy-device-symm-key?tabs=linux#create-a-symmetric-key-enrollment-group

                HMACSHA256 hmacsha256 = new HMACSHA256();
                hmacsha256.Key = Convert.FromBase64String(key);
                var sig = hmacsha256.ComputeHash(ASCIIEncoding.ASCII.GetBytes(registrationId));
                var derivedkey = Convert.ToBase64String(sig);

                return GenerateSasToken($"{dpsIdScope}/registrations/{registrationId}", derivedkey, "registration");

            }
        }

        private static string GenerateSasToken(string resourceUri, string key, string policyName, int expiryInSeconds = 3600)
        {
            TimeSpan fromEpochStart = DateTime.UtcNow - new DateTime(1970, 1, 1);
            string expiry = Convert.ToString((int)fromEpochStart.TotalSeconds + expiryInSeconds);

            string stringToSign = WebUtility.UrlEncode(resourceUri) + "\n" + expiry;

            HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(key));
            string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));

            string token = String.Format(CultureInfo.InvariantCulture, "SharedAccessSignature sr={0}&sig={1}&se={2}", WebUtility.UrlEncode(resourceUri), WebUtility.UrlEncode(signature), expiry);

            if (!String.IsNullOrEmpty(policyName))
            {
                token += "&skn=" + policyName;
            }

            return token;
        }

Answer 3

Consider the following and note how I calculate a deviceKey from the DPS key and then generate the SAS Token from the device key. Some people overlook that. :)

from base64 import b64encode, b64decode
from hashlib import sha256
from time import time
from urllib.parse import urlencode, quote_plus
from hmac import HMAC
import requests

def generate_sas_token(uri, key, policy_name, expiry=3600):
    ttl = time() + expiry
    sign_key = "%s\n%d" % ((quote_plus(uri)), int(ttl))
    sign_key = sign_key.encode('utf-8')
    signature = b64encode(HMAC(b64decode(key), sign_key, sha256).digest())

    rawtoken = {
        'sr' :  uri,
        'sig': signature,
        'se' : str(int(ttl))
    }
    
    if policy_name:
        rawtoken['skn'] = policy_name

    return 'SharedAccessSignature ' + urlencode(rawtoken)

device_id = "3c71bfa95f7c"
scope_id = "0neYOURSCOPEHERE39"
dpskey = 'CsBYOURSASCODEHEREXWyfUMl/OA=='

deviceKey = b64encode(HMAC(b64decode(dpskey), device_id.encode('utf-8'), sha256).digest())
uri = scope_id + '/registrations/' + device_id
policy= 'registration'
url = "https://global.azure-devices-provisioning.net/" + scope_id + "/registrations/" + device_id + "/register?api-version=2021-06-01"
headers = {'Authorization': generate_sas_token(uri=uri, key=deviceKey, policy_name=policy), 'User-Agent': 'MicroPython', 'content-type': 'application/json', 'Content-Encoding': 'utf-8'}
data = '{"registrationId" : "' + device_id + '"}'

print(generate_sas_token(uri=uri, key=deviceKey, policy_name=policy))

print(requests.request("PUT", url=url, data=data, headers=headers).json())

The response shows "assigning", which is success. Hope this helps.

C:/Python38-64/python.exe h:/test/sastoken.py
SharedAccessSignature sr=0ne00223A39%2Fregistrations%2F3c71bfa95f7c&sig=UH5KKeREMOVED0vcs%3D&se=1651880438&skn=registration
{'operationId': '4.4cb788ae24922c84.6856a3d0-857f-4c79-a90c-360fdf3355f8', 'status': 'assigning'}