Create Keycloak client using REST API from Bash

Question

I have a script that automatically creates a local Kubernetes cluster (for development purposes), installing a bunch of applications including Keycloak on it. The next step that I need to automate is creating some Keycloak resources. I need to:

• create a realm
• create two clients
• create two users
• assign a specific composite role to each user
• add the users to a specific group

I have managed to create a realm through the API, but creating a client or user is causing errors.

This is my code:

##################### Create variables ########################
LB_ADDRESS="192.168.49.3.nip.io"
KEYCLOAK_URL="https://auth.${LB_ADDRESS}/auth"
REALM_NAME="account-1"
ADMIN_USERNAME="admin"
ADMIN_PASSWORD="aic"

project_id_1="7677"

client_1_id="ais-${project_id_1}"
client_1_name="project${project_id_1}.${LB_ADDRESS}"

user_1_name="myuser"
user_1_password="mypassword"

group_name="$client_1_name"

#################### Create helper functions ####################
function get_access_token() {
    curl --insecure -s -X POST "${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&client_id=admin-cli&username=${ADMIN_USERNAME}&password=${ADMIN_PASSWORD}" | jq -r '.access_token'
}

function create_realm() {
    curl --insecure -s -X POST "${KEYCLOAK_URL}/admin/realms" -H "Authorization: Bearer $ACCESS_TOKEN" -H "Content-Type: application/json" -d "{
        \"realm\": \"$1\",
        \"enabled\": true
    }"
}

function create_client() {
    client_id="$1"
    client_name="$2"
    curl --insecure -X POST "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/clients" \
    -H "Authorization: Bearer $ACCESS_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
        "clientId": "'"$client_id"'",
        "name": "'"$client_name"'",
        "standardFlowEnabled": true,
        "directAccessGrantsEnabled": true,
        "serviceAccountsEnabled": true,
        "publicClient": false,
        "attributes": {
            "validRedirectUris": ["*"],
            "webOrigins": ["*"]
        }
    }'
}

function create_user() {
    username="$1"
    password="$2"
    curl --insecure -X POST "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/users/" \
    -H "Authorization: Bearer $ACCESS_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
        "username": "'"$username"'",
        "emailVerified": true,
        "serviceAccountsEnabled": true,
        "credentials": [{
          "type": "password",
          "value": "'"$password"'",
          "temporary": false
        }]
    }'
}

function assign_role_to_user() {
    client="$1"
    username="$2"
    role="$3"
    curl --insecure -X POST "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/users/${username}/role-mappings/clients/${client}" \
    -H "Authorization: Bearer $ACCESS_TOKEN" \
    -H "Content-Type: application/json" \
    -d '[{
        "id": "'"$role"'",
        "composite": true
    }]'
}

function add_user_to_group() {
    group="$1"
    username="$2"
    curl --insecure -X PUT "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/users/${username}/groups/${group}" \
    -H "Authorization: Bearer $ACCESS_TOKEN"
}

################### Access Keycloak API #######################
ACCESS_TOKEN="$(get_access_token)"

create_realm $REALM_NAME
create_client "$client_1_id" "$client_1_name"
create_user "$user_1_name" "$user_1_password"
assign_role_to_user "$client_1_id" "$user_1_name" "my-composite-role"
add_user_to_group "$group_name" "$user_1_name"

The access token is retrieved successfully, the realm is created successfully, but then it fails:

create_client "$client_1_id" "$client_1_name" results in:

{"error":"HTTP 401 Unauthorized"}

What am I doing wrong?

Edit: Do I maybe have to create an access token for the newly created realm before creating resources inside of it? But that seems impossible because the newly created realm doesn't have an admin account by default and in order to create one, I would need an access token again.

Btw I'm using Keycloak v21.1.1: https://quay.io/repository/keycloak/keycloak?tab=tags&tag=21.1.1

Answer 1

You said:

even after applying your fixes, I still get {"error":"HTTP 401 Unauthorized"} for create_client "$client_1_id" "$client_1_name" (and for create_user "$user_1_name" "$user_1_password" too).

As a workaround, you might consider using first the master realm's admin account to perform operations in the newly created realm. The master realm's admin account has sufficient privileges to manage other realms.

Then, use the access token obtained from the master realm (with the admin-cli client), to perform operations in the new realm: it should carry the necessary permissions.

If needed, after setting up the new realm, you can create an admin user specific to that realm. That can be done using the master realm token. Once this new admin user is set up, you can obtain tokens specific to the new realm for subsequent operations.

##################### Create variables ########################
LB_ADDRESS="192.168.49.3.nip.io"
KEYCLOAK_URL="https://auth.${LB_ADDRESS}/auth"
REALM_NAME="account-1"
ADMIN_USERNAME="admin"
ADMIN_PASSWORD="aic"

project_id_1="7677"

client_1_id="ais-${project_id_1}"
client_1_name="project${project_id_1}.${LB_ADDRESS}"

user_1_name="myuser"
user_1_password="mypassword"

group_name="$client_1_name"

#################### Create helper functions ####################
# Obtain access token from master realm
function get_master_realm_access_token() {
    curl --insecure -s -X POST "${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token" \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "grant_type=password&client_id=admin-cli&username=${ADMIN_USERNAME}&password=${ADMIN_PASSWORD}" | jq -r '.access_token'
}

# Create a new realm using master realm token
function create_realm() {
    curl --insecure -s -X POST "${KEYCLOAK_URL}/admin/realms" \
    -H "Authorization: Bearer $1" -H "Content-Type: application/json" \
    -d "{
        \"realm\": \"$2\",
        \"enabled\": true
    }"
}

# Create a client in new realm using master realm token (with JSON structure fix)
function create_client() {
    curl --insecure -X POST "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/clients" \
    -H "Authorization: Bearer $1" \
    -H "Content-Type: application/json" \
    -d '{
        "clientId": "'"$2"'",
        "name": "'"$3"'",
        "standardFlowEnabled": true,
        "directAccessGrantsEnabled": true,
        "serviceAccountsEnabled": true,
        "publicClient": false,
        "redirectUris": ["*"],
        "webOrigins": ["*"]
    }' #'
}

# Create a user in new realm using master realm token (with JSON structure fix)
function create_user() {
    curl --insecure -X POST "${KEYCLOAK_URL}/admin/realms/$REALM_NAME/users/" \
    -H "Authorization: Bearer $1" \
    -H "Content-Type: application/json" \
    -d '{
        "username": "'"$2"'",
        "emailVerified": true,
        "credentials": [{
          "type": "password",
          "value": "'"$3"'",
          "temporary": false
        }]
    }' #'
}

# [other function definitions (e.g., assign_role_to_user, add_user_to_group) with similar modifications]

################### Access Keycloak API #######################
# Get access token from master realm
MASTER_REALM_ACCESS_TOKEN="$(get_master_realm_access_token)"

# Use master realm token to create resources in new realm
create_realm "$MASTER_REALM_ACCESS_TOKEN" $REALM_NAME
create_client "$MASTER_REALM_ACCESS_TOKEN" "$client_1_id" "$client_1_name"
create_user "$MASTER_REALM_ACCESS_TOKEN" "$user_1_name" "$user_1_password"
# [rest of the operations using MASTER_REALM_ACCESS_TOKEN]

The get_master_realm_access_token function retrieves an access token from the master realm. That token is used for all subsequent API calls.
The script creates a new realm and then proceeds to create a client and a user within that realm using the master realm token. It does not create a separate admin user for the new realm, which simplifies the process and uses the master realm's admin privileges to manage the new realm.

The functions create_realm, create_client, etc., are modified to accept an access token as their first argument. That token is the one obtained from the master realm and is used to authorize operations in the new realm.

As per dreamcrash's answer, the JSON structures for creating clients and users have been corrected.

---

If the master realm and admin user was already used, but, as commented, the access token is too short-lived, causing the HTTP 401 Unauthorized error, you could, instead of using a single access token for all operations, use a refresh token to obtain a new access token when needed. That approach makes sure you always have a valid token for your operations.

If the script takes a long time between obtaining the token and using it, try to minimize this gap. You can fetch the token just before it is needed for each operation. Before making an API call, check if the token is still valid. If it is not, obtain a new one before proceeding.

As a less secure but more convenient approach, you could increase the lifespan of the access token in Keycloak's settings. But... that is generally not recommended for production environments due to security concerns but might be acceptable in a development setup.

# [previous parts of the script]

# Function to refresh the access token using a refresh token
function refresh_access_token() {
    curl --insecure -s -X POST "${KEYCLOAK_URL}/realms/master/protocol/openid-connect/token" \
    -H "Content-Type: application/x-www-form-urlencoded" \
    -d "grant_type=refresh_token&client_id=admin-cli&refresh_token=$1" | jq -r '.access_token'
}

################### Access Keycloak API #######################
# Get initial access token from master realm
ACCESS_TOKEN="$(get_master_realm_access_token)"

# Use the access token and refresh it if needed
create_realm "$ACCESS_TOKEN" $REALM_NAME
# Refresh the token
ACCESS_TOKEN="$(refresh_access_token "$ACCESS_TOKEN")"
create_client "$ACCESS_TOKEN" "$client_1_id" "$client_1_name"
# Refresh the token
ACCESS_TOKEN="$(refresh_access_token "$ACCESS_TOKEN")"
create_user "$ACCESS_TOKEN" "$user_1_name" "$user_1_password"
# and so on for other operations ...

# The script assumes that the refresh token is part of the initial token response.
# If the initial token response does not include a refresh token, additional modifications will be needed.

The refresh_access_token function is used to obtain a new access token using the refresh token whenever necessary. That should help in situations where the access token expires too quickly for the operations to complete successfully.

Answer 2

The first problem is that the json on the create client is wrong instead of:

-d '{
    "clientId": "'"$client_id"'",
    "name": "'"$client_name"'",
    "standardFlowEnabled": true,
    "directAccessGrantsEnabled": true,
    "serviceAccountsEnabled": true,
    "publicClient": false,
    "attributes": {
        "validRedirectUris": ["*"],
        "webOrigins": ["*"]
    }
}'

it should be:

-d '{
    "clientId": "'"$client_id"'",
    "name": "'"$client_name"'",
    "standardFlowEnabled": true,
    "directAccessGrantsEnabled": true,
    "serviceAccountsEnabled": true,
    "publicClient": false,
    "redirectUris": ["*"],
    "webOrigins": ["*"]
}'

Second on the user json the field 'serviceAccountsEnabled' does not actually exist in Keycloak, the closes thing you have to it is 'serviceAccountClientId'. So you user json should be :

-d '{
    "username": "'"$username"'",
    "emailVerified": true,
    "credentials": [{
      "type": "password",
      "value": "'"$password"'",
      "temporary": false
    }]
}'

Your 'assign_role_to_user' function is using the wrong endpoint. The endpoint is not:

"${KEYCLOAK_URL}/admin/realms/$REALM_NAME/users/${username}/role-mappings/clients/${client}"

but

POST /{realm}/users/{id}/role-mappings/clients/{client}

you need to use the 'id' of the user, not its username. Moreover, in the payload the 'id' should the role ID, have a look at this SO thread for more information.

Answer 3

You can try creating the client in the realm step, that seems more correct to me,
The simple way to do this is to pre-define a realm and then export the JSON.
Anyway, I can suggest the following things to check:

• make sure the ACCESS_TOKEN has not expired when the request is sent (default is short time)
• 401 alone not really help to understand the issue
  (considering the fact that I assume that it worked in the creation of the realm)
  I would suggest checking keycloak logs for more information

Another thing I would suggest to consider, I don't know how you install the keycloak, but if you work with helm or argo you can provide a pre-defined realm on the startup