Querying Items with DynamoDBClient using an Global Secondary Index (GSI) with a partition key (deviceId) and a sort key (timestamp)

Question

So I have a Lambda function acting as a Express.js server and an API. I am hitting all of the endpoint correctly and data is being returned the way I want. The sole problem I am left with is a function that is supposed to return a list of items in the following format:

[
 {
 "id": "d8aaa50e-5b4d-4a6f-84f5-e508e594f84d",
 "deviceId": "dkajshdaksdhj",
 "humidity": "45",
 "light": "35",
 "ph": "55",
 "temp": "65",
 "timestamp": 1696600485,
 "waterTemp": "75"
 },
 {...}
]

However when performing a QueryCommand on my table like so:

      const data = await dynamoDb.send(
        new QueryCommand({
          TableName: process.env.DYNAMODB_TABLE_NAME_READINGS,
          IndexName: 'deviceId-timestamp-index', // Use the name of your Global Secondary Index
          KeyConditionExpression: '#deviceId = :deviceId AND #t BETWEEN #e AND #s',
          ExpressionAttributeNames: {
            '#deviceId': 'deviceId',
            '#t': 'timestamp',
            '#e': 'end',
            '#s': 'start',
          },
          ExpressionAttributeValues: marshall(
            {
              ':deviceId': deviceId,
              ':s': start,
              ':e': end,
            },
            {
              removeUndefinedValues: true,
            },
          ),
          ScanIndexForward: false,
        }),
      );

I am getting hit with the following error:

{
    "status": "error",
    "statusCode": "500",
    "stack": "Error: ValidationException: One or more parameter values were invalid: Condition parameter type does not match schema type\n    at Object.getHistoricalReadingsForDeviceId (file:///var/task/code/services/deviceService.js:240:11)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async getHistoricalReadings (file:///var/task/code/controllers/deviceController.js:75:32)"
}

Now I have checked both with ChatGPT, AWS Documentation and StackOverflow but after a day of trying out things I am now without options.

In the AWS Console the following Query works properly and it is returning items:

However when I try both on my deployed website and in Postman the result is the same (this error):

Link to repository in Github and exact file and line of the function is located: click

My speculation is that it is complaining either because I do not have start or end as attributes in my table or because I am trying to marshall multiple things at once? Link to marshall/unmarshall docs. It is basically an utility function that is helping with converting from/to DynamoDB attribute types.

Answer 1

ExpressionAttributeNames are used for attribute names, such as deviceId and timestamp.

ExpressionAttributeValues are used for values, such as start and end.

Your code has mixed these up. Below should work.

const data = await dynamoDb.send(
        new QueryCommand({
          TableName: process.env.DYNAMODB_TABLE_NAME_READINGS,
          IndexName: 'deviceId-timestamp-index', // Use the name of your Global Secondary Index
          KeyConditionExpression: '#deviceId = :deviceId AND #t BETWEEN :s AND :e',
          ExpressionAttributeNames: {
            '#deviceId': 'deviceId',
            '#t': 'timestamp'
          },
          ExpressionAttributeValues: marshall(
            {
              ':deviceId': deviceId,
              ':s': start,
              ':e': end,
            }
          ),
          ScanIndexForward: false,
        }),
      );

As it's clear you'd rather work with native JSON, I believe you would see greater benefit using the DocumentClient, you can read more about that here

No marshalling required, it also means your results won't need to be unmarshalled:

const { DynamoDBClient } = require("@aws-sdk/client-dynamodb");
const { DynamoDBDocumentClient, QueryCommand } = require("@aws-sdk/lib-dynamodb");

const client = new DynamoDBClient({});
const docClient = DynamoDBDocumentClient.from(client);

const data = await docClient.send(
        new QueryCommand({
          TableName: process.env.DYNAMODB_TABLE_NAME_READINGS,
          IndexName: 'deviceId-timestamp-index', // Use the name of your Global Secondary Index
          KeyConditionExpression: '#deviceId = :deviceId AND #t BETWEEN :s AND :e',
          ExpressionAttributeNames: {
            '#deviceId': 'deviceId',
            '#t': 'timestamp'
          },
          ExpressionAttributeValues: {
              ':deviceId': deviceId,
              ':s': start,
              ':e': end,
            },
          ScanIndexForward: false,
        }),
      );

Answer 2

TL;DR

When getting values from query parameters keep in mind they are always strings. I was so sure that I am passing numbers from my frontend, hence the reason why I was so stuck. In the end a simple parseInt function solved my issue with a combination of the provided answers as well.

Working code below:

      const data = await dynamoDb.send(
        new QueryCommand({
          TableName: process.env.DYNAMODB_TABLE_NAME_READINGS,
          IndexName: 'deviceId-timestamp-index', // Use the name of your Global Secondary Index
          KeyConditionExpression: '#deviceId = :deviceId AND #t BETWEEN :s AND :e',
          ExpressionAttributeNames: {
            '#deviceId': 'deviceId',
            '#t': 'timestamp',
          },
          ExpressionAttributeValues: marshall(
            {
              ':deviceId': deviceId,
              ':s': parseInt(start),
              ':e': parseInt(end),
            },
            {
              removeUndefinedValues: true,
            },
          ),
          ScanIndexForward: false,
        }),
      );