DynamoDB table model code supporting design with more than one sort keys

Question

I'm working with a DynamoDB table where each partition key is associated with multiple sort keys. I need to fetch all items in one partition key. I understood that I would need to get List of these items with same partition keys and different sort keys and respective attributes. Then map them to a single DTO model if needed. I've set up a ProfileModel with partition (pk) and sort (sk) keys, along with a Map<String, AttributeValue> to hold various attributes. However, the application throws an IllegalStateException during startup, indicating that a converter for EnhancedType<Map<String, AttributeValue>> can't be found. I suspect this might be related to how the MapAttributeConverter is integrated or potentially a missing configuration. Below is the relevant code snippet and error message:

ProfileDynamoDb.java:

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Repository;
import software.amazon.awssdk.core.pagination.sync.SdkIterable;
import software.amazon.awssdk.enhanced.dynamodb.*;
import software.amazon.awssdk.enhanced.dynamodb.model.*;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.DynamoDbException;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@Slf4j
@Repository
public class ProfileDynamoDb {

    private final DynamoDbTable<ProfileModel> table;
    private final DynamoDbEnhancedClient enhancedClient;

    @Autowired
    public ProfileDynamoDb(DynamoDbEnhancedClient enhancedClient) {
        this.enhancedClient = enhancedClient;
        table = enhancedClient.table("profiles", TableSchema.fromBean(ProfileModel.class));
    }
    public List<ProfileModel> fetchByPartitionKey(String profileName) {
        try {
            QueryConditional queryConditional = QueryConditional.keyEqualTo(k -> k.partitionValue("PROFILE#" + profileName));
            SdkIterable<ProfileModel> results = table.query(r -> r.queryConditional(queryConditional)).items();
            return results.stream().toList();
        } catch (DynamoDbException e) {
            log.error("Error while getting profile by name: {}", profileName, e);
            throw e;
        }
    }

ProfileModel.java:

import lombok.Data;
import software.amazon.awssdk.enhanced.dynamodb.internal.converter.attribute.MapAttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.*;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.util.HashMap;
import java.util.Map;

@Data
@DynamoDbBean
public class ProfileModel {
    private String pk;
    private String sk;
    private Map<String, AttributeValue> attributes = new HashMap<>();

    public void addAttribute(String key, AttributeValue value) {
        attributes.put(key, value);
    }

    @DynamoDbAttribute("Attributes")
    @DynamoDbConvertedBy(MapAttributeConverter.class)
    public Object getAttribute(String key) {
        return attributes.get(key);
    }

    @DynamoDbPartitionKey
    public String getPk() {
        return pk;
    }

    @DynamoDbSortKey
    public String getSk() {
        return sk;
    }
}

Error Message:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.booking.profile.infrastructure.dynamodb.ProfileDynamoDb]: Constructor threw exception
Caused by: java.lang.IllegalStateException: Converter not found for EnhancedType(java.util.Map<java.lang.String, software.amazon.awssdk.services.dynamodb.model.AttributeValue>)

Could someone help in identifying what might be causing the converter issue and how to fix it? Is there a specific way to define the model and DAO for the Map attribute in DynamoDB entities using Spring Boot and AWS SDK for Java? or Is there a way to do the same with many attributes with same partition keys and different soft keys, like when you need to fetch single item, DynamoDB library maps all attributes itself,

Answer 1

Your model defines a Map<String, AttributeValue> to store various attributes.
If @DynamoDbConvertedBy(MapAttributeConverter.class) is not a standard converter available in the SDK for such a type, you might consider implementing a custom attribute converter that implements the AttributeConverter<Map<String, AttributeValue>> interface.

import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeConverterContext;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.util.HashMap;
import java.util.Map;

public class MapAttributeValueConverter implements AttributeConverter<Map<String, AttributeValue>> {

    @Override
    public AttributeValue transformFrom(Map<String, AttributeValue> input, AttributeConverterContext context) {
        return AttributeValue.builder().m(input).build();
    }

    @Override
    public Map<String, AttributeValue> transformTo(AttributeValue input, AttributeConverterContext context) {
        return input.m();
    }

    @Override
    public EnhancedType<Map<String, AttributeValue>> type() {
        return EnhancedType.mapOf(String.class, AttributeValue.class);
    }

    @Override
    public AttributeValueType attributeValueType() {
        return AttributeValueType.M;
    }
}

You then need to update your ProfileModel to use this new converter:

@DynamoDbConvertedBy(MapAttributeValueConverter.class)
public Map<String, AttributeValue> getAttributes() {
    return attributes;
}

---

The custom attribute converter MapAttributeValueConverter addresses the specific concern of handling a Map<String, AttributeValue> within a DynamoDB item.

To address your questions more directly:

• Fetching items with the same partition key and different sort keys: That is a common access pattern in DynamoDB and is supported natively through query operations.
  When you perform a query operation specifying only the partition key, DynamoDB retrieves all items that share that partition key, regardless of their sort keys. That is how you can fetch multiple items that are logically grouped together by the partition key, but differentiated by their sort keys.
  See "composite primary key".

• Mapping attributes: When fetching items, the DynamoDB Enhanced Client automatically maps the attributes of the items fetched from DynamoDB to the fields of your model class, based on the annotations you have provided in your model class.
  If you have a field in your model annotated to handle a Map<String, AttributeValue>, the custom attribute converter (MapAttributeValueConverter) comes into play to convert the DynamoDB AttributeValue type to the Map<String, AttributeValue> type and vice versa.

• Concern about Map<String, AttributeValue> fields: If your intention is to dynamically handle a variable set of attributes within a single item, using a Map<String, AttributeValue> is a valid approach. The custom converter enables you to store and retrieve these dynamic attributes as part of your model.
  However, this does not change the fundamental way DynamoDB handles queries and mappings; it just allows your application to flexibly handle dynamic attributes within the constraints of your model's structure.

---

What version of Dynamodb I am supposed to use? There is AttributeConverterContext class. Additionally there is no transformFrom and To accepting two parameters.
I use version implementation("software.amazon.awssdk:dynamodb-enhanced:2.17.128").

In that case, that would be:

import software.amazon.awssdk.enhanced.dynamodb.AttributeConverter;
import software.amazon.awssdk.enhanced.dynamodb.AttributeValueType;
import software.amazon.awssdk.enhanced.dynamodb.EnhancedType;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

import java.util.Map;

public class MapAttributeValueConverter implements AttributeConverter<Map<String, AttributeValue>> {

    @Override
    public AttributeValue transformFrom(Map<String, AttributeValue> input) {
        return AttributeValue.builder().m(input).build();
    }

    @Override
    public Map<String, AttributeValue> transformTo(AttributeValue input) {
        return input.m();
    }

    @Override
    public EnhancedType<Map<String, AttributeValue>> type() {
        return EnhancedType.of(new TypeToken<Map<String, AttributeValue>>() {});
    }

    @Override
    public AttributeValueType attributeValueType() {
        return AttributeValueType.M;
    }
}

That would include the following changes:

• The transformFrom and transformTo methods now correctly accept only the relevant input (Map<String, AttributeValue> for transformFrom and AttributeValue for transformTo) and return the appropriate types without the AttributeConverterContext.
• The use of EnhancedType.of with new TypeToken<Map<String, AttributeValue>>(){} might need adjustment based on your specific implementation or JDK version due to type erasure in generics. If TypeToken is not directly available or causes issues, you might simply use EnhancedType.mapOf(String.class, AttributeValue.class) as a more straightforward approach to specify the EnhancedType for Map<String, AttributeValue>.