Facing java.nio.charset.MalformedInputException : Input length = 1 - with Spring Data Cassandra and Amazon Keyspace,How to Resolve?

Question

I am facing an issue while making a query to retrieve data (approximately 2000 rows) from a specific table ( it works for other entities and i have 50+ tables ); I receive a 500 error. What is perplexing is that sometimes it works (for the exact same data), but once it fails, it doesn't work again until I restart the server. Here's the error I am encountering:

2023-08-23T17:13:53.563+02:00 ERROR 63647 --- [nio-8091-exec-7] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: java.lang.IllegalArgumentException: java.nio.charset.MalformedInputException: Input length = 1] with root cause

java.nio.charset.MalformedInputException: Input length = 1
    at java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274) ~[na:na]
    at java.base/java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:820) ~[na:na]
    at com.datastax.oss.driver.internal.core.type.codec.StringCodec.decode(StringCodec.java:110) ~[java-driver-core-4.17.0.jar:na]
    at com.datastax.oss.driver.internal.core.type.codec.StringCodec.decode(StringCodec.java:35) ~[java-driver-core-4.17.0.jar:na]
    at com.datastax.oss.driver.api.core.data.GettableByIndex.getObject(GettableByIndex.java:153) ~[java-driver-core-4.17.0.jar:na]

The exception seems to originate from java.nio.charset.CoderResult.throwException. I'm using the java-driver-core-4.17.0.jar and spring-data-cassandra-4.1.2.jar.

I've tried different approaches, but the problem persists intermittently. Once it starts failing, the only way to get it working again is to restart the server. I've searched for a solution but have been unable to find any relevant information that helps to diagnose the issue.

I've configured Cassandra based on this Amazon Keyspaces example repository

my keyspaces-application.conf:

datastax-java-driver {
    basic.contact-points = [ "<basic.contact-points>" ]
    advanced.reconnect-on-init = true

    # Amazon Keyspaces supports LOCAL_QUORUM for Inserts, Updates, and Deletes
    # Amazon Keyspaces supports ONE, LOCAL_ONE, and LOCAL_QUORUM for Selects
    basic.request.consistency = "LOCAL_QUORUM"
    basic.request.serial-consistency = "LOCAL_SERIAL"

    # set to true for retry policy to apply on writes
    basic.request.default-idempotence = true
    advanced.retry-policy {
        class = com.methodz.seo_jungle.config.AmazonKeyspacesRetryPolicy
        max-attempts = 6
    }

    basic.load-balancing-policy {
        local-datacenter = "<local-datacenter>"
        slow-replica-avoidance = false
    }

    #Use IAM ROLES and service roles
    # https://github.com/aws/aws-sigv4-auth-cassandra-java-driver-plugin
   # advanced.auth-provider = {
   #     class = software.aws.mcs.auth.SigV4AuthProvider
   #     aws-region = us-east-1
   # }
#
    #Use service specific credentials
    # https://docs.aws.amazon.com/keyspaces/latest/devguide/programmatic.credentials.ssc.html
    advanced.auth-provider{
        class = PlainTextAuthProvider
        username = "<username>"
        password = "<password>"
    }

    advanced.ssl-engine-factory {
        class = DefaultSslEngineFactory
        truststore-path = "<truststore-path>"
        truststore-password = "<truststore-password>"
        #hostname validation must be set to false
        hostname-validation = false
    }

    # For every connection, Amazon Keyspaces allows 3000 request per connection per endpoint
    # If pool.local.size = 3 * 3 endpoints = 9 total connections for session
    # For 9 connections at 3000 request per connection for a total of 27,000 rps
    advanced.connection.pool.local.size = 3

}

Here are the dependencies I'm using in my project:

org.springframework.boot:spring-boot-starter:3.0.4
javax.servlet:javax.servlet-api:4.0.1
org.springframework.boot:spring-boot-starter-actuator:3.0.4
org.springframework.boot:spring-boot-starter-test:3.0.4
org.springframework.boot:spring-boot-starter-web:3.0.4
org.springframework.boot:spring-boot-starter-security:3.0.4
io.jsonwebtoken:jjwt-api:0.11.5
io.jsonwebtoken:jjwt-jackson:0.11.5
io.jsonwebtoken:jjwt-impl:0.11.5
com.fasterxml.jackson.module:jackson-module-kotlin:2.15.2
org.springdoc:springdoc-openapi-starter-webmvc-api:2.1.0
org.springframework.boot:spring-boot-starter-aop:3.0.4
org.springframework.data:spring-data-elasticsearch:5.0.0
javax.xml.bind:jaxb-api:2.4.0-b180830.0359
org.springframework.data:spring-data-cassandra:4.1.2
com.datastax.oss:java-driver-core:4.17.0
org.springframework.boot:spring-boot-starter-data-cassandra:3.0.4
software.aws.mcs:aws-sigv4-auth-cassandra-java-driver-plugin:4.0.6
org.jetbrains.kotlin:kotlin-reflect:1.8.20-RC
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.20-RC
org.springframework.boot:spring-boot-devtools:3.0.4
org.springframework.boot:spring-boot-starter-logging:3.0.4
org.springframework:spring-context-support:6.0.6
org.springframework.boot:spring-boot-starter-mail:3.0.4
org.jetbrains.kotlinx:kotlinx-coroutines-core:pom:1.7.1
com.google.code.gson:gson:2.10.1
com.google.cloud:libraries-bom:pom:26.11.0
org.junit.jupiter:junit-jupiter-engine:5.9.2
javax.persistence:javax.persistence-api:2.2
org.jetbrains.kotlinx:kotlinx-coroutines-reactor:1.7.1
org.springframework.boot:spring-boot-starter-validation:3.1.2
org.hibernate.validator:hibernate-validator:8.0.1.Final
org.jsoup:jsoup:1.15.3
us.codecraft:xsoup:0.3.7
com.squareup.okhttp3:okhttp:4.9.1

The model responsible for this issue can be seen in the code snippet below:

package com.methodz.seo_jungle.dao.organization

import org.springframework.data.annotation.Id
import org.springframework.data.cassandra.core.cql.PrimaryKeyType
import org.springframework.data.cassandra.core.mapping.PrimaryKeyColumn
import org.springframework.data.cassandra.core.mapping.Table
import java.util.*

@Table("organization")
data class Organization(
    @Id
    @PrimaryKeyColumn("id", type = PrimaryKeyType.PARTITIONED)
    var id: UUID = UUID.randomUUID(),
    var name: String,
    var type: String = OrganizationType.COMPANY.name, //OrganizationType = OrganizationType.COMPANY
    var siret: String = "",
    var tva: String = "",
    var email: String = "",
    var address: String = "",
    var additionalAddressDetails: String = "",
    var city: String = "",
    var zipCode: String = "",
    var country: String = "",
    var legalRepresentativeFirstname: String = "",
    var legalRepresentativeLastname: String = "",
    var legalRepresentativeCountry: String = "",
    var legalRepresentativeBirthdate: Long = 0,
    var legalRepresentativeNationality: String = "",
    var currentCreditAmount: Double = 0.0,
    var totalCreditSpend: Double = 0.0,
    var totalCreditEarned: Double = 0.0,
    var status: String = OrganizationStatus.DRAFT.name
    var created: Long = DateUtils.getTimestamp(),
    var updated: Long = 0,
    var deleted: Long = 0
)

And here the schema of the associated table (auto generated)

The repository that is used is a simple repository with no override

@Repository
interface SupportRepository : CassandraRepository<Support, UUID> {}

I tried using Amazon Corretto JDK 11 to match the exemple repo and still facing the same issue ( i'm using openJDK 20 )

i don't know what to try next

The error seems to be related to the String codec in the DataStax Java driver. It happens inconsistently, and restarting the server temporarily fixes it. I've searched for a solution but have been unable to find any relevant information that helps to diagnose the issue.

Any help or insight would be greatly appreciated!

Answer 1

After some investigation, I found that the issue was with the prepared queries that Spring Data Cassandra is using. Somehow, disabling them fixed the issue, but I don't know how to properly fix it.

    @Bean
    open fun cassandraTemplate(sessionFactory: SessionFactory?, converter: CassandraConverter?): CassandraOperations {
        val template = CassandraTemplate(sessionFactory!!, converter!!)
        template.isUsePreparedStatements=false
        return template
    }