Idempotent Delivery Not Working as Expected in KafkaJS Version 2.2.4

Question

I am facing an issue with achieving idempotent delivery in Kafkajs version 2.2.4. Despite sending the same message multiple times, the message is delivered to the consumer each time, contrary to the expected idempotent behavior. I have meticulously followed the configuration steps outlined in the documentation to enable idempotence, yet the issue remains unresolved. At this point, I am starting to question whether the feature for idempotent message delivery — where Kafkajs should notify or prevent the re-sending of a message that has already been sent — is functioning as intended in this version. The configuration instruction from the documentation has been followed, but the feature does not seem to be working properly. Could this be a version-specific issue, or am I missing something in the configuration process? Any assistance or clarification on this matter would be highly appreciated.

That is the configuration i'm talking about: print of documentation

that's my code:

const { Kafka, logLevel } = require("kafkajs");
const fs = require("fs");

const kafka = new Kafka({
  clientId: "my-app",
  brokers: ["ip"],
  logLevel: logLevel.WARN,
  ssl: {
    rejectUnauthorized: false,
    ca: fs.readFileSync("./ca.crt", "utf-8"),
  },
  sasl: {
    mechanism: "scram-sha-256", 
    username: "myuser",
    password: "mypass",
  },
  // ssl:true
});

const producer = kafka.producer({
  retries: 0,
  idempotent: true, // Enable idempotent producer
});

const run = async () => {
  // Producing
  await producer.connect();
  setInterval(async function () {
    try {
      await producer.send({
        topic: "topicTest",
        messages: [{ value: JSON.stringify({ teste: "new test message" }) }],
      });
    } catch (e) {
      console.log(e);
    }
  }, 1000);

};

run().catch(console.error);

I have already tried using the idempotent setting and also acks=all in the message, without success. I always receive the duplicated message on the consumer