<Error> StorageKafka (user_kafka): [rdk:ERROR] [thrd:xxx.cloud.redpanda.com:]: 1/1 brokers are down

Question

I am trying to setup the kafka engine and getting multiple errors.

Error which is too frequent
<Error> StorageKafka (user_kafka): [rdk:ERROR] [thrd:xxx.cloud.redpanda.com:]: 1/1 brokers are down

Error which is visible sometimes

2024.01.11 04:00:01.875084 [ 61840 ] {} <Error> StorageKafka (user_kafka): [rdk:ERROR] [thrd:app]: ClickHouse-clickhouse-poc-1.c.x.internal-test-user_kafka#consumer-2: xxx.cloud.redpanda.com:9092/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 0ms in state APIVERSION_QUERY, 4 identical error(s) suppressed)

My table and configurations are like this

// Base Table
CREATE TABLE test.user (name String, id Int32, email String)
    ENGINE = MergeTree()
    ORDER BY id;

// Kafka Engine Table
CREATE TABLE test.user_kafka (name String, id Int32, email String)
    ENGINE = Kafka()
    SETTINGS
        kafka_broker_list = 'seed-27ac08c5.chllqaunch049g9dc0t0.byoc.prd.cloud.redpanda.com:9092',
        kafka_topic_list = 'otlp_spans',
        kafka_group_name = 'user_group',
        kafka_format = 'Protobuf',
        kafka_schema = 'trace:User',
        kafka_num_consumers = 1,
        kafka_skip_broken_messages = 1,
        kafka_max_block_size = 1000000,
        kafka_row_delimiter = '\n'

CREATE MATERIALIZED VIEW test.user_mv TO test.user AS
SELECT name, id, email
FROM test.user_kafka;

Kafka Configuration inside config.xml

<clickhouse>
                <kafka>
                        <sasl_username>user</sasl_username>
                        <sasl_password>password</sasl_password>
                        <security_protocol>SASL_SSL</security_protocol>
                        <sasl_mechanisms>SCRAM-SHA-256</sasl_mechanisms>
                        <debug>all</debug>
                        <enable_ssl_certificate_verification>false</enable_ssl_certificate_verification>
                        <ssl_endpoint_identification_algorithm>none</ssl_endpoint_identification_algorithm>
                </kafka>
        </clickhouse>

Tried multiple configuration but still not able to setup, can someone please help

this is my redpanda/kafka configuration

export REDPANDA_BROKERS="seed-xxx.redpanda.com:9092"

export REDPANDA_SASL_MECHANISM="<SCRAM-SHA-256 or SCRAM-SHA-512>"

export REDPANDA_SASL_USERNAME="<username>"

export REDPANDA_SASL_PASSWORD="<password>"

I have tried every security protocol in the config.xml file of the clickhouse-server but always getting this error. I want to consume the data from redpanda.