Kafka Go Client using Go Modules

892 Views Asked by At

I'm trying to implement Kafka producer in Go using Confluent-Kafka Client for Go, following the documentation here https://github.com/confluentinc/confluent-kafka-go#using-go-modules

My setup: Have a Lambda Function that reads Kafka Messages from Topic1 and sends out a different kafka Message to Topic2 after some processing.

For creating KafkaProducer, here's the code:

    producer, err := kafka.NewProducer(&kafka.ConfigMap{
        "bootstrap.servers": "host:port",
        "sasl.mechanisms":   "PLAIN",
        "security.protocol": "SASL_SSL",
        "sasl.jaas.config":  fmt.Sprintf("org.apache.kafka.common.security.plain.PlainLoginModule required username='%s' password='%s';", params.UserName, params.Password),
        "client.dns.lookup": "use_all_dns_ips"})

    if err != nil {
        fmt.Printf("Failed to create producer: %s", err)
        os.Exit(1)
    }

To send the kafka Message:

    err := producer.Produce(&kafka.Message{
        TopicPartition: kafka.TopicPartition{Topic: Topic2, Partition: kafka.PartitionAny},
        Key:            []byte(MyId),
        Value:          []byte(MyId),
    }, nil)
    if err != nil {
        fmt.Println("Error while sending message:", err)
        return
    }

To build this, I have a Makefile with command: env GOARCH=amd64 GOOS=linux go build -tags musl -ldflags="-s -w" -o bin/app cmd/app.go

make build throws error

internal/messageBus/createProducer.go:10:44: undefined: kafka.Producer
internal/messageBus/createProducer.go:14:25: undefined: kafka.NewProducer
internal/messageBus/createProducer.go:14:44: undefined: kafka.ConfigMap
make: *** [build] Error 2

I don't think I misread the documentation mentioned above, maybe something's missing in documentation??

0

There are 0 best solutions below