How to run RedPanda Kafka container in GitLab CI for tests?

Question

I am wrestling with getting a job in GitLab to spin-up the RedPanda Kafka container so I can run my integration tests.

Locally, using this image vectorized/redpanda it works fine running my tests from my host (external) to the container.

My GitLab job looks like this below (the template is just invoking Maven)

I can connect to the broker but it is when I connect to the nodes that I have the issue.

I believe I need to change this --advertise-kafka-addr kafka:9092 but no matter what I do it does not connect.

kafka_tests:
  extends: .java_e2e_template
  services:
  - name: vectorized/redpanda
    alias: kafka
    command: [
      "redpanda",
      "start",
      "--advertise-kafka-addr kafka:9092",
      "--overprovisioned"
    ]
  variables:
    MVN_PROPERTIES: "-Dkafka.bootstrap.servers=kafka:9092"

Answer 1

First the container should be from redpandadata instead of vectorized

docker pull `redpandadata/redpanda:latest

I think gitlab ci runs docker underneath the hood, so you probably have to look at how you expose the host:port. Meaning the advertise address needs to be reachable by your clients.