I am working on a data ingestion flow based on the Spring Cloud Data Flow framework. I have created two custom microservices that use Kafka as a messaging broker and I use MongoDB pre-built sink to bulk the information. In a first phase, I would like to implement this architecture locally with Docker, therefore, I have prepared a docker-compose with several services such us Kafka, MongoDB, Spring Cloud Config Server.
At the time of deploying the stream, I have tried to configure the container network deployment property in order to join the container to the network that I have created with docker-compose and be able to use DNS to access Kafka and other services.
When the stream deployment is done, I can see on Skipper Server logs how it ignores the network property that I configured before and uses bridge at container network name.
o.s.c.d.spi.local.LocalAppDeployer : Command to be executed: docker run --network **bridge** --rm -e deployerId=test-stream.epg-ingestion-source-v1 -e SPRING_APPLICATION_JSON={"management.endpoints.web.exposure.include":"health,info,bindings","endpoints.jmx.unique-names":"true","endpoints.shutdown.enabled":"true","spring.cloud.dataflow.stream.name":"test-stream","management.metrics.tags.instance.index":"${vcap.application.instance_index:${spring.cloud.stream.instanceIndex:0}}","spring.cloud.stream.kafka.streams.binder.brokers":"PLAINTEXT://kafka:29092"
In addition to that, when I inspect the container metadata, I can see clearly mapping the network bridge to the container
Does anyone know if it is a bug or if it is necessary to configure it in another way?
I am using these versions:
- springcloud/spring-cloud-skipper-server:2.9.0-SNAPSHOT
- springcloud/spring-cloud-dataflow-server:2.10.0-SNAPSHOT-jdk11