OpenStack's Ceilometer metering notifications export to kafka - Most efficient way

Question

as in the subject I need to put the defined metrics' measures on a kafka topic in order to feed our monitoring system. I've seen that Ceilometer's Kafka publisher is deprecated so it's not a deal, but it's possible to:

• configure the publisher for writing to a file;
• use a notifier like amqp, even if in this case is not clear if I've to setup an external broker.

Both solutions seem compliant with Kafka-Connect. Can anybody share his/her experience about this stuff? OpenStack's documentation is a little poor about this topic (at least for me). Any suggestion would be of help. Thanks for your time ny

Answer 1

As you said, kafka publisher was deprecated in very new version due to adapt ceilometer-gnocchi solution. But this doesn't mean that you can not use the deprecated code and former abilities.

You can just configure kafka publisher in pipeline.yaml file directly.

---
sources:
    - name: meter_kafka
      interval: 600
      meters:
          - "*"
      sinks:
          - kafka_sink
sinks:
    - name: kafka_sink
      transformers:
      publishers:
          - kafka://[kafka_broker_ip]:[kafka_broker_port]?topic=[topic]