How to use both mysql & postgresql extensions in Druid?

Question

Druid only allows one of mysql-metadata-storage or postgresql-metadata-storage in the extensions loadList. If we try to add both, it fails. But we have a use case where we are using mysql for metadata storage, and postgres for lookups and need to use both.

druid.extensions.loadList=["druid-s3-extensions","druid-kafka-indexing-service","druid-multi-stage-query","druid-histogram","druid-kinesis-indexing-service","clarity-emitter","druid-parquet-extensions","druid-azure-extensions","druid-datasketches","mysql-metadata-storage","postgresql-metadata-storage","druid-lookups-cached-global"]

This does not work. It only accepts either mysql or postgresql extension.

How do I add postgres Driver to Druid's classpath?

Answer 1

If using Apache Druid's docker or helm charts, Druid uses this file as the Docker entrypoint.

https://github.com/apache/druid/blob/master/distribution/docker/druid.sh#L43

Use this environment variable to provide additional jars to Druid classpath (in our case, we need to pass the postgresql driver jar).

DRUID_ADDITIONAL_CLASSPATH: "/opt/druid/extensions/postgresql-metadata-storage/postgresql-42.4.3.jar"

This can also be passed in helm's values.yaml in configVars - https://github.com/apache/druid/blob/master/helm/druid/values.yaml#L34.

All values here are passed as env variables to Druid.

So, use the above variable to include Postgresql driver jar in classpath while mysql can simply be loaded as an standard extension.

druid_extensions_loadList: '[..., "mysql-metadata-storage"]