Access composite data in Postgres via Apache Drill

Question

I am fairly new to Apache Drill. I need to query my PostgreSQL database using Apache Drill. I went through the documentation and could configure a storage plugin for my Postgres database.

{

  "type": "jdbc",

  "driver": "org.postgresql.Driver",

  "url": "jdbc:postgresql://localhost/mydb",

  "username": <username>,

  "password": <password>,

  "enabled": true

}

I can successfully query tables with primary datatypes but unable to query the ones that have composite datatype. I have tried querying it using various ways, example.

This is the exception i get:

Caused by: java.lang.NullPointerException: null
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:201)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructure(RelStructuredTypeFlattener.java:225)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.restructureFields(RelStructuredTypeFlattener.java:205)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.RelStructuredTypeFlattener.rewrite(RelStructuredTypeFlattener.java:184)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.calcite.sql2rel.SqlToRelConverter.flattenTypes(SqlToRelConverter.java:435)
~[calcite-core-1.4.0-drill-r21.jar:1.4.0-drill-r21]
at
org.apache.drill.exec.planner.sql.SqlConverter.toRel(SqlConverter.java:270)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRel(DefaultSqlHandler.java:638)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:196)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:165)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:131)
~[drill-java-exec-1.11.0.jar:1.11.0]
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:79)
~[drill-java-exec-1.11.0.jar:1.11.0]
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:1050)
[drill-java-exec-1.11.0.jar:1.11.0]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:280)
[drill-java-exec-1.11.0.jar:1.11.0]

Is querying composite datatype in drill not feasible at all? Please let me know if am missing something.

Thanks

Answer 1

Apache Drill v1.11 documentation for data types: https://drill.apache.org/docs/supported-data-types/

In this release of Drill, you cannot reference a composite type by name in a query, but Drill supports array values coming from data sources.

EDIT

I suggest trying the ways provided by the Apache Drill community in the documentation. This is a bit from the section Composite Types:

For example, you can use the index syntax to query data and get the value of an array element:

a[1]

You can refer to the value for a key in a map using dot notation:

t.m.k

The section “Query Complex Data” shows how to use composite types to access nested arrays. "Handling Different Data Types" includes examples of JSON maps and arrays. Drill provides functions for handling array and map types:

• "KVGEN"
• "FLATTEN"