I have an application running Spark/Iceberg/MinIO with Nessie catalog for Iceberg, everything works correctly (Iceberg data is beeing correctly loaded to both MinIO and Nessie).
Now I'm trying to connect Trino and it fails on executing GET request against 'http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL' (this request works when I execute from browser or Insomnia).
In Trino GUI this is caused by "java.net.ConnectException: undefined".
What am I missing?
I'm using Trino official helm chart 0.18.0 (running Trino version 432) and tested this on Nessie 0.77.1 (current newest) and 0.71.0 (matching Nessie version in Trino 432 pom.xml here: https://github.com/trinodb/trino/blob/432/plugin/trino-iceberg/pom.xml
My Trino connector properties:
connector.name=iceberg
iceberg.catalog.type=nessie
iceberg.nessie-catalog.uri=http://localhost:19120/api/v1
iceberg.nessie-catalog.default-warehouse-dir=s3a://data/test
hive.s3.aws-access-key=VUtRVIf0hg7szCp3k0Pz
hive.s3.aws-secret-key=lHzYClEjh2AH5mEfRdPS720pMl3UZl7riR3uL4pL
hive.s3.endpoint=http://127.0.0.1:9000
hive.s3.path-style-access=true
hive.s3.ssl.enabled=false
My Spark session configuration (Java):
SparkSession.builder()
.appName("Test App")
.master("local")
.config("spark.ui.enabled", "false")
.config("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem")
.config("fs.s3a.endpoint", "http://127.0.0.1:9000")
.config("fs.s3a.access.key", VUtRVIf0hg7szCp3k0Pz)
.config("fs.s3a.secret.key", lHzYClEjh2AH5mEfRdPS720pMl3UZl7riR3uL4pL)
.config("fs.s3a.path.style.access", "true")
.config("spark.sql.warehouse.dir", WAREHOUSE_PATH)
.config("spark.jars.packages", "org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:1.3.0,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.5_2.13:0.77.1")
.config("spark.sql.extensions", "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions")
.config("spark.sql.catalog.nessie", "org.apache.iceberg.spark.SparkCatalog")
.config("spark.sql.catalog.nessie.catalog-impl", "org.apache.iceberg.nessie.NessieCatalog")
.config("spark.sql.catalog.nessie.authentication.type", "NONE")
.config("spark.sql.catalog.nessie.uri", "http://localhost:19120/api/v1")
.config("spark.sql.catalog.nessie.ref", "main")
.config("spark.sql.defaultCatalog", "nessie")
.config("spark.sql.catalog.nessie.warehouse", "s3a://data/test")
.getOrCreate();
Entire stack trace from Trino GUI:
io.trino.spi.TrinoException: Error listing schemas for catalog nessie: Failed to execute GET request against 'http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL'.
at io.trino.metadata.MetadataListing.handleListingException(MetadataListing.java:349)
at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:97)
at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:88)
at io.trino.connector.informationschema.InformationSchemaPageSource.addSchemataRecords(InformationSchemaPageSource.java:322)
at io.trino.connector.informationschema.InformationSchemaPageSource.buildPages(InformationSchemaPageSource.java:226)
at io.trino.connector.informationschema.InformationSchemaPageSource.getNextPage(InformationSchemaPageSource.java:184)
at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:266)
at io.trino.operator.Driver.processInternal(Driver.java:395)
at io.trino.operator.Driver.lambda$process$8(Driver.java:298)
at io.trino.operator.Driver.tryWithLock(Driver.java:694)
at io.trino.operator.Driver.process(Driver.java:290)
at io.trino.operator.Driver.processForDuration(Driver.java:261)
at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887)
at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187)
at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565)
at io.trino.$gen.Trino_432____20240219_102654_2.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.projectnessie.client.http.HttpClientException: Failed to execute GET request against 'http://localhost:19120/api/v1/trees/tree/main?fetch=MINIMAL'.
at org.projectnessie.client.http.impl.jdk11.JavaRequest.executeRequest(JavaRequest.java:128)
at org.projectnessie.client.http.HttpRequest.get(HttpRequest.java:80)
at org.projectnessie.client.rest.v1.RestV1TreeClient.getReferenceByName(RestV1TreeClient.java:83)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.projectnessie.client.rest.v1.RestV1Client$ExceptionRewriter.invoke(RestV1Client.java:78)
at jdk.proxy4/jdk.proxy4.$Proxy236.getReferenceByName(Unknown Source)
at org.projectnessie.client.rest.v1.HttpGetReference.get(HttpGetReference.java:34)
at org.apache.iceberg.nessie.NessieIcebergClient.loadReference(NessieIcebergClient.java:114)
at org.apache.iceberg.nessie.NessieIcebergClient.lambda$new$0(NessieIcebergClient.java:81)
at org.apache.iceberg.relocated.com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:181)
at org.apache.iceberg.nessie.NessieIcebergClient.getRef(NessieIcebergClient.java:89)
at org.apache.iceberg.nessie.NessieIcebergClient.withReference(NessieIcebergClient.java:501)
at org.apache.iceberg.nessie.NessieIcebergClient.listNamespaces(NessieIcebergClient.java:207)
at io.trino.plugin.iceberg.catalog.nessie.TrinoNessieCatalog.listNamespaces(TrinoNessieCatalog.java:104)
at io.trino.plugin.iceberg.IcebergMetadata.listSchemaNames(IcebergMetadata.java:355)
at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.listSchemaNames(ClassLoaderSafeConnectorMetadata.java:197)
at io.trino.tracing.TracingConnectorMetadata.listSchemaNames(TracingConnectorMetadata.java:131)
at io.trino.metadata.MetadataManager.listSchemaNames(MetadataManager.java:254)
at io.trino.tracing.TracingMetadata.listSchemaNames(TracingMetadata.java:168)
at io.trino.metadata.MetadataListing.doListSchemas(MetadataListing.java:103)
at io.trino.metadata.MetadataListing.listSchemas(MetadataListing.java:94)
... 17 more
Caused by: java.net.ConnectException: undefined
at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:951)
at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
at org.projectnessie.client.http.impl.jdk11.JavaHttpClient.lambda$newRequest$0(JavaHttpClient.java:68)
at org.projectnessie.client.http.impl.jdk11.JavaRequest.executeRequest(JavaRequest.java:110)
... 38 more
Caused by: java.net.ConnectException: undefined
at java.net.http/jdk.internal.net.http.common.Utils.toConnectException(Utils.java:1028)
at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:227)
at java.net.http/jdk.internal.net.http.PlainHttpConnection.checkRetryConnect(PlainHttpConnection.java:280)
at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$2(PlainHttpConnection.java:238)
at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773)
... 3 more
Caused by: java.nio.channels.ClosedChannelException: undefined
at java.base/sun.nio.ch.SocketChannelImpl.ensureOpen(SocketChannelImpl.java:202)
at java.base/sun.nio.ch.SocketChannelImpl.beginConnect(SocketChannelImpl.java:786)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:874)
at java.net.http/jdk.internal.net.http.PlainHttpConnection.lambda$connectAsync$1(PlainHttpConnection.java:210)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
at java.net.http/jdk.internal.net.http.PlainHttpConnection.connectAsync(PlainHttpConnection.java:212)
... 9 more