Unable to access H2O Flow using H2O_connection_url

Question

I am using both H2O and Sparkling Water on Amazon Clusters. I have been using Qubole and have been able to access the Flow UI on that platform. I am currently testing Databricks and Sagemaker, but I am unable to access the Flow UI using either platform (using port 54321). I am using H2O_cluster_version: 3.32.1.3. Do I need to use another port?

Answer 1

Getting the right Flow URL can be tricky because of the changes in the base URL at DBC. There were some improvements in more recent releases of SW that give the proper URL within Databricks, so make sure you try the latest version.

You should get it from your print/output, when you create an H2OContext. The port would be 9009. If you want to change it, you can use spark.ext.h2o.client.web.port.

You can also find the link in "Spark UI" -> "Sparkling Water" tab

The format would be something like: https://your-dbc-domain/driver-proxy/o/xxxxxxxx/yyyyyyy/9009/flow/index.html

From the docs for reference:

Flow is accessible via the URL printed out after H2OContext is started. Internally we use open port 9009. If you have an environment where a different port is open on your Azure Databricks cluster, you can configure it via spark.ext.h2o.client.web.port or corresponding setter on H2OConf.

Answer 2

Did you get the HTTP ERROR 500 java.lang.NoSuchMethodError: org.apache.spark.ui.UIUtils$.listingTable(Lscala/collection/Seq;Lscala/Function1;Lscala/collection/Iterable;ZLscala/Option;Lscala/collection/Seq;ZZLscala/collection/Seq;)Lscala/collection/Seq; ? There's seems to be the breaking change since spark 3.2 and the issues is still open. https://github.com/h2oai/sparkling-water/issues/2870

You could add more details in the issues post with the AWS also.