Druid Mapreduce job not listed on Google Dataproc dashboard

68 Views Asked by At

I will explain my situation. I have an on-prem Apache Druid, I managed to let Druid execute the Hadoop ingestion job (Mapreduce) on a Dataproc cluster. The job is running fine, but the problem isn't visible on the Dataproc dashboard.

Note: The only connection between the Dataproc cluster and Druid is a VPN connection between the master node and the Druid cluster.

1

There are 1 best solutions below

1
On BEST ANSWER

If you are not submitting the MapReduce job through Dataproc API, but through other ways e.g., submitting to YARN directly, then it won't be available in the Dataproc dashboard. But in this case, you should be able to view the jobs through YARN resource manager UI in Component Gateway.