Add custom logs in azure HDInsight application

Question

I am deploying my scala+apache spark 2.0 application on azure HDInsight cluster. We can see default yarn logs of the application through azure portal. But, Our requirement is to add our own custom logger (error, debug logs) for application specific (business cases) logs. We are not able to create custom logger which can be accessible outside the cluster (by storing azure blob storage).

Answer 1

We are working towards HDInsight Integration with Azure Operations management suite. Which let's you add custom logs in addition to common Spark logs. Let us know if this is something you are interested in and we can invite you to our preview.

Thanks, Ashish [email protected]