h20 driverless ai ROC curve How to identify threshold for Multiclass Confusion Matrix

121 Views Asked by At

I have created a training model and ROC Curves shows 0.9748 on Multiclass Confusion Matrix. I ran this model on test data using "score on another dataset" and got the predictions. I would like to understand how to get the Threshold for these predictions so that we can publish the future values to the users.

1

There are 1 best solutions below

0
On

DAI returns prediction values not labels. This means you have to set the threshold yourself. For example you could download the predictions file, then import it into your favorite language (let's use H2O-3's python api for example) and then run an a boolean check to see if a given column has a value that is above the threshold for it to be a specific label.

Details on the multiclass experiment graphs and how DAI decides to display different threshold metrics can be found in the documentation here