WSO2 ML Cross Validation and Grid Search

48 Views Asked by At

I would like to know if the WSO2 ML implement Cross-Validation and Grid Search for best model selection.

1

There are 1 best solutions below

0
On

Presently, (as of version 1.1.0) WSO2 Machine Learner does not have a direct method for hyper-parameters optimization. As mentioned in your question, we are planning to include Random Search and Grid Search in one of the upcoming releases. In order to track the progress of this process, I have created a public JIRA [1]. So when the new feature is ready I will notify you via this SO Question.

Next, let me briefly describe cross-validation process we use in WSO2 Machine Learning server. In the third step of the ML Wizard of the ML Server, you can set the training data fraction (please see the attached screen shot).

enter image description here

So let's say you pick 0.7 of your data for training. Then, model building process will use 70% of your data for training and rest of the dataset (i.e. 30%) will be used for cross-validation. As you might recognize this a most basic approach for cross-validation and it is not particularly suitable for small datasets. So in upcoming releases, we are planning to include K-fold cross-validations [2] in addition to the currently available cross-validation method.

Yandi, if you need further help regarding this question or anything related to our product please let me know.

Thanks,

Upul

[1] https://wso2.org/jira/browse/ML-313

[2] https://en.wikipedia.org/wiki/Cross-validation_(statistics)#k-fold_cross-validation