Does Google Cloud ML Engine use grid search for tuning hyperparameters?

Question

The grid search technique is an easy to use and an embarrassingly parallel approach for finding the best set of hyperparameters for machine learning models. Does Google Cloud Machine Learning (ML) Engine use grid search?

Answer 1

Not by default. Grid search is easy to use and easy to understand but it suffers from the curse of dimensionality problem. Instead of grid search, Google Cloud ML Engine uses a Bayesian optimization technique that based on an algorithm called Gaussian process bandits.

The underlying technology used by Cloud ML Engine is from a Google Research project Vizier which is a Google-internal service for performing black-box optimization that has become the de facto parameter tuning engine at Google.

However, if you really want to use grid search, you can force Cloud ML Engine to use it by specifying the "algorithm" parameter in your hyperparameter yaml file as described in the Cloud ML Engine documentation