How can I optimize the number of layers and hidden layer size in a neural network using MLPClassifier
from sklearn and skopt
?
Usually I'd specify my space something like:
Space([Integer(name = 'alpha_2', low = 1, high = 2),
Real(10**-5, 10**0, "log-uniform", name='alpha_2')])
( let's say hyperparameters alpha_1
and alpha_2
).
With the neural network implementation in sklearn I need to tune hidden_layer_sizes
which is a tuple:
hidden_layer_sizes : tuple, length = n_layers - 2, default=(100,)
The ith element represents the number of neurons in the ith
hidden layer.
How can I represent this in Space
?
If you are using
gp_minimize
you can include the number of hidden layers and the neurons per layer as parameters inSpace
. Inside the definition of the objective function you can manually create the hyperparameterhidden_layer_sizes
.This is an example from the scikit-optimize homepage, now using an
MLPRegressor
: