How does n_points in skopt BayesSearchCV work?

Question

I am confused on how does n_points work in skopt BayesSearchCV. As I understood, Bayes Search is sequential. But in skopt BayesSearchCV, we can set n_point parameter which specifies the number of parameter settings to sample in parallel. How does this parallelism work? Does it do n_points number of independent BayesSearches or does it perform batch Bayesian optimization?

Answer 1

Based on the source code, BayesSearchCV is generating and trying a batch of parameter sets of size n_points at each step of the optimization. (see BayesSearchCV._step and optimzer.ask)

So the parameter sets in the batch are generated with the same amount of "knowledge" of param space. This trades off more quickly searching the parameter space (assuming n_jobs > 1) with increased risk of trying poor parameter sets.

Note that the batch size will be subtracted from the n_iter tally, so there becomes a distinction between number of parameter sets tried and the number of iterations of Bayes optimization. For instance if n_iter=100 and n_points=5 then there will be 20 rounds of optimization.