How to declare parameter grid in XGBRegressor

Question

How do I implement the param_grid and get the best hyperparameters for xgb?

regressor = xgb.XGBRegressor()
regressor.fit(X_train,y_train)

param_grid = {
    'max_depth' :[3,4,5],
    'learning_rate':[0.1, 0.01, 0.5],
    'gamma':[0,0.25,1],
    'reg_lambda':[0, 1.0, 10.0],
    'scale_pos_weight':[1,3,5]
}

optimal_params = GridSearchCV(estimator = xgb.XGBRegressor(subsample=0.9, colsample_bytree=0.5), param_grid = param_grid, verbose = 0,)
optimal_params.fit(X_train, y_train, verbose = False)

It takes forever to compile and shows this warning repeatedly:

[16:45:55] WARNING: /workspace/src/objective/regression_obj.cu:152: 
reg:linear is now deprecated in favor of reg:squarederror.

Answer 1

This all looks correct.

You have 3*3*3*3*3=243 hyperparameter combinations to check, so it probably will take some time (you can estimate this by fitting one of them first and multiplying by 243, though of course some hyperparameters will affect the training time). You might consider RandomizedSearchCV instead if it is prohibitively long.

The warning is just a warning, not an error; you can silence it as it suggests, by setting your objective to reg:squarederror (or upgrading your xgboost package, since that is the current default).