Invalid parameter learning_rate for estimator RegressorChain(base_estimator=XGBRegressor

Question

I am trying to apply RandomizedSearchCV on a RegressorChain XGBoost model but I got an error : Invalid parameter learning_rate for estimator RegressorChain(base_estimator=XGBRegressor. If I comment all the values in grid dict, it works otherwise it doesn't accept any param.

Same models (XGBRegressor and RegressorChain) are working fine alone. The RandomizedSearchCV is not accepting the the params in grid dict

# Setup the parameters grid
grid = {
        'n_estimators': [100, 500, 1000],
        'max_depth': [5, 10, 20, 30],
        'max_features': ["auto", "sqrt"],
        'eta': [0.09, 0.1, 0.2],
        'booster': ["dart", "gblinear"]
        }



clf = XGBRegressor(objective='reg:squarederror')
chain = RegressorChain(base_estimator=clf, order=[0, 1, 2, 3, 4,5])

# Setup RandomizedSearchCV
rs_clf = RandomizedSearchCV(estimator=chain,
                            param_distributions=grid, 
                            n_iter=10, # number of models to try
                            cv=5,
                            verbose=1,
                            random_state=42,
                            refit=True) 

# Fit the RandomizedSearchCV version of clf
rs_clf.fit(X_train, y_train) # 'rs' is short

Answer 1

Since the XGBRegressor is the base_estimator of RegressorChain, the parameters of XGBRegressor become nested and must be addressed with base_estimator__xxx:

grid = {
    'base_estimator__n_estimators': [100, 500, 1000],
    'base_estimator__max_depth': [5, 10, 20, 30],
    'base_estimator__max_features': ["auto", "sqrt"],
    'base_estimator__eta': [0.09, 0.1, 0.2],
    'base_estimator__booster': ["dart", "gblinear"]
}