I have been developing my model with XGBRegressor and I have a technical question about the early stopping rounds working. This is the code:
model = XGBRegressor(n_jobs=-1, n_estimators=1000, early_stopping_rounds=50, random_state=42)
model.fit(X_train, y_train, eval_set=[(X_train, y_train),(X_val,y_val)])
This is the output:
[0] validation_0-rmse:12740.82085 validation_1-rmse:14354.43509
[1] validation_0-rmse:9487.04070 validation_1-rmse:10616.13605
[2] validation_0-rmse:7283.04039 validation_1-rmse:8239.62752
[3] validation_0-rmse:5858.70801 validation_1-rmse:6670.77442
[4] validation_0-rmse:4965.48363 validation_1-rmse:5681.79305
[5] validation_0-rmse:4385.77247 validation_1-rmse:5110.99709
[6] validation_0-rmse:4043.46385 validation_1-rmse:4774.20268
[7] validation_0-rmse:3763.10815 validation_1-rmse:4635.01149
[8] validation_0-rmse:3591.72373 validation_1-rmse:4561.27221
[9] validation_0-rmse:3459.59007 validation_1-rmse:4520.31052
[10] validation_0-rmse:3364.49960 validation_1-rmse:4493.90137
[11] validation_0-rmse:3292.77090 validation_1-rmse:4488.63371
[12] validation_0-rmse:3261.68385 validation_1-rmse:4477.47174
[13] validation_0-rmse:3194.66407 validation_1-rmse:4479.46967
[14] validation_0-rmse:3131.46617 validation_1-rmse:4483.96315
[15] validation_0-rmse:3105.17158 validation_1-rmse:4496.17191
[16] validation_0-rmse:2980.64759 validation_1-rmse:4513.39312
[17] validation_0-rmse:2969.50538 validation_1-rmse:4515.67529
[18] validation_0-rmse:2928.73793 validation_1-rmse:4514.08650
[19] validation_0-rmse:2885.23440 validation_1-rmse:4512.02239
[20] validation_0-rmse:2877.47452 validation_1-rmse:4511.70923
[21] validation_0-rmse:2775.98275 validation_1-rmse:4541.79559
[22] validation_0-rmse:2665.77462 validation_1-rmse:4555.71092
[23] validation_0-rmse:2636.37427 validation_1-rmse:4549.20621
[24] validation_0-rmse:2562.55110 validation_1-rmse:4556.94927
...
[58] validation_0-rmse:1443.62609 validation_1-rmse:4736.20431
[59] validation_0-rmse:1423.84305 validation_1-rmse:4746.66728
[60] validation_0-rmse:1387.68330 validation_1-rmse:4747.30871
[61] validation_0-rmse:1367.73335 validation_1-rmse:4747.11801
I have understood that the model has trained 62 trees and the best number of trees is 13, but if 13 is the optimal number why does it report the results until 62 if the RMSE is increasing?