P-value in XGBresressor

Question

How to compute the p-value in XGBregressor and SVR? The suggested method is Bonferroni's method. I know adjusted p-value= min{1, raw p-value * the number of features}. In fact, I do not know how to compute the p-value.

Answer 1

YOu can do this (this is an example):

from sklearn.datasets import fetch_california_housing
from sklearn.feature_selection import f_regression
import xgboost as xgb

california_housing = fetch_california_housing(as_frame=True)

X, y = california_housing.data, california_housing.target
xgb_reg = xgb.XGBRegressor()
xgb_reg.fit(X, y)

f_vals, p_vals = f_regression(X, y)

feat_idx = 0

raw_p_val = p_vals[feat_idx]
num_features = X.shape[1]

adj_p_val = min(1, raw_p_val * num_features)

print("Raw p-value:", raw_p_val)
print("Adjusted p-value:", adj_p_val)

which givs

Raw p-value: 0.0
Adjusted p-value: 0.0