I'm performing an hyperparameter tuning using both LinearSVC and SVC classes from scikit-learn and even though I'm performing 10 times more searches with the SVC class than with LinearSVC, the execution time is much short, what could be the reason for that? I thought that LinearSVC was more optimized.
I'm working with the Olivetti faces dataset
These are the two searches that I'm performing:
from sklearn.svm import LinearSVC
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
#LinearSVC hyperparameter tuning -----------------------
param_grid = [
{'svc__C': np.logspace(-3,3, num=10)}
]
full_pipeline = Pipeline([
("preprocessing", StandardScaler(with_mean=False)),
("svc",LinearSVC(random_state=0))
])
svc_rnd_search = GridSearchCV(full_pipeline, param_grid=param_grid, cv=10,
scoring='accuracy',n_jobs=-1)
# Measure the execution time
start_time = time()
#Run search
svc_rnd_search.fit(X_train, y_train)
# Calculate the execution time
end_time = time()
execution_time_ms = (end_time - start_time) * 1000
print(f"Execution time: {execution_time_ms:.3f}ms")
#SVC hyperparameter tuning-----------------------------------
param_grid = [
{'svc__C': np.logspace(-2,3, num=10),
'svc__gamma': np.logspace(-5,1, num=10),
'svc__kernel': ['rbf']}
]
full_pipeline = Pipeline([
("preprocessing", StandardScaler(with_mean=False)),
("svc",SVC())
])
svc_rnd_search = GridSearchCV(full_pipeline, param_grid=param_grid, cv=10,
scoring='accuracy',n_jobs=-1)
# Measure the execution time
start_time = time()
#Run search
svc_rnd_search.fit(X_train, y_train)
# Calculate the execution time
end_time = time()
execution_time_ms = (end_time - start_time) * 1000
print(f"Execution time: {execution_time_ms:.3f}ms")
The execution time for the first block of code (LinearSVC) is 1087635 ms whereas for the SVC class is 36961 ms.