Why is `sklearn.svm.LinearSVC` taking longer to execute than `sklearn.svm.SVC`?

40 Views Asked by At

I'm performing an hyperparameter tuning using both LinearSVC and SVC classes from scikit-learn and even though I'm performing 10 times more searches with the SVC class than with LinearSVC, the execution time is much short, what could be the reason for that? I thought that LinearSVC was more optimized.

I'm working with the Olivetti faces dataset

These are the two searches that I'm performing:

from sklearn.svm import LinearSVC
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

#LinearSVC hyperparameter tuning -----------------------
param_grid = [
    {'svc__C': np.logspace(-3,3, num=10)}
]

full_pipeline = Pipeline([
    ("preprocessing", StandardScaler(with_mean=False)),
    ("svc",LinearSVC(random_state=0))
    ])

svc_rnd_search = GridSearchCV(full_pipeline, param_grid=param_grid, cv=10,
                           scoring='accuracy',n_jobs=-1)

# Measure the execution time
start_time = time()

#Run search
svc_rnd_search.fit(X_train, y_train)

# Calculate the execution time
end_time = time()
execution_time_ms = (end_time - start_time) * 1000
print(f"Execution time: {execution_time_ms:.3f}ms")

#SVC hyperparameter tuning-----------------------------------
param_grid = [
    {'svc__C': np.logspace(-2,3, num=10),
     'svc__gamma':  np.logspace(-5,1, num=10),
     'svc__kernel': ['rbf']}
]

full_pipeline = Pipeline([
    ("preprocessing", StandardScaler(with_mean=False)),
    ("svc",SVC())
    ])

svc_rnd_search = GridSearchCV(full_pipeline, param_grid=param_grid, cv=10,
                           scoring='accuracy',n_jobs=-1)

# Measure the execution time
start_time = time()

#Run search
svc_rnd_search.fit(X_train, y_train)

# Calculate the execution time
end_time = time()
execution_time_ms = (end_time - start_time) * 1000
print(f"Execution time: {execution_time_ms:.3f}ms")

The execution time for the first block of code (LinearSVC) is 1087635 ms whereas for the SVC class is 36961 ms.

0

There are 0 best solutions below