sklearn: Evaluating LinearSVC's AUC

Question

I know that one would evaluate the AUC of sklearn.svm.SVC by passing in the probability=True option into the constructor, and having the SVM predict probabilities, but I'm not sure how to evaluate sklearn.svm.LinearSVC's AUC. Does anyone have any idea how?

I'd like to use LinearSVC over SVC because LinearSVC seems to train faster on data with many attributes.

Answer 1

You can use the CalibratedClassifierCV class to extract the probabilities. Here is an example with code.

from sklearn.svm import LinearSVC
from sklearn.calibration import CalibratedClassifierCV
from sklearn import datasets

#Load iris dataset
iris = datasets.load_iris()
X = iris.data[:, :2] # Using only two features
y = iris.target      #3 classes: 0, 1, 2

linear_svc = LinearSVC()     #The base estimator

# This is the calibrated classifier which can give probabilistic classifier
calibrated_svc = CalibratedClassifierCV(linear_svc,
                                        method='sigmoid',  #sigmoid will use Platt's scaling. Refer to documentation for other methods.
                                        cv=3) 
calibrated_svc.fit(X, y)


# predict
prediction_data = [[2.3, 5],
                   [4, 7]]
predicted_probs = calibrated_svc.predict_proba(prediction_data)  #important to use predict_proba
print predicted_probs

Answer 2

Looks like it's not possible. https://github.com/scikit-learn/scikit-learn/issues/4820