I just realized that support vector machine can be used for regression thanks to the nice article However, I am quite confused with the definition of the hyperparameter C.
I am well aware of the slack variables \xi_i associated with each data point and the hyperparameter C in classification SVM. There, the objective function is \min_{w, b} \frac{|w|}{2} + C\sum_{i=1}^N \xi_i, such that
y_i (w \cdot x_i + b) \ge 1 - \xi_i and \xi_i \ge 0.
In SVM, the larger C is, the larger the penalty and hence soft SVM reduces to hard SVM as C goes to infinity. (sorry for the raw latex code, i remember latex is supported but it seems not the case)
From the linked article, the objective function and the constraints are as follows
I think the equations also imply that the larger C is, the larger the penalty. However, the author of the article claims the opposite,
I noticed that someone asked the author the same question at the end of the article but there has been no response.
I guess there might be a typo in the equation, so I looked for support from any reference and then I found that SVR in Python uses the same convention that the strength of regularization is inversely proportional to C. I tried to check the source code of SVR but I can't find any formula. Can someone help resolve this? Thanks!
C is regularization parameter, which means its before the square of W, not before the relaxation, so the C parameter may be equal to 1/C.