When we use polynomial features in case of polynomial regression, logistic regression, svm , does the loss function becomes non convex ?
Does loss function becomes non convex when we add polynomial features?
777 Views Asked by arjun pukale At
1
There are 1 best solutions below
Related Questions in MACHINE-LEARNING
- How to cluster a set of strings?
- Enforcing that inputs sum to 1 and are contained in the unit interval in scikit-learn
- scikit-learn preperation
- Spark MLLib How to ignore features when training a classifier
- Increasing the efficiency of equipment using Amazon Machine Learning
- How to interpret scikit's learn confusion matrix and classification report?
- Amazon Machine Learning for sentiment analysis
- What Machine Learning algorithm would be appropriate?
- LDA generated topics
- Spectral clustering with Similarity matrix constructed by jaccard coefficient
- Speeding up Viterbi execution
- Memory Error with Classifier fit and partial_fit
- How to find algo type(regression,classification) in Caret in R for all algos at once?
- Difference between weka tool's correlation coefficient and scikit learn's coefficient of determination score
- What are the approaches to the Big-Data problems?
Related Questions in REGRESSION
- Backwards stepwise regression approach in Stata 13
- Determining regression coefficients for data - MATLAB
- For Loop for Regression based on variable name in R
- How to find algo type(regression,classification) in Caret in R for all algos at once?
- Trying to find a best fit line for multiple noisy sine waves
- Why does this model fail?
- Coefficients of my polynomial model in R don't match graph
- glmnet training throws error on x,y dataframe arguments: am I using it wrong?
- An error while looping a linear regression
- How to output "Summary of Stepwise Selection" table from SAS PROC REG into data set?
- SAS RobustReg Procedure output parameter estimates
- How to find an optimal adstock decay factor for an independent variable in panel analysis in R?
- Regression of a complex model in R
- How to get values on testdata in RSNNS
- perl regression without intercept
Related Questions in CLASSIFICATION
- Feature selection SVM-Recursive Feature elimination (SVM-RFE) with Libsvm, the accuracy result is worse than without feature selection, why?
- How to find algo type(regression,classification) in Caret in R for all algos at once?
- scatter plot for a multiclass dataset with class imbalance and class overlapping
- Gaussian Naive Bayes classification
- Train And Use Classifier Weka In Java
- ROC curve in R using rpart package?
- Matlab example code for deep belief network for classification
- Chained CostSensitiveClassifier
- Difference between segmentation and classification
- How to train a LogisticRegression classifier to read from a second dataset?
- Tagging a phrase to learn a classifier using NLTK in Python
- Which classifiers provide weight vector?
- NaiveBayes Classifier: Do I have to concatenate all files of one class?
- Scikit Learn - Identifying target from loading a CSV
- Deciding output style for ANN classifier
Related Questions in LOSS-FUNCTION
- keras combining two losses with adjustable weights
- Keras neural network outputting function parameters / how to construct loss function?
- Weighted mse custom loss function in keras
- Keras: Which loss function for grayscale image as label
- What exactly does the loss value give while training a LSTM model in keras?
- How to use an external variable in a custom loss function for an lstm model in Keras?
- Custom loss function Tensorflow / Keras penalizing relative distance
- Multiple loss functions on (somewhat) overlapping sub-models in Keras
- Difficulties calculating mean square error between 2 tensors
- Appropriate loss function in pytorch when output is an array of float numbers
- IndexError: Target 60972032 is out of bounds
- Use additional *trainable* variables in Keras/Tensorflow custom loss function
- How can I create a custom loss function in keras ? (Custom Weighted Binary Cross Entropy)
- Tensorflow InvalidArgumentError: 2 root error(s) found. indices[28,0] = 11292 is not in [0, 11272)
- How is cross entropy loss work in pytorch?
Related Questions in NON-CONVEX
- Cubic and non convex optimisation problems in portfolio optimisation
- SDP Problem: No Correct Solutions Obtained using cvxpy in Python
- Wrong convergence of a complex MINLP
- Non-convex function optimization
- Non convex quadratic optimization solvers in python
- finding the global maximum of an unknown surface
- CPLEX: Error 5002 Objective is not convex -> Problem can be solved to global optimality with optimality target 3 ->
- does CPLEX really bound the global solution of a nonconvex MIQP?
- Does loss function becomes non convex when we add polynomial features?
- Suggesting a bound for an MINLP solver
- How to compute the intersection between a convex polyhedron and another polyhedron?
- How important is to formulate a convex optimization for a proposed algorithm?
- Standard mesh for Concave Hexagons with two Mouths?
- Finding a common interior point to two polygons
- find the maximum convex area
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
If a loss function is convex for any choice of
X -> yyou're trying to estimate then adding a fixed set of polynomial features won't change that. You're simply trading your initial problem with the estimation problemX' -> y, whereX'has the additional features.If you're additionally trying to estimate the parameters for the new feature(s) then it's pretty easy to get a non-convex loss in those dimensions (assuming there are parameters to choose -- if you're just talking about adding a polynomial basis then this doesn't apply).
As some measure of proof, take the example of a 1D estimation problem and choose the feature
f(x) = (x-a)^3. Assume your dataset has the single point(0, 0). With a little work you can show that the loss even for linear regression over the new feature is non-convex in places with respect to the parametera. Note that the loss IS still convex with respect to the new features -- standard linear regression always satisfies that property -- it's the fact that we used linear regression along with a choice of polynomial to build a new non-convex regressor that causes this behavior.