Why features extraction?

Question

In pattern recognition why is it important feature extraction? Why we have to reduce the feature space? Is only a computational problem or this procedure improves the generalization ability of a classifier?

Answer 1

Feature extraction is not just about computational complexity. For a fixed number of training samples, if the number of features becomes sufficiently large, performance of the classifier can decrease significantly. See Curse of Dimensionality.

As a practical case, consider a classifier using multivariate normal statistics (mean and covariance). For N training samples and k features, the covariance matrix will become singular for N < k. Therefore, if the number of samples cannot be increased, it is necessary to reduce the number of features in order to use the classifier.

Answer 2

Both: a smaller feature space is (obviously?) more tractable, and eliminating unimportant details removes things that need to be generalized "around".