R - Get a matrix with the reduced number of features with SVD

Question

I'm using the SVD package with R and I'm able to reduce the dimensionality of my matrix by replacing the lowest singular values by 0. But when I recompose my matrix I still have the same number of features, I could not find how to effectively delete the most useless features of the source matrix in order to reduce it's number of columns.

For example what I'm doing for the moment:

This is my source matrix A:

  A B C D
1 7 6 1 6
2 4 8 2 4
3 2 3 2 3
4 2 3 1 3

If I do:

s = svd(A)
s$d[3:4] = 0  # Replacement of the 2 smallest singular values by 0
A' = s$u %*% diag(s$d)  %*% t(s$v)

I get A' which has the same dimensions (4x4), was reconstruct with only 2 "components" and is an approximation of A (containing a little bit less information, maybe less noise, etc.):

      [,1]     [,2]      [,3]     [,4]
1 6.871009 5.887558 1.1791440 6.215131
2 3.799792 7.779251 2.3862880 4.357163
3 2.289294 3.512959 0.9876354 2.386322
4 2.408818 3.181448 0.8417837 2.406172

What I want is a sub matrix with less columns but reproducing the distances between the different rows, something like this (obtained using PCA, let's call it A''):

        PC1        PC2
1 -3.588727  1.7125360
2 -2.065012 -2.2465708
3  2.838545  0.1377343   # The similarity between rows 3 
4  2.815194  0.3963005   # and 4 in A is conserved in A''

Here is the code to get A'' with PCA:

p = prcomp(A)
A'' = p$x[,1:2]

The final goal is to reduce the number of columns in order to speed up clustering algorithms on huge datasets.

Thank you in advance if someone can guide me :)

Answer 1

I would check out this chapter on dimensionality reduction or this cross-validated question. The idea is that the entire data set can be reconstructed using less information. It's not like PCA in the sense that you might only choose to keep 2 out of 10 principal components.

When you do the kind of trimming you did above, you're really just taking out some of the "noise" of your data. The data still as the same dimension.