Factor Analysis in sklearn: Explained Variance

Question

PCA in scikit-learn has an attribute called "explained_variance" which captures the variance explained by each component. I don't see a similar thing like this for FactorAnalysis in scikit-learn. How can I compute the variance explained by each component for Factor Analysis?

Answer 1

Here is how you can do it :

First get the components matrix and the noise variance once you have performed factor analysis,let fa be your fitted model

m = fa.components_
n = fa.noise_variance_

Square this matrix

m1 = m**2

Compute the sum of each of the columns of m1

m2 = np.sum(m1,axis=1)

Now the %variance explained by the first factor will be

pvar1 = (100*m2[0])/np.sum(m2)

similarly, second factor

pvar2 = (100*m2[1])/np.sum(m2)

However, there is also a variance explained by the noise component, if you account for that in your variance explained you will need to compute

pvar1_with_noise = (100*m2[0])/(np.sum(m2)+np.sum(n))
pvar2_with_noise = (100*m2[1])/(np.sum(m2)+np.sum(n))

and so on. Hope this helps.

Answer 2

In terms of the usual nomenclature of FA/PCA, the components_ output by scikit-learn may be referred to as loadings elsewhere. For example, the package FactorAnalyzer outputs loadings_ which are equivalent, once you change the settings to match scikit-learn (i.e. set rotation=None, set method='ml', and make sure your data is standardized when input into the scikit-learn function, as FactorAnalyzer standardizes the data internally).

Compared to the components_ output of PCA from scikit-learn, which are unit-length eigenvectors, the FA ones are already scaled, so the explained variance can be extracted by summing the squares. Note that proportion of variance explained is expressed here in terms of the total variance of the original variables, not the variance of the factors, as in the answer from @Gaurav.

from sklearn.decomposition import FactorAnalysis
k_fa = 3   # e.g.

fa_k = FactorAnalysis(n_components=k_fa).fit(X_in)

fa_loadings = fa_k.components_.T    # loadings

# variance explained
total_var = X_in.var(axis=0).sum()  # total variance of original variables,
                                    # equal to no. of vars if they are standardized

var_exp = np.sum(fa_loadings**2, axis=0)
prop_var_exp = var_exp/total_var
cum_prop_var_exp = np.cumsum(var_exp/total_var)

print(f"variance explained: {var_exp.round(2)}")
print(f"proportion of variance explained: {prop_var_exp.round(3)}")
print(f"cumulative proportion of variance explained: {cum_prop_var_exp.round(3)}")

# e.g. output:
#   variance explained: [3.51 0.73]
#   proportion of variance explained: [0.351 0.073]
#   cumulative proportion of variance explained: [0.351 0.425]