Imputation MICE in R still NA left in dataset

Question

After running MICE package, the number of missing values are shrinked from 147428 to 46093 in each of the 5 complete imputation sets. But isn't it supposed to be 0 NAs instead???

Thanks!

Here is my MICE code:

imp = mice(newdata)

imputationSet1 = complete(imp)
imputationSet2 = complete(imp,2)
imputationSet3 = complete(imp,3)
imputationSet4 = complete(imp,4)
imputationSet5 = complete(imp,5)

Answer 1

Yeah there should be no missing values left.

I bet there are some rows in your data set that are so badly mangled with missingness that mice's imputation models break down. Is it possible that there are rows in your dataset where every value is missing? That would do it.

Another thing to try on a whim - crank up the number of iterations to 15: imp = mice(newdata, maxit = 15). Does that change anything?

Answer 2

Try to handover an additional parameter called threshold, whose default is 0.999. If you set this to something closer to 1 or even larger one, your problem should disappear.

Be aware though that this issue arises only if the collinearity in the data is high.

Answer 3

Ben, the mice() function detects multicollinearity, and solves the prob- lem by removing one or more predictors for the matrix. Each removal is noted in the loggedEvents element of the mids object. For example,

imp <- mice(cbind(nhanes, chl2 = 2 * nhanes$chl), print = FALSE)

imp$loggedEvents

informs us that the duplicate variable chl2 was removed before iteration. The algorithm also detects multicollinearity during iterations.

Another measure to control the algorithm is the ridge parameter. The ridge parameter is specified as an argument to mice(). Setting ridge=0.001 or ridge=0.01 makes the algorithm more robust at the expense of bias.

At the terminal node, we can apply a simple method like mice.impute.sample() that does not need any predictors for itself.

This information is taken from the book Flexible Imputation of Missing Data by Stef van Buuren, p. 129

Answer 4

What helped me is to convert as-character variables to as-factor variables and NAs have disappeared from the imputed dataset.