Replace values in numpy array containing NaN

Question

aa = np.array([2.0, np.NaN])
aa[aa>1.0] = np.NaN

On running the code above, I get the foll. warning, I understand the reason for this warning, but how to avoid it?

RuntimeWarning: invalid value encountered in greater

Answer 1

Store the indices of the valid ones (non - NaNs). First off, we will use these indices to index into the array and perform the comparison to get a mask and then again index into those indices with that mask to retrieve back the indices corresponding to original order. Using the original-ordered indices, we could then assign elements in the input array to NaNs.

Thus, an implementation/solution would be -

idx = np.flatnonzero(~np.isnan(aa))
aa[idx[aa[idx] > 1.0]] = np.nan

Sample run -

In [106]: aa  # Input array with NaNs
Out[106]: array([  0.,   3.,  nan,   0.,   9.,   6.,   6.,  nan,  18.,   6.])

In [107]: idx = np.flatnonzero(~np.isnan(aa)) # Store valid indices

In [108]: idx
Out[108]: array([0, 1, 3, 4, 5, 6, 8, 9])

In [109]: aa[idx[aa[idx] > 1.0]] = np.nan # Do the assignment

In [110]: aa #  Verify
Out[110]: array([  0.,  nan,  nan,   0.,  nan,  nan,  nan,  nan,  nan,  nan])