Java - sometimes return NaN when calculating Mahalanobis distance

Question

I use org.apache.mahout.common.distance.MahalanobisDistanceMeasure to calculate the distance between rows in a matrix and the mean vector, but it sometimes returns NaN. I tried to debug and it seems that a NullPointerException is thrown in the Object class. But for other rows everything is fine. I'd be grateful if someone could give me some guidance.

import com.opencsv.CSVReader;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;

import com.opencsv.CSVWriter;
import org.apache.commons.math.MathException;
import org.apache.mahout.common.distance.MahalanobisDistanceMeasure;
import org.apache.mahout.math.*;
import org.apache.mahout.math.Matrix;
import org.apache.commons.math3.linear.RealMatrix;
import org.apache.commons.math3.stat.correlation.Covariance;
import org.apache.commons.math.distribution.ChiSquaredDistributionImpl;

public class FindMultivariateOutliers {
    public static void main(String[] args) {
        String url = "VIC_20160401_201606301.csv";
        double[][] data = extractRealData(readCSV(url), 3);
        double[][] dataSet = new double[30][49];
        for (int i = 30; i < 60; i++) {
            dataSet[i-30] = data[i];
        }

        double[] mean = calculateMeanVector(dataSet);
        Vector meanVector = new DenseVector(mean);
        Matrix covarianceMatrix = covarianceMatrix(dataSet);

        MahalanobisDistanceMeasure measure = new MahalanobisDistanceMeasure();
        measure.setMeanVector(meanVector);
        measure.setCovarianceMatrix(covarianceMatrix);

        for (int i = 0; i < dataSet.length; i++) {
            DenseVector ve = new DenseVector(dataSet[i]);
            double x = measure.distance(dataSet[centroid(dataSet)[0]][centroid(dataSet)[1]],meanVector,ve);
            System.out.println(i+" "+x);
        }
}

Outputs:

0 NaN
1 NaN
2 1.3382137932701006
3 5.140281428741069
4 5.448118335171329
5 4.658774790167001
6 3.055235041048766
7 5.577659807980593
8 2.9899726295069784
9 6.095988936666251
10 5.188517209151716
11 3.2929774499538014
12 5.090550175124932
13 5.801822265633947
14 4.714239296215186
15 5.02905587450129
16 4.981122780626051
17 5.195044166268684
18 5.325097238194922
19 4.7899888250142375
20 5.506442897174045
21 5.266585564849615
22 5.403384368592266
23 4.110229775894713
24 5.960687924915147
25 4.5745629099807745
26 5.0580441561885205
27 5.146058878694013
28 5.1375323540721425
29 3.7919178679466015

centroid()is a method that calculate centroid of a matrix, returns int[2](first element is x coordinate and the second is y). dataSet is the matrix I'm focusing on.

Answer 1

"NaN" stands for "not a number". "Nan" is produced if a floating point operation(Double/Float) has some input parameters that cause the operation to produce some undefined result. For example, 0.0 divided by 0.0 is arithmetically undefined. Taking the square root of a negative number is also undefined.

NaN is static variable in Double and Float wrapper classes.It is Not a Numeric value so in Your Scenario, When you try to measure the distance between two coordinate it return Double.NaN, then It tries to convert it into a primitive number so it will give you a NullPointerException.