Anomaly Detection Using Keras - Low Prediction Rate

Question

I built an Anomaly detection system using Autoencoder, implemented in keras. My input is a normalized vector with length 13. My dataset contains about 25,000 non anomaly inputs which dedicated for learning. I get about 10^-5 MSE after learning with 1-3 epochs. The problem is that although I get to a small MSE my AE can't detect anomalies good enough.. Model class:

class AutoEncoder:
    def __init__(self, inputLen, modelName,Batch,epochs):
        self.modelName       = modelName
        self.DL_BATCH_SIZE   = Batch
        self.DL_EPOCHS       = epochs
        self.DL_LAYER1       = 200
        self.DL_LAYER2       = 150
        self.DL_LAYER3       = 100
        self.DL_LAYEROUT     = 13
        self.DL_LOADMODEL    = None
        #self.DL_SAVEMODEL    = fileConfig.getConfigParam(DL_SAVEMODEL)
        #print(tensorflow.version.VERSION)
        
        
        if self.DL_LOADMODEL == None or self.DL_LOADMODEL == 0:
            my_init = keras.initializers.glorot_uniform(seed=1)
            self.dlModel = keras.models.Sequential()
            
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYER1, activation='tanh', input_dim=inputLen,kernel_initializer=my_init))
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYER2, activation='tanh',kernel_initializer=my_init))
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYER3, activation='tanh',kernel_initializer=my_init))
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYER2, activation='tanh',kernel_initializer=my_init))
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYER1, activation='tanh',kernel_initializer=my_init))
            self.dlModel.add(keras.layers.Dense(units=self.DL_LAYEROUT, activation='tanh',kernel_initializer=my_init))
            #sgd = keras.optimizers.SGD(lr=0.0001, decay=0.0005, momentum=0, nesterov=True)
            #adam = keras.optimizers.Adam(learning_rate=0.005,decay=0.005)
            simple_adam=keras.optimizers.Adam()
            self.dlModel.compile(loss='mse', optimizer=simple_adam, metrics=['accuracy'])
        else:
            self.dlModel = keras.models.load_model(self.DL_LOADMODEL + ".h5")

After training I find the max reconstruction MSE on a specific dataset of 2500 non anomalies. Then i test my Anomaly detector and mark as anomaly every input that its reconstruction has more than the max MSE*0.9 value.
find max error:

N = len(non_anomaly)
max_se = 0.0;
max_ix = 0
second_max=0
predicteds = kerasNN.predict(non_anomaly)
for i in range(N):
  curr_se = np.square(np.subtract(non_anomaly[i],predicteds[i])).mean()
  if curr_se > max_se:
    second_max=max_se
    max_se = curr_se; 
    max_ix = i

Testing the model:

    predicteds=kerasNN.predict(x_train_temp)

#errors vector generation
anomaly_binary_vector = []
i=0
anomalies=0
for x_original,x_reconstructed in zip(x_train_temp,predicteds):
    MSE=np.square(np.subtract(x_original,x_reconstructed)).mean()
    if(MSE>=0.95*max_se):
        anomaly_binary_vector.append(1)
    else:
        anomaly_binary_vector.append(0)
    i+=1

Output:

anomalies 2419
not detected anomalies   2031
non anomaly but marked as anomaly   2383
percentage of non anomaly instructions that marked as anomalies out of non anomalies : 0.3143384777733808
percentage of  anomaly instructions that wasn't detected out of anomalies: 0.8396031417941298

How can I Improve My anomaly detection?

Answer 1

A problem can be that you are "blowing up" you're autoencoder. You have an input of 13 and than map that to 200, 150, 100 and 13 again. There is no bottleneck in the network where it has to learn a compressed representation of the input.

Maybe try 13 -> 6 -> 4 -> 6 -> 13 or something like that. Than it lears a compressed representation and the task to reconstruct the input is not trivial anymore.

Additionally play around with other hyperparameters like the activation function. Maybe change it to 'relu' in the intermediate models.