Val_loss is increasing and val_accuracy is decreasing after around 30 epochs in keras

Question

we have a few problems with our val_loss and val_acc. After a few epochs (around 30) the val_acc is going down at around 50-60% and the val_loss is increasing to between 0.98 - 1.4 (see the pictures below). At the end of the post is the end of the 45th epoch.

[

    import pickle
    from datetime import time
    import matplotlib.pyplot as plt
    import numpy as np
    import tf as tf
    from keras import optimizers
    from keras.models import Sequential
    from keras.layers import *
    from keras.callbacks import TensorBoard
    from keras.utils import np_utils

    pickle_in = open("X.pickle", "rb")
    X = pickle.load(pickle_in)

    pickle_in = open("y.pickle", "rb")
    y = pickle.load(pickle_in)

    pickle_in = open("PredictionData\\X_Test.pickle", "rb")
    X_Test = pickle.load(pickle_in)

    X = X/255.0
    X_Test = X_Test/255.0

    y = np_utils.to_categorical(y, 5)

    NAME = "Emotion Detection"

    model = Sequential()

    model.add(Conv2D(32, (1, 1), activation="relu", use_bias=True,
                     bias_initializer="Ones",
                     input_shape=(145, 65, 1),
                     dim_ordering="th"))

    model.add(Conv2D(64, (3, 3),
                     activation="relu"))

    model.add(Conv2D(128, (3, 3),
                     activation="relu"))
    model.add(Dropout(0.2))

    model.add(Conv2D(64, (3, 3),
                     activation="relu"))

    model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors

    model.add(Dense(128,
                    activation="relu"))
    model.add(Dropout(0.2))

    model.add(Dense(32,
                    activation="relu"))

    model.add(Dense(5,
                    activation='sigmoid'))

    tensorboard = TensorBoard(log_dir="Tensorboard\\".format(time))

    sgd = optimizers.SGD(lr=0.001, decay=1e-6,
                         momentum=0.9, nesterov=True)

    model.compile(loss="categorical_crossentropy",
                  optimizer=sgd,
                  metrics=['accuracy'])

    history = model.fit(X, y, batch_size=16,
                        epochs=45, validation_split=0.12,
                        callbacks=[tensorboard])


    plt.plot(history.history['accuracy'])
    plt.plot(history.history['val_accuracy'])
    plt.title('Model accuracy')
    plt.ylabel('Accuracy')
    plt.xlabel('Epoch')
    plt.legend(['Accuracy', 'Val_Accuracy'], loc='upper left')
    plt.show()

    plt.plot(history.history['loss'])
    plt.plot(history.history['val_loss'])
    plt.title('Model loss')
    plt.ylabel('Loss')
    plt.xlabel('Epoch')
    plt.legend(['Loss', 'Val_Loss'], loc='upper left')
    plt.show()

    classes = model.predict(X_Test)
    plt.bar(range(5), classes[0])
    plt.show()
    print("prediction: class", np.argmax(classes[0]))


    model.summary()

    model.save("TrainedModel\\emotionDetector.h5") 

2493/2493 [==============================] - 35s 14ms/step - loss: 0.2324 - accuracy: 0.9202 - val_loss: 1.3789 - val_accuracy: 0.6353

_________________________________________________________________
Layer (type)                 Output Shape              Param    
=================================================================
conv2d_1 (Conv2D)            (None, 32, 65, 1)         4672      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 30, 63, 64)        640       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 28, 61, 128)       73856     
_________________________________________________________________
dropout_1 (Dropout)          (None, 28, 61, 128)       0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 26, 59, 64)        73792     
_________________________________________________________________
flatten_1 (Flatten)          (None, 98176)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 128)               12566656  
_________________________________________________________________
dropout_2 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_2 (Dense)              (None, 32)                4128      
_________________________________________________________________
dense_3 (Dense)              (None, 5)                 165       
_________________________________________________________________
Total params: 12,723,909
Trainable params: 12,723,909
Non-trainable params: 0
_________________________________________________________________

Hopefully you can help us. Thanks in advance.

Answer 1

These plots are a classic example of overfitting. I recommend watching it https://en.wikipedia.org/wiki/Overfitting