Keras Extremely High Loss

Question

I'm trying to predict price by characteristics. I chose a pretty simple model, but it works very strange. Loss function is extremely high and I can't see where the problem is.

Here is my model:

# define base model
def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(62, input_dim = 62, kernel_initializer='normal', activation='relu'))
    model.add(Dense(31, kernel_initializer='normal', activation='relu'))
    model.add(Dense(15, kernel_initializer='normal', activation='relu'))
    model.add(Dense(1, kernel_initializer='normal'))
    # Compile model
    model.compile(loss='mean_squared_error', optimizer='adam')
    return model

That's how I prepare the data: (One-Hot and I split all data to train and test)

df = encode_onehot(dataframe, cols=['Shape', 'Cut', 'Color', 'Clarity', 'Polish', 'Symmetry', 'Culet', '\tFluorescence'])

dataset = df.values
X = dataset[1:,4:66]
Y = dataset[1:,2]

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.25, random_state=42)

Finally, training:

baseline_model().fit(X_train, y_train, epochs=10, batch_size=64)
scores = baseline_model().evaluate(X_test, y_test, verbose=0)
print(baseline_model().summary())

And results are very sad:

Epoch 1/10
149767/149767 [==============================] - 4s - loss: 104759338.0333     
Epoch 2/10
149767/149767 [==============================] - 4s - loss: 104594236.9627     
Epoch 3/10
149767/149767 [==============================] - 4s - loss: 104556662.2948     

And it doesn't get better.

What am I doing wrong?

Answer 1

As @Yu-Yang said you are using mean squared error as loss function. I had this same problem before where the loss value will be very large, on changing the loss function to mean_squared_logarithmic_error, I got the desired result.

model %>% compile(
optimizer = optimizer_rmsprop(lr=0.0001),
loss = loss_mean_squared_logarithmic_error,
metrics = c("accuracy")
)

The loss value changed to

Epoch 1/10
326981/326981 [==============================] - 17s - loss: 0.0048 - acc: 0.9896

Hope this finds useful !

Answer 2

1. Normalize your dataset like preprocess step
2. As I can see, you are using a regression model, so with a regression model your loss will be completely different from what it would be if you used the model for classification
3. If you need classify. set the loss function in the likeness of categorical_crossentropy or another. And set activation func for last layer in your ml.

Best regards!