How to train on the cosine similarity with the Keras Dot function?

Question

I want to use the cosine similarity in a Keras Model when comparing the embeddings of two documents.

• In which value range should the targets be for the training? Meaning in which value range could be the outcome of the Dot function? What value should the target be, if the embeddings are very similar and what should it be if they are very different?
• What does the axis parameter mean in this context? Is it right to set axis=1?

I have built my model as follows:

from tensorflow.python.keras.layers import Dot
from tensorflow.keras import Input, Model
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential

input_document1 = Input(200)
model_1 = Sequential()
model_1.add(Dense(100, activation='relu'))
encoded_document1 = model_1(input_document1)

input_document2 = Input(200)
model_2 = Sequential()
model_2.add(Dense(100, activation='relu'))
encoded_document2 = model_2(input_document2)

distance_layer = Dot(axes=1, normalize=True) # cosine proximity
prediction = distance_layer([encoded_document1, encoded_document2])

siamese_net = Model(inputs=[input_document1, input_document2], outputs=prediction)

In the documentation they say:

Whether to L2-normalize samples along the dot product axis before taking the dot product. If set to True, then the output of the dot product is the cosine proximity between the two samples.

Answer 1

The cosine similarity between outputs values between [-1,1], where 1 means that the vectors are completely opposite and the -1 means perfect overlapping/similarity.

You can have a look here on how to compute the cosine similarity in Keras: Computing cosine similarity between two tensors in Keras

As for the axes/axis explanation, the documentation here does a good job: https://www.tensorflow.org/guide/tensor