Why accuracy does not increase

Question

I use TFF 0.12.0 and I run a code of federated learning for image classification with VGG16, and here is a part of my code:

def create_compiled_keras_model():
    layer1 = tf.keras.layers.GlobalAveragePooling2D()(output)
    layer1 = tf.keras.layers.Dense(units=256)(output)
    model_output = tf.keras.layers.Dense(units=2, activation='relu')(layer1)
    model = tf.keras.Model(model.input, model_output)
    return model

def model_fn():
     keras_model = create_compiled_keras_model()
     return tff.learning.from_keras_model(keras_model, sample_batch, loss=tf.keras.losses.CategoricalCrossentropy(),metrics=[tf.keras.metrics.CategoricalAccuracy()]) 

After running, I see that accuracy does not increase, knowing that I initialize 100 rounds.

round  1, metrics=<categorical_accuracy=0.5,loss=8.059043884277344,keras_training_time_client_sum_sec=0.0>
round  2, metrics=<categorical_accuracy=0.5,loss=8.059045791625977,keras_training_time_client_sum_sec=0.0>
round  3, metrics=<categorical_accuracy=0.5,loss=8.05904769897461,keras_training_time_client_sum_sec=0.0>
round  4, metrics=<categorical_accuracy=0.5,loss=8.059043884277344,keras_training_time_client_sum_sec=0.0>
round  5, metrics=<categorical_accuracy=0.5,loss=8.059045791625977,keras_training_time_client_sum_sec=0.0>
round  6, metrics=<categorical_accuracy=0.5,loss=8.059045791625977,keras_training_time_client_sum_sec=0.0>