TF-lite model fails to invoke with RuntimeError

Question

I am training a detection model using the keras-retinanet library. The results are promising and now I intend to convert the TensorFlow model to TensorFlow Lite for deployment.

After the conversion, the model's input details are as follows:

interpreter = tf.lite.Interpreter(model_path='path/to/the/model/model.tflite')
interpreter.allocate_tensors()
interpreter_input_details

Resulting:

[{'dtype': numpy.float32,
  'index': 0,
  'name': 'serving_default_input_1:0',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
   'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32)},
  'shape': array([  1, 800, 800,   3], dtype=int32),
  'shape_signature': array([ -1, 800, 800,   3], dtype=int32),
  'sparsity_parameters': {}}

Edit: After the conversion, I was getting an input shape of [1, 1, 1, 3], what seemed to be wrong. To overcome this issue, I had to define the model with a fixed input shape to get the [1, 800, 800, 3] as follows:

fixed_input = Input((800,800,3))
fixed_model = Model(fixed_input,model(fixed_input))
converter = tf.lite.TFLiteConverter.from_keras_model(fixed_model)
tflite_model = converter.convert()

The problem comes when I try to make a prediction. To do that, firstly I load an image and preprocess it to match the input size. Then, I pass the image to the interpreter and invoke it:

from PIL import Image

im = np.array(Image.open(filepath))
imp = preprocess_image(im) #custom function
imp = resize_image(im) #custom function
imp = imp.astype(np.float32) / 255.
imp = np.expand_dims(imp, 0)

# Prediction starts here
interpreter.set_tensor(interpreter.get_input_details()[0]['index'], imp)
interpreter.invoke()
boxes, scores, labels = interpreter.get_tensor(interpreter.get_output_details()[0]['index'])

I get an error when executing interpreter.invoke() which I couldn't fix:

RuntimeError: tensorflow/lite/kernels/gather_nd.cc:135 indices_has_only_positive_elements was not true.Node number 15 (GATHER_ND) failed to invoke.Node number 263 (WHILE) failed to invoke.

Anyone knows what is happening? Any suggestion is welcome.