Multiple object detection using a custom model in Python and imageai

Question

I have trained my own model with my own dataset of images for object recognition in an image, but it doesn't seem to recognize all the objects. I only have 2 objects (images of different ways a person types a specific letter of the alphabet). For example the letter 'a' and the letter 'o' as seen below.

When I run the test code on a sample of handwritten text, to a certain extent, it does say what percentage accuracy it has, but no bounding boxes. This is the image of hand written text:

This is the output I am getting:

I am using imageai to train the custom model. I am wondering if it is possible to use this trained model to run multiple object detection on the hand written image and perhaps show the bounding boxes?

Here is how my working directory looks like, in case it provides additional help:

And here is my code for training the model (custom_detector.py):

from imageai.Prediction.Custom import ModelTraining

# Instanciating the model
model_trainer = ModelTraining()
model_trainer.setModelTypeAsResNet()
# Setting our dataset directyory
model_trainer.setDataDirectory("characters")
# training the model
model_trainer.trainModel(num_objects=2, num_experiments=100, enhance_data=True, batch_size=10)

This is my code for testing the trained model (test.py):

from imageai.Prediction.Custom import CustomImagePrediction
import os

# get the working directory
execution_path = os.getcwd()
print(execution_path)
# instanciate prediction
prediction = CustomImagePrediction()
prediction.setModelTypeAsResNet()

# Set model path
prediction.setModelPath(os.path.join(execution_path, "characters", "models", "myModel.h5"))

# Set JSON path
# This is how the JSON file looks like:
#{
#   "0" : "A",
#   "1" : "O"
#}
prediction.setJsonPath(os.path.join(execution_path, "characters", "json", "model_class.json"))

# Initialize the number of objects you have retrained
prediction.loadModel(num_objects=2)

# run prediction
predictions, probabilities = prediction.predictImage(os.path.join(execution_path, "HandTextTest.jpg"), result_count=2)

# Print each prediction
for eachPrediction, eachProbability in zip(predictions, probabilities):
    print(eachPrediction, " : ", eachProbability)

Any help or advice will be highly appreicated.

Answer 1

# run prediction
predictions, probabilities = prediction.predictImage(os.path.join(execution_path, "HandTextTest.jpg"), result_count=2)

The predictImage function is responsible for predicting bounding boxes for the corresponding image. But this function does not draw bounding boxes over the image, it just displays the probabilities for the different classes. See the predictImage code .

Whereas detectCustomObjectsFromVideo function draws bounding box on the result video frames and saves the result in a file. See the detectCustomObjectsFromVideo code.

So its not the question of what the model can do because the predict function will not support drawing of bounding boxes over the image. You can either modify the code to draw bounding box over your image or use some other package or framework that will give you image with bounding box.

Feel free to comment if you have any questions.