How can I load a pre-trained model on COCO dateset for image segmentation?

Question

I want to do semantic segmentation of objects in my video file. I prefer to use a pre-trained model on the COCO dataset (or COCO stuff dataset) and start using it for semantic segmentation and object detection on my own video files. Most of the threads I came across talk about training algorithm on COCO dataset. I am looking for a pre-trained model (a frozen graph file) that I can directly use for segmentation on my own video files.

Can anyone please guide me how to do that?

P.S. I could do object detection using the pre-trained model from tensorflow model zoo but I am unable to extend that for object segmentation.

Answer 1

I performed semantic segmentation on images downloaded from the iNaturalist.org this week with promising results. The model I ended up using was the DeepLab v3 model which is readily available in the tensorflow research folder in the repository

• https://github.com/tensorflow/models/tree/master/research/deeplab.

Please see their excellent demo for off-the-shelf segmentation:

• https://colab.research.google.com/github/tensorflow/models/blob/master/research/deeplab/deeplab_demo.ipynb.

Just go through the notebook. You can actually test it in the browser by supplying image urls, as explained in the notebook, to see if it fulfills your needs. After that, if you are familiar with tensorflow, it should be reasonably easy to manually download one of their pre-trained models, in the same way as they do in the demo, and setup something which runs on your computer.

I also found this blog post

• http://blog.qure.ai/notes/semantic-segmentation-deep-learning-review,

to be very informative on what models are available today, and what their respective pros and cons are

Have fun!