How to use mscoco stuff dataset with "counts" in binary?

Question

I would like to train my model on MSCOCO semantic segmentation (only stuff) by using the json file but the key "counts" contains binary characters. Did I miss something?

I'm using MXNet and the dataloader is directly looking for the json files. Not sure how I can use the png annotated images.

Here is a sample:

{"segmentation": {"counts": "[6c0\\>0O1O100O2N1O101N2N1O2N1O010O000000000000001N101O0O2N101O0000O2L3N2N2N2N2M3H8G\\lg8", "size": [478, 640]}, "area": 1167.0, "iscrowd": 0, "image_id": 581781, "bbox": [0.0, 203.0, 40.0, 40.0], "category_id": 123, "id": 20032797}

Answer 1

Ok I got it. Here we can find the information: https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/mask.py

It's as simple as:

from pycocotools import mask
sample = {"segmentation": {"counts": "[6c0\\>0O1O100O2N1O101N2N1O2N1O010O000000000000001N101O0O2N101O0000O2L3N2N2N2N2M3H8G\\lg8", "size": [478, 640]}, "area": 1167.0, "iscrowd": 0, "image_id": 581781, "bbox": [0.0, 203.0, 40.0, 40.0], "category_id": 123, "id": 20032797}
mask.decode(sample['segmentation'])

And about the png files, it's even easier because it can be loaded directly as label.