Deeplab new dataset not finding new class

230 Views Asked by At

Hey I am attempting to add a new class to the pascal dataset to make it 21 detectable classes. I have followed these steps:

  1. Segment Images. I picked chickens and made 4 separate chicken segementations. Here's an example:

enter image description here enter image description here enter image description here

  1. place corresponding images in Voc2012 SegementationClass, SegmentationClassRaw, SegmentationObject, and JPEGImages

  2. Add Annotation XML

  3. Add new image to all txt files in "VOC2012\ImageSets\Main"

  4. Add 3 new txt files in "VOC2012\ImageSets\Main": chicken_train.txt, chicken_trainval.txt, and chicken_val.txt. Add a 1 for the file name with chickens in it. Add 2 chicken files to VOC2012\ImageSets\Segmentation\train.txt and 2 others to trainval + val.

  5. Create a new dataset in "data_generator.py":

    _EXTENDED_SEG_INFORMATION = DatasetDescriptor( splits_to_sizes={ 'train': 1466, 'train_aug': 10583, 'trainval': 2915, 'val': 1451, }, num_classes=22, ignore_label=255, )

  6. create a new colormap for "extended" dataset:

    def create_extended_label_colormap(): return np.asarray([ [0, 0, 0], #background [128, 0, 0], #aeroplane [0, 128, 0], #bike [128, 128, 0], #bird [0, 0, 128], #boat [128, 0, 128], #bottle [0, 128, 128], #bus [128, 128, 128], #car [64, 0, 0], #cat [192, 0, 0], #chair [64, 128, 0], #cow [64, 0, 128], #diningtable [64, 0, 128], #dog [192, 0, 128], #horse [64, 128, 128], #motorbike [192, 128, 128], #person [0, 64, 0], #pottedplant [128, 64, 0], #sheep [0, 192, 0], #sofa [128, 192, 0], #train [0, 64, 128], #tv [128, 64, 128], #chicken ])

  7. Generate TFRecord with command:

    py "research\deeplab\datasets\build_voc2012_data.py" --image_folder="D:\DeepLab\models-master\research\deeplab\datasets\pascal_voc_seg\VOCdevkit\VOC2012\JPEGImages" --semantic_segmentation_folder="research\deeplab\datasets\pascal_voc_seg\VOCdevkit\VOC2012\SegmentationClassRaw" --list_folder="research\deeplab\datasets\pascal_voc_seg\VOCdevkit\VOC2012\ImageSets\Segmentation" --image_format="jpg" --output_dir="research\deeplab\datasets\extended\tfrecord"

  8. Artificially, set weights towards new chicken class to get it to show up. Train with command:

    python "D:\DeepLab\models-master\research\deeplab\train.py" --logtostderr --train_split="trainval" --model_variant="xception_65" --atrous_rates=6 --atrous_rates=12 --atrous_rates=18 --output_stride=16 --decoder_output_stride=4 --train_crop_size="513,513" --train_batch_size=2 --dataset="extended" --training_number_of_steps=1000 --fine_tune_batch_norm=true --tf_initial_checkpoint="D:\DeepLab\models-master\research\deeplab\datasets\pascal_voc_seg\init_models\deeplabv3_pascal_train_aug\model.ckpt" --train_logdir="D:\DeepLab\models-master\research\deeplab\datasets\extended\exp\train_on_trainval_set\train" --dataset_dir="D:\DeepLab\models-master\research\deeplab\datasets\extended\tfrecord" --label_weights=.001 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=.01 --label_weights=100

  9. exporting the model and testing it give me this which isn't great:

enter image description here

  1. eval command on the image also produces poor results:

    eval/miou_1.0_class_1[0.00715755112] eval/miou_1.0_class_16[0.00142123573]eval/miou_1.0_class_17[0.000168795494] eval/miou_1.0_class_12[0] eval/miou_1.0_class_14[0] eval/miou_1.0_class_0[0.126532018] eval/miou_1.0_class_5[0] eval/miou_1.0_class_13[0.0135749253] eval/miou_1.0_class_15[0.0179808438] eval/miou_1.0_class_3[0.000226198768] eval/miou_1.0_class_8[0.229160205]

    eval/miou_1.0_class_2[8.26003941e-07] eval/miou_1.0_class_7[0.000108588596] eval/miou_1.0_class_4[0] eval/miou_1.0_class_19[0] eval/miou_1.0_class_18[0.00898866542] eval/miou_1.0_class_6[4.23762758e-05] eval/miou_1.0_overall[0.0191962905] eval/miou_1.0_class_21[0] eval/miou_1.0_class_10[0.00145707454] eval/miou_1.0_class_9[4.08720407e-05] eval/miou_1.0_class_20[0.00886027142] eval/miou_1.0_class_11[0.00659790076]

I have tried training 10,000 iterations to see if that fixes it and it does populate every other class but chicken class (class_21) still stays at 0 miou. What am I doing wrong here?!?

ADDITIONAL NOTES: I do have everything working correctly if I stick to the base pascal classes I can get a pretty good model at like .81 miou overall. Issues only show up when I try to introduce the new class.

I am unsure if i need to do another to substitute out "pascal_label_map" in "research\object_detection\data" or if that is unneeded.

0

There are 0 best solutions below