Annotations causing errors with Object Detection in Turi Create

Question

The guide for Object Detection in Turi Create doesn't cover setting up the data, including how to attach the "annotations" category.

What I've done is to create a separate annotations file, which looks like this:

{
  "1.jpg": {
    "type": "rectangle",
    "coordinates": {
      "height": 97,
      "width": 243,
      "x": 4224,
      "y": 1821
    },
    "label": "cw"
}

I then setup my data using load_images, and this file:

# Load images
data = tc.image_analysis.load_images('train', with_path=True)
# Open annotations file as dict
annotations = eval(open("annotations").read())
# Add annotations column to SFrame, using the annotations dict key with the same name as the file name
data["annotations"] = data["path"].apply(lambda path: bounds[os.path.split(path)[1]])

That works well, and if I print data, I get something like this:

+-------------------------------+---------------------------+
|              path             |           image           |
+-------------------------------+---------------------------+
| /Users/Andrew/Code/turi/cw... | Height: 3816 Width: 11056 |
| /Users/Andrew/Code/turi/cw... | Height: 3888 Width: 10672 |
| /Users/Andrew/Code/turi/cw... |  Height: 3656 Width: 9700 |
| /Users/Andrew/Code/turi/cw... |  Height: 3872 Width: 8280 |
+-------------------------------+---------------------------+
+-------------------------------+
|          annotations          |
+-------------------------------+
| {'type': 'rectangle', 'coo... |
| {'type': 'rectangle', 'coo... |
| {'type': 'rectangle', 'coo... |
| {'type': 'rectangle', 'coo... |
+-------------------------------+

I don't know why that's separated onto 2 lines in the console - likely just for sizing reasons.

So then I get to this line in the Object Detection guide, where it intends to visualise the annotations applied to the data:

tc.object_detector.util.draw_bounding_boxes(data["image"], data["annotations"])

When I run this, I get this error in the console:

Traceback (most recent call last):
  File "app.py", line 62, in <module>
    load_data(bounds)
  File "app.py", line 23, in load_data
    tc.object_detector.util.draw_bounding_boxes(data["image"], data["annotations"])
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/toolkits/object_detector/util/_visualization.py", line 139, in draw_bounding_boxes
    .apply(draw_single_image))
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/data_structures/sframe.py", line 2463, in apply
    dryrun = [fn(row) for row in test_sf]
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/toolkits/object_detector/util/_visualization.py", line 124, in draw_single_image
    _annotate_image(pil_img, anns, confidence_threshold=confidence_threshold)
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/toolkits/object_detector/util/_visualization.py", line 49, in _annotate_image
    for ann in reversed(anns):
TypeError: argument to reversed() must be a sequence

In addition, if I comment that out, and then go ahead and do:

model = tc.object_detector.create(data, feature="image", annotations="annotations")

I get the error:

Traceback (most recent call last):
  File "app.py", line 65, in <module>
    learn()
  File "app.py", line 37, in learn
    model = tc.object_detector.create(data, feature="image", annotations="annotations")
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/toolkits/object_detector/object_detector.py", line 170, in create
    require_annotations=True)
  File "/Users/Andrew/turi/lib/python2.7/site-packages/turicreate/toolkits/object_detector/object_detector.py", line 66, in _raise_error_if_not_detection_sframe
    raise _ToolkitError("Annotations column must contain lists")
turicreate.toolkits._main.ToolkitError: Annotations column must contain lists

Presumably I'm setting up my annotations column incorrectly to what its' expecting.

Answer 1

The annotations column is missing opening and closing brackets. If you look at the visualization output in the tutorial, you will see there is an open bracket at the beginning of each annotation (...and presumably a close bracket at the end).

Not exactly sure how to fix this up at the moment. But will try to post when I get a solution.

Answer 2

A bit late, but I recently had this issue, or at least the same error report. When you get something like:

turicreate.toolkits._main.ToolkitError: Annotations column must contain lists 

try

annotations.column_types() or data.column_types()

you will most likely see something like

 str, image, str

instead of the desired

str, image, list

and the issue will be a string malformed so that the Python parser barfs. In my case the defect was simply an extra space after the closing square bracket on each line. Fix it, re run column_types to verify list processed correctly, and move forward to the next roadblock!