Output shape of mlmodel NNClassifier is Multiarray, VNClassificationObservation not working?

Question

Need help in deploying coreML model generated from GCP to be built, and deployed on Xcode ?

The app on my iPhone opens up and I can take a picture, but the model gets stuck at 'classifying...'

This was initially due to the input image size (I changed it to 224*224) which I was able to fix using coremltools but looks like for the output I need to have a dictionary output when the .mlmodel that I have has a multiarray(float32) output. Also, GCP coreML provided two files, a label.txt file and .mlmodel.

So, I have two questions:

1. How do I leverage the label.text file during the classification/Xcode build process ?
2. My error happens at

     { guard let results = request.results as? [VNClassificationObservation] else {
        fatalError("Model failed to load image")
     }

Can I change my mlmodel output from multiarray to dictionary with labels to suit VNClassificationObservation OR VNCoreMLFeatureValueObservation can be used in someway with multiarray output ? I tried it but app on the iphone gets stuck.

Not sure how to use the label file in Xcode. Any help is much appreciated. I have spent a day researching online.

Answer 1

You will only get VNClassificationObservation when the model is a classifier. If you're getting an MLMultiArray as output, then your model is NOT a classifier according to Core ML.

It's possible to change your model into a classifier using coremltools. You need to write a Python script that:

1. loads the mlmodel
2. assigns the layers from model._spec.neuralNetwork to model._spec.neuralNetworkClassifier
3. adds two outputs, one for the winning class label & one for the dictionary with the probabilities for all class labels
4. fill in the class labels
5. save the mlmodel