How to create CoreML MLMultiArray for a 4D data for prediction on iOS with Swift?

Question

I have a model I trained with Keras that expects 4D input, N being the number of samples, and each sample being a 3D data in the shape of (9,15,1) and they are NOT images. I converted Keras model using CoreML Converter and imported the model into xcode as seen below

I have a 4D Float array that represents my data and I am having hard time feeding it to my model to make a prediction with it.

The model expects an MLMultiArray, and from what I tried, I can neither just convert my 4D float array to MLMultiArray, nor create it from scratch.

Let's say I have "data" which is a 4D array of Float

let data = [[[[Float]]]]()
let arr = try? MLMultiArray(data)  

That gives me

Initializer 'init(_:)' requires that '[[[[Float]]]]' conform to 'FixedWidthInteger'

I found another article at https://itnext.io/train-your-own-ml-model-using-scikit-and-use-in-ios-app-with-coreml-and-probably-with-augmented-99928a3757ad showing how to build the MLMultiArray for a 2D array, well that works fine because you know first dimension is the number of samples, so you really need to add the second dimension values as NSNumber. But that doesn't help explain what to do with 3rd or 4th dimension of the data.

I still tried using that method to set each 3D sample one by one in the MLMultiArray

var arr = try? MLMultiArray(shape: [NSNumber(value:9), NSNumber(value:15), NSNumber(value:1)], dataType: MLMultiArrayDataType.float32)

for (ind,sample) in data.enumerated() {
    let arr = try? MLMultiArray(sample)
    arr?[ind] = arr
}

When I run that, I get the following error...

Cannot assign value of type 'MLMultiArray?' to type 'NSNumber'.

So it expects an NSNumber even tho it's 4D ?? What is the proper way of creating an MLMultiArray from your 4D Float data to input into a CoreML Model?

Answer 1

If you're going to use a Float array to initialize the MLMultiArray, it needs be a 1-dimensional array. This means you need to "flatten" your 4D data into a 1D array first.

(This is why MLMultiArray has a strides property. You need to use these strides to calculate the position of each element in the flattened array.)