into_shape after remove_index fails

Question

I have an &[f64] with 309.760 elements. This dataset is an array of 7040 property sets. Each property set contains a pair of f64s and 14 triplets of f64.

I am only interested in the triplets.

I can read this dataset into an ndarray like this:

let array = Array::from_iter(data);
let mut propertysets = vector.into_shape(IxDyn(&[7040, 44])).unwrap();

and I can remove the first two f64 of each property set like this:

propertysets.remove_index(Axis(1), 0);
propertysets.remove_index(Axis(1), 0);
println!("{:?}", propertysets.shape()); // [7040, 42]

which looks promising. But now I want to reshape the array into [7040, 14, 3], which should work because 3 * 14 = 42, but:

let result = propertysets.into_shape(IxDyn(&[7040, 14, 3])).unwrap();

panics with this message:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ShapeError/IncompatibleLayout: incompatible memory layout'

The documentation of remove_index says:

the elements are not deinitialized or dropped by this, just moved out of view

which probably explains why this fails. But how to do it right? Do I have to copy propertysets somehow into a new ndarray of the correct shape? But how?

Using Array::from_iter(propertysets.iter()) results in an ndarray of &f64 instead of f64.

Answer 1

For the into_shape operation to work arrays have to be c-contiguous or fortran-contigous in memory (see docs). After

vector.into_shape(IxDyn(&[7040, 44])).unwrap();

they are contiguous. But after

propertysets.remove_index(Axis(1), 0); 

they are not. Why? The whole array is not moved with remove_index. The elements are just moved out of view (see docs).

How to solve this?

1. reassemble using from_shape_vec
2. use the new to_shape func. (not it the docs yet, but here is some info and lots of examples here)

Example:

use ndarray::{Array, Order, ViewRepr};
use ndarray::IxDyn;
use ndarray::Axis;
use ndarray::ArrayBase;
use ndarray::CowRepr;
use ndarray::Dim;
use ndarray::OwnedRepr;

fn into_shape_reassemble(data: Vec<f64>) -> Array<f64, IxDyn>
{
    let array = Array::from_iter(data);
    let mut result = array.into_shape(IxDyn(&[7040, 44])).unwrap();

    result.remove_index(Axis(1), 0);
    result.remove_index(Axis(1), 0);
    let result = Array::from_shape_vec((7040, 42), result.iter().cloned().collect()).unwrap();

    let result = result.into_shape(IxDyn(&[7040, 14, 3])).unwrap();
    println!("{:?}", result.shape());
    result
}
fn to_shape(data: Vec<f64>) -> ArrayBase<OwnedRepr<f64>, IxDyn>
{
    let array = Array::from_iter(data);
    let mut result = array.into_shape(IxDyn(&[7040, 44])).unwrap();

    result.remove_index(Axis(1), 0);
    result.remove_index(Axis(1), 0);

    let result = result.to_shape((7040, 14, 3)).unwrap().to_owned();
    println!("{:?}", result.shape());
    result.into_dyn()
}

#[cfg(test)]
mod tests {
    #[test]
    fn test_into_shape() {
        let data = vec![0.; 7040 * 44];
        super::into_shape_reassemble(data);
    }
    #[test]
    fn test_to_shape() {
        let data = vec![0.; 7040 * 44];
        super::to_shape(data);

    }

}

Output:

[7040, 14, 3]
[7040, 14, 3]