Creating BlockMatrix of m rows n columns where m is not equal to n without using 'transpose'

Question

I am working on an algorithm which has the following things

  A = np.array([[10,  5, 1, 0, 0],
          [6, 6, 0, 1, 0],
          [4.5, 18,0, 0, 1]])

 nonbasis = np.array([0, 1])
 basis = np.array([2, 3, 4])

I am doing the following to create BlockMatrix from the above given info.

dm2 = Matrices.dense(3, 2, A[:, nonbasis].flatten().tolist())

blocks2 = sc.parallelize([((0, 0), dm2)])

mat3 = BlockMatrix(blocks2, 3, 2)

I am expecting mat3 as follows,

mat3 = DenseMatrix([[ 10. ,   5. ],
         [  6. ,   6.],
         [  4.5 ,  18. ]])

The result I get is,

mat3 = DenseMatrix([[ 10. ,   6. ],
         [  5. ,   4.5],
         [  6. ,  18. ]])

Ideally if it was 3X3 matrix or nxm where n=m then I would have used mat3 =mat3.transpose().

Here if I do that then the 2X3 matrix becomes 3X2 which is creating problems further in my algorithm. Can anyone suggest a simple solution.

Answer 1

I'd go with intermediate IndexedRowMatrix:

from pyspark.mllib.linalg.distributed import IndexedRowMatrix

IndexedRowMatrix(sc.parallelize(enumerate(A))).toBlockMatrix()