MemoryError when trying to allocate big numpy array

91 Views Asked by At

I have a big matrix which is a QuTiP object. I am trying to run this line of code:

ops_numpy = [op.full() for op in m_ops] # convert the QuTiP Qobj to numpy arrays

But I am getting the following error:

MemoryError: Unable to allocate 16.0 TiB for an array with shape (1048576, 1048576) and
data type complex128

Here, m_ops is a list with len(m_ops) = 27 and every m_ops[i] is a quantum object of shape

In [91]: m_ops[1].shape
Out[91]: (1048576, 1048576)

Ok, I can see that I am trying to convert a QuTiP object into a numpy array but this object is so big that I have a memory issue. My question is simple: is there any way to overcome this issue? can I 'cut' the object in smaller pieces to convert it and then put the "pieces back together"?

I really have no idea. Maybe I am not doing in the optimal way but I was working with really smaller matrix until this one and I didn't foresaw this problem.


EDIT with the full code:

"""."""
import numpy as np
import tensorflow as tf
from qutip import tensor
from qutip import sigmax, sigmaz, sigmay
from qutip import coherent, coherent_dm, expect, Qobj, fidelity, hinton
from tqdm.auto import tqdm

#%load_ext autoreload
tf.keras.backend.set_floatx('float64') # Set float64 as the default

# Local paths:
local_path = "0_qst_master/cgan_tf_20qb/%s"
data_path = "0_qst_master/cgan_tf_20qb/data/%s"

# Reading projectors
projs_settings = np.loadtxt(data_path % 'measurement_settings.txt', dtype=str)

X = sigmax()
Y = sigmay()
Z = sigmaz()

m_ops = [] # measurement operators

def string_to_operator(basis):  
    mat_real = []

    for j in range(len(basis)):
        if basis[j] == 'X':
            mat_real.append(X)     
        if basis[j] =='Y':
            mat_real.append(Y)     
        if basis[j] =='-Y':
            mat_real.append(-Y)     
        if basis[j] == 'Z':
            mat_real.append(Z)   
    return mat_real

for i in range(27):
    U = string_to_operator(projs_settings[i])
    U = tensor(U)
    m_ops.append(U)

ops_numpy = [op.full() for op in m_ops] # convert the QuTiP Qobj to numpy arrays

Another EDIT:

The measurement_settings.txt is a .txt file with the following:

enter image description here

0

There are 0 best solutions below