I'm trying to experiment with Stephan Marslands 2D Self-Organzing Map (In conjunction with Principal Components Analysis)
import som as sm
#put it into som
def som_algorithm(inputs,nIterations,x=0,y=0):
print "Running Self-Organizing Map!"
if x == 0 and y == 0:
y = len(inputs[0])
x = len(inputs)
print "x axis is " + str(x) + " / y axis is " + str(y)
som = sm.som(x,y,inputs)
som.somtrain(inputs, nIterations)
print som.somfwd(inputs)
som_algorithm(training, 100)
And based on my data set after preprocessing, produces this error
Running Self-Organizing Map!
x axis is 600 / y axis is 173
Traceback (most recent call last):
File "\source\bag_of_words.py", line 112, in <module>
som_algorithm(training, 100)
File "\source\bag_of_words.py", line 87, in som_algorithm
som = sm.som(x,y,inputs)
File "\som\__init__.py", line 44, in __init__
self.mapDist = zeros((self.x*self.y,self.x*self.y))
MemoryError
Currently my Y
represents the amount of elements per array (array dimension contained within the larger dimension) and my X
represents the larger dimension containing smaller array dimensions. What could be causing this memory error?
The code is trying to create an array of zeros with shape x*y by x*y. Thats 10774440000 elements. Each value in the array is 8 bytes (double precision). That's over 80 gigabytes. You are simply running out of memory.