I have a tensor of shape (3072,1000)
which represents the weights in my neural network. I want to:
- randomly set 60% of its elements to zero.
- After updating the weights, keep 60% of the elements equal to zero but again randomly i.e., not the same previous elements.
Note: my network is not the usual artificial neural network which uses backpropagation algorithm but it is a biophysical model of the neurons in the brain so I am using special weight updating rules. Therefore, I think the ready functions in pytorch, if any, might not be helpful.
I tried the follwoing code, it is working but it takes so long because after every time I update my weight tensor, I have to run that code to set the weight tensor again to be 60% zeros
row_indices = np.random.choice(np.size(mytensor.weight, 0),
replace=False,size=int(np.size(mytensor.weight, 0)* 0.6))
column_indices = np.random.choice(np. size(mytensor.weight, 1),
replace=False, size=int(np. size(mytensor.weight, 1) * 0.6))
for r in row_indices:
for c in column_indices:
(mytensor.weight)[r][c] = 0
You can use the
dropout
function for this: