I am learning the basics of pytorch and thought to create a simple 4 layer nerual network with dropout to train IRIS dataset for classification. After refering to many tutorials I wrote this code.
import pandas as pd
from sklearn.datasets import load_iris
import torch
from torch.autograd import Variable
epochs=300
batch_size=20
lr=0.01
#loading data as numpy array
data = load_iris()
X=data.data
y=pd.get_dummies(data.target).values
#convert to tensor
X= Variable(torch.from_numpy(X), requires_grad=False)
y=Variable(torch.from_numpy(y), requires_grad=False)
print(X.size(),y.size())
#neural net model
model = torch.nn.Sequential(
torch.nn.Linear(4, 10),
torch.nn.ReLU(),
torch.nn.Dropout(),
torch.nn.Linear(10, 5),
torch.nn.ReLU(),
torch.nn.Dropout(),
torch.nn.Linear(5, 3),
torch.nn.Softmax()
)
print(model)
# Loss and Optimizer
optimizer = torch.optim.Adam(model.parameters(), lr=lr)
loss_func = torch.nn.CrossEntropyLoss()
for i in range(epochs):
# Forward pass
y_pred = model(X)
# Compute and print loss.
loss = loss_func(y_pred, y)
print(i, loss.data[0])
# Before the backward pass, use the optimizer object to zero all of the
# gradients for the variables it will update (which are the learnable weights
# of the model)
optimizer.zero_grad()
# Backward pass
loss.backward()
# Calling the step function on an Optimizer makes an update to its parameters
optimizer.step()
There are currently two problems I am facing.
- I want to set a batch size of
20
. How should I do this? - At this step
y_pred = model(X)
its showing this error
Error
TypeError: addmm_ received an invalid combination of arguments - got (int, int, torch.DoubleTensor, torch.FloatTensor), but expected one of:
* (torch.DoubleTensor mat1, torch.DoubleTensor mat2)
* (torch.SparseDoubleTensor mat1, torch.DoubleTensor mat2)
* (float beta, torch.DoubleTensor mat1, torch.DoubleTensor mat2)
* (float alpha, torch.DoubleTensor mat1, torch.DoubleTensor mat2)
* (float beta, torch.SparseDoubleTensor mat1, torch.DoubleTensor mat2)
* (float alpha, torch.SparseDoubleTensor mat1, torch.DoubleTensor mat2)
* (float beta, float alpha, torch.DoubleTensor mat1, torch.DoubleTensor mat2)
didn't match because some of the arguments have invalid types: (int, int, torch.DoubleTensor, !torch.FloatTensor!)
* (float beta, float alpha, torch.SparseDoubleTensor mat1, torch.DoubleTensor mat2)
didn't match because some of the arguments have invalid types: (int, int, !torch.DoubleTensor!, !torch.FloatTensor!)
For data processing and loading, PyTorch provide two classes, one is
Dataset
, which is used to represent your dataset. Specifically,Dataset
provide the interface to get one sample from the whole dataset using the sample index.But
Dataset
is not enough, for large dataset, we need to do batch processing. So PyTorch provide a second classDataloader
, which is used to generate batches from theDataset
given the batch size and other parameters.For your specific case, I think you should try
TensorDataset
. Then use aDataloader
to set batch size to 20. Just look through the PyTorch official examples to get a sense how to do it.The error message is pretty informative. Your input
X
to the model is typeDoubleTensor
. But your model parameters have typeFloatTensor
. In PyTorch, you can not do operation between Tensors of different types. What you should do is replace the linewith
Now,
X
has typeFloatTensor
, the error message should disappear.Also, as a gentle reminder, there are pretty much materials on the Internet about your question which can sufficiently solve your problem. You should try hard to solve it by yourself.