Usage of torch.optim.Adam in pytorch

678 Views Asked by At

I am new to python and pytorch. I am struggling to understand the usage of Adam optimizer. Please review the below line of code:

opt = torch.optim.Adam([y], lr=0.1)

This line is taking a input list. Print the value of y, I get : The value of [y]= [tensor([-0.7854, -0.7854, -0.7854], requires_grad=True)]

Now,when I print opt, I get:

The value of opt=  Adam (
Parameter Group 0
    amsgrad: False
    betas: (0.9, 0.999)
    capturable: False
    eps: 1e-08
    foreach: None
    lr: 0.1
    maximize: False
    weight_decay: 0
)

I am not able to understand the reason for finding opt and how it affects the code functionality?

Then further in the code, they do:

opt.zero_grad()      # What is the use this function call?
loss, expval = cost(y)      # This is a user defined function. I am understanding it
loss.backward()         # This is a user defined function. I am understanding it
opt.step()              # What is the use this function call?

What zero_grad() and step() function do and how are they dependent on opt? A small explanation can help me up a lot. Thank you for great help.

0

There are 0 best solutions below