Im trying to build a cross-view localization network, using a Triplet-loss.
I want the query photo (anchor), to pass through "model1", and the negative & positive to pass through "model2" (as they are from different domains, i dont want to share their weights).
I wonder how can I backprop correctly, when I compute loss from features of 2 different models.
Will this code be correct?:
criterion_triplet = nn.TripletMarginLoss(margin=args.margin, p=2, reduction="sum")
model1 = MyNet(args)
model2 = MyNet(args)
optimizer1 = torch.optim.Adam(...)
optimizer2 = torch.optim.Adam(...)
...training loops...
features1 = model1(images_anchor.to(args.device))
features2 = model2(images_satelite.to(args.device))
loss_triplet += criterion_triplet(features1,
features2,
features2)
optimizer1.zero_grad()
optimizer2.zero_grad()
loss_triplet.backward()
optimizer1.step()
optimizer2.step()