Data Normalisation in transformation then Batch Normalisation in ResNet50 pytorch

Question

I have a question regarding normalization.My current approach for training using pytorch ResNet50 on my image dataset is as follows:

First step: I calculate the mean and standard deviation of my entire dataset,then I use the following code for normalization of my images in the ImageFolder of pytorch:-

  data_transform = transforms.Compose([
    transforms.Resize(input_size),
    transforms.RandomHorizontalFlip(),
    transforms.ColorJitter(0.1,0.1,0.1),
    transforms.ToTensor(),
    transforms.Normalize([0.554, 0.450, 0.343],[0.231, 0.241, 0.241]),
  ]) 
data = datasets.ImageFolder(root=data_directory, transform=data_transform)

second step: I use ResNet50 in Pytorch for training which does batch normalization in its architecture, as shown in the below figure:-

My questions are:

Q1) Do I need both normalizations or one of these is normalization is more than enough?

Q2) what will be the reason for choosing that normalization approach??

Answer 1

tl;dr use both, they do different things

Preprocessing normalization and batchnorm are both doing normalization operations, but they serve different functions.

Preprocessing normalization is done because giving the model "nice and reasonable" numbers has better numerical stability. It generally works well to normalize inputs to have mean 0, variance 1. The normalization transform does this for your inputs with the per-channel mean and variance values.

Note that if you want to use a pretrained model, you need to use the same normalization parameters as the training data for that model.

Batchnorm is a normalization applied per layer in the model. Batchnorm tracks the mean and variance of activations and uses those values to normalize them. Batchnorm typically also has learnable parameters to shift and scale the activations (ie learning the right mean and variance).

Strictly speaking batchnorm is optional, but it tends to improve the model.