BatchNorm1d channel size cannot match

Question

I get an error when trying to implement batchnorm1d in the model. The error saying RuntimeError: running_mean should contain 32 elements not 256, so I print out the x size after pass through fc1 which is torch.Size([60, 32, 256]). It seems that batchnorm1d will view the second column as channel size. How can I deal with this problem?

class Model(nn.Module):
    
    def __init__(self, input_dim=512) -> None:
        super().__init__()

        self.fc1 = nn.Linear(input_dim, 256)
        self.bn1 = nn.BatchNorm1d(256)
        self.relu1 = nn.ReLU()
        #self.dropout1 = nn.Dropout(0.6)

        self.fc2 = nn.Linear(256, 128)
        self.bn2 = nn.BatchNorm1d(128)
        self.relu2 = nn.ReLU()
        #self.dropout2 = nn.Dropout(0.6)
        
        self.fc3 = nn.Linear(128,32)
        self.bn3 = nn.BatchNorm1d(32)
        self.dropout3 = nn.Dropout(0.6)

        self.fc4 = nn.Linear(32, 1)
        self.sig = nn.Sigmoid()

    def forward(self, x: Tensor) -> Tensor: 
        
        x = self.relu1(self.bn1(self.fc1(x)))
        x = self.relu2(self.bn2(self.fc2(x)))
        x = self.dropout3(self.fc3(x))
        x = self.sig(self.fc4(x))
        return x

Answer 1

This is correct, the second dimension is the channel dimension. As a convention, PyTorch works in a channel-first format. Layer nn.BatchNorm1d expects an input of shape (B, C, L) or (B, C), where C is the num_features argument provided to the layer on initialization.