PyTorch Certified Associate (PTCA) free question 6 / 10 · Domain 2: Model Development
A convolutional network applies nn.BatchNorm2d(16) to feature maps of shape (N, 16, H, W). Over which values does the BatchNorm2d page say one mean and one variance are computed?
- The channels, height and width positions of one sample, giving a separate pair for the sample
- The batch, height and width positions of a channel, giving a separate pair per channel
- The whole batch tensor at once, giving a single pair for the layer
- The batch dimension at a fixed height and width, giving a pair per pixel position
Show the answer and explanation
Answer: B
Why this is correct
BatchNorm2d expects a 4D input, a mini-batch of 2D inputs with an additional channel dimension. The page says the mean and standard deviation are calculated per dimension over the mini-batches, and it explains the statistic layout: because the normalization is done over the C dimension, statistics are computed on (N, H, W) slices. That is why num_features is set to the number of channels, 16 here, and why the layer holds one mean and one variance for each channel, estimated from every sample and pixel position of that channel in the mini-batch. The page adds that it is common terminology to call this Spatial Batch Normalization.
Why the other options are wrong
"The channels, height and width positions of one sample, giving a separate pair for the sample" … The BatchNorm2d page says the normalization is done over the C dimension, with statistics computed on (N, H, W) slices, so the samples of the mini-batch are pooled and there is one pair per channel, not one pair per sample.
"The whole batch tensor at once, giving a single pair for the layer" … The page says the mean and standard deviation are calculated per dimension over the mini-batches, and it ties the statistics to the C dimension. One pair per channel is computed, not one pair for the layer.
"The batch dimension at a fixed height and width, giving a pair per pixel position" … The statistics are computed on (N, H, W) slices, so the height and width positions of a channel are pooled with the batch and the pair belongs to the channel, not to a pixel position.
Key points
Think of BatchNorm2d as one statistic pair per channel, pooled over batch, height and width.