PyTorch Certified Associate (PTCA) free question 5 / 10 · Domain 2: Model Development
Two runs of the same Conv2d training script on a CUDA device with cuDNN give slightly different results even with identical seeds. Which setting does the Conv2d page point to for making the operation repeatable?
- Set padding_mode='zeros', the default, to keep the convolution reproducible
- Set the module to evaluation mode with model.eval() before the convolution runs
- Set groups=1 to make cuDNN fall back to a fixed algorithm
- Set torch.backends.cudnn.deterministic = True, at a possible cost in performance
Show the answer and explanation
Answer: D
Why this is correct
The Conv2d page carries a note about CUDA tensors and cuDNN: in some circumstances the operator may select a nondeterministic algorithm to increase performance. If that is undesirable, the note says you can try to make the operation deterministic, potentially at a performance cost, by setting torch.backends.cudnn.deterministic = True. The note also refers the reader to the Reproducibility documentation for more information.
Why the other options are wrong
"Set the module to evaluation mode with model.eval() before the convolution runs" … The note on the page ties the behavior to the algorithm cuDNN selects for the operator. Evaluation mode changes layers such as Dropout and BatchNorm and does not select a convolution algorithm.
"Set padding_mode='zeros', the default, to keep the convolution reproducible" … padding_mode chooses how the borders are padded, with the values 'zeros', 'reflect', 'replicate' or 'circular'. The page does not connect it to the choice of algorithm.
"Set groups=1 to make cuDNN fall back to a fixed algorithm" … groups controls which input channels connect to which output channels. The page attributes the nondeterminism to the algorithm that cuDNN may select to increase performance, not to the grouping.
Key points
Differences between runs on a GPU can come from the convolution algorithm. The cudnn.deterministic flag trades some speed for repeatable results.