Course materials · AI Applications

CNN layer operations and Tensor shapes

English translation

AI Applications · Week 9

This week's goal · Calculate how each layer changes channel count and spatial dimensions.

Run: Save a copy to your Drive in Colab, then run Code Cells from the top using ▶ or Shift+Enter. Double-click Text Cells marked ⟦Enter your response⟧ to edit them.

From Filter calculations to Layer roles

Convolution multiplies values at matching positions in a small input region and a Filter, then sums them. For example, input Patch [[1,2],[3,4]] and Filter [[1,0],[0,-1]] yield −3 in one output cell. Moving the same Filter creates a Feature Map; Filter values are learned. ReLU sets negative values to 0, and MaxPool retains the maximum in each region. In this exercise, examine how image width, height, and channel count change after each computation.

1. Images also have an axis order

PyTorch 2D Convolution usually takes input (N,C,H,W). (8,1,28,28) means eight grayscale images, not one image with 8 channels. Write the meaning of each number before building layers.

Images also have an axis order

Convolution applies small filters at multiple positions, sharing the same parameters. The output channel count corresponds to the number of filters to learn and is independent of the input image count.

2. Calculate Convolution and pooling sizes

For dilation=1, the output length is ⌊Lin+2p−ks⌋+1\left\lfloor\frac{L_{\mathrm{in}}+2p-k}{s}\right\rfloor+1. kernel=3, padding=1, stride=1 preserves spatial dimensions. 2×2 pooling summarizes spatial information.

Calculate Convolution and pooling sizes

The final pooling in this structure uses padding=1. Substitute into the formula to check why 7 becomes 4. Channels increase 1→32→64→128, while spatial dimensions decrease 28→14→7→4.

3. Preserve the Batch axis when flattening

flatten(start_dim=1) converts (N,128,4,4) to (N,2048). Connect this to Linear(2048,625), then to Linear(625,10). Including the Batch axis would mix samples, so take care.

Preserve the Batch axis when flattening

This week, inspect the array shapes produced as an input image passes through each Layer. Outputs from random inputs are not meaningful classifications; they verify the structure of an untrained model.

4. Inspect each layer's output using a few images

Print each layer's shape on a separate line and locate the first difference from your prediction. Check that the output is (N,10) for Batch sizes 1, 8, and 32. The model parameter count must remain the same when Batch size changes.

Inspect each layer's output using a few images

Dropout randomly sets some values to 0 in train mode. In eval mode, it stops dropping values and passes inputs through unchanged. Set the mode explicitly when checking shapes and comparing reproducible outputs.

Check your understanding

  1. What values should go into the code, and what does each row and column of the resulting array represent? If calculating physical quantities, have you also checked the units?
  2. Can you identify which parts to rerun when you change one condition?
  3. Do the results match your expectations? If they differ, can you explain why using the output values or plots?

Official PyTorch Tutorials

Practice materials