Course materials · AI Applications

Tensor Shapes and operations

English translation

AI Applications · Week 5

This week's goal · Check data counts and axis meanings before combining, reshaping, and computing.

Run: Save a copy to your Drive in Colab, then run Code Cells from the top using ▶ or Shift+Enter. Double-click Text Cells marked ⟦Enter your response⟧ to edit them.

From tables to image batches: why axes matter

Illustrative examples · A student measurement table and an image batch illustrate axis meanings. In this exercise, inspect axes and operations using numerical Tensors you create yourself.

In Week 3, you organized measurements into rows and columns. For four students with three measurements each, create an array (4,3), defining rows as students and columns as measurements. Positions carry meaning as well as values.

What about photos? One grayscale photo assigns a brightness value to each vertical and horizontal position. Color photos also distinguish color components at each position, and multiple photos require an image index.

Data to represent Examples of items to distinguish
One grayscale photo Vertical position, horizontal position
One color photo Color component, vertical position, horizontal position
Multiple color photos Image index, color component, vertical position, horizontal position

In this exercise, arrays of numbers arranged along multiple axes are treated as Tensors. Axis order may differ by tool and model, so define and check it explicitly. PyTorch Tensors support array computations and automatic differentiation for AI training.

The same principle applies to per-student and per-item means in this exercise. Averaging across students leaves per-item results; averaging across items leaves per-student results. Thinking about what to combine and what to retain before memorizing axis numbers helps predict the output shape.

Read NumPy axes in PyTorch

A Tensor's rank is its number of axes (ndim). torch.Size([4,3]) is a shape of 4 rows and 3 columns. The axis specified with axis in NumPy is specified with dim in PyTorch. Although modeling exercises use PyTorch Tensors, the same check applies: “Which axis represents samples, and which represents features?”

1. Give meaning to the axes of numerical arrays

A Tensor is a numerical array with multiple axes. (students,features)=(4,3) means three features for four students. Even with the same numbers, misunderstanding the axes can lead to incorrect means.

Give meaning to the axes of numerical arrays

Check input dtype as well as shape. Means and training inputs commonly use float; class indices use long. Check device to see which device holds the Tensor.

2. Distinguish cat and stack

cat concatenates along an existing axis. Concatenating two (2,3) Tensors with dim=0 gives (4,3). stack adds a new axis, giving (2,2,3) with dim=0. Write your predicted shape on paper before running the code.

Distinguish cat and stack

For cat, sizes must match except along the concatenation axis. For stack, all input shapes must match. If an error occurs, check these conditions before focusing on the function name.

3. reshape does not add data

12 elements can be reshaped to (3,4), (4,3), or (2,2,3). -1 means infer one axis size from the total element count. unsqueeze adds an axis of length 1, and squeeze removes such axes.

reshape does not add data

Using view requires a compatible memory layout. After transpose, use reshape, or make memory contiguous with contiguous() before using view. Specify clone when you need data independent of the original.

4. Which axis was reduced?

For (4,3), mean(dim=0) reduces the student axis to obtain three feature means. mean(dim=1) reduces the feature axis to obtain four student means. For Matrix Multiplication, check that the inner dimensions match.

Which axis was reduced?

Broadcasting aligns sizes from the rightmost axis, requiring equal sizes or a size of 1 on each axis. Successful execution alone does not establish that the computation matches your intention.

Check your understanding

  1. What values should go into the code, and what does each row and column of the resulting array represent? If calculating physical quantities, have you also checked the units?
  2. Can you identify which parts to rerun when you change one condition?
  3. Do the results match your expectations? If they differ, can you explain why using the output values or plots?

Official PyTorch Tutorials

Practice materials