Tensor Operations

Learning objectives

A matching shape does not guarantee the intended operation. Relate axes, matrix multiplication, and broadcasting to the meaning of the output.

Batch and feature axes

Starting conditions

Use X from the first code cell. Rows are samples and columns are features.

Procedure

1. Record the shape and dtype of X, and the meanings of X[1] and X[:,2].
2. First predict the shapes of torch.cat([X,X], dim=0) and torch.cat([X,X], dim=1), and of torch.stack([X,X], dim=0) and torch.stack([X,X], dim=1).
3. Run each of the four operations. Use six rows for the two selections and four operations, recording the predicted shape, actual shape, and axis meanings. Identify which operations create a new axis.

Results to retain

• A comparison table for the two selections and four operations.

import torch
X=torch.arange(12,dtype=torch.float32).reshape(3,4)
print(X,X.shape,torch.cat([X,X],dim=0).shape)
# TODO: Record predictions and actual results for the remaining concatenation operations.

Record results and self-check

Operation:

Expected shape:

Actual shape:

Axis meanings:

Points to check

• Did you avoid describing cat and stack as simply the same concatenation?
• Did you check both values and shapes?

Elementwise multiplication and matrix multiplication

Starting conditions

Using the same X as in the preceding activity, compute X*X, X.T@X, and X@X.T.

Procedure

1. Predict the shape of each of the three operations before running the code. After execution, compare three columns: operation, predicted shape, and actual shape.
2. Calculate element (0,1) of X.T@X by hand. Record your hand calculation and the corresponding computed element side by side, and state whether they agree.
3. Set Y=X+1, compute X*Y and X.T@Y, and display both result arrays. Use two rows to record each operation and its actual shape, distinguishing the two products.

Results to retain

• Shapes for each operation, the hand-calculated and computed values of one element, and the outputs of both operations using Y.

print(X*X,X.T@X,X@X.T)
Y=X+1
# TODO: Compare the two operations using the selected elements and Y.

Record results and self-check

Operation:

Result shape:

Calculation of a selected element by hand:

Actual value:

Points to check

• Did you distinguish the rules for matrix multiplication and elementwise multiplication?
• Did you explain what the axes mean after transposing?

Finding reshape and broadcasting errors

Starting conditions

Reshape the 12 elements into (2,6),(4,3),(2,2,3), and add the vector [1,2,3,4] to X.

Procedure

1. Reshape X into the three specified shapes, and compare the results of numel() and flatten() with the original X. Record whether the element count and order are preserved.
2. Keep X at shape $(3,4)$ and add [1,2,3,4] and [1,2,3] separately. Record the output shape for the successful operation and the error message for the failed call.
3. To add 1, 2, and 3 to the respective rows, first write the shape needed for these three elements, then reshape them and perform the addition. Record the corrected input shape and resulting array.

Results to retain

• At least 1 example each of a successful operation, an error, and a correction.

for shape in [(2,6),(4,3),(2,2,3)]:print(shape,X.reshape(shape).shape)
# TODO: Record successful Broadcasts, errors, and corrections that achieve the intended operation.

Record results and self-check

Operation:

Input shape:

Output or error:

Corrected shape:

Points to check

• Did you record errors with try/except so subsequent cells can still run?
• Did you check whether a shape that runs successfully also has the intended axis meanings?