Tensor Operations · Learning Materials

Topics covered

A matching shape does not guarantee the intended operation. Relate axes, matrix multiplication, and broadcasting to the meaning of the output.

Overall workflow

Tensor axes and shapes → Concatenation and matrix multiplication → reshape and broadcasting

Batch and feature axes

With three samples and four features, the shape is (3,4). cat concatenates along an existing axis, while stack creates a new axis. Track the meaning of samples and features as axes are added or removed.

A valid operation versus a correct calculation

Broadcasting expands arrays of different sizes for operations, but it can also act along unintended axes. Writing the expected shape before multiplication and checking the results with a small numerical example helps reveal silent errors.

Worked example

X=arange(12).reshape(3,4), where rows are samples and columns are features.

import torch
X=torch.arange(12,dtype=torch.float32).reshape(3,4)
print(X,X.shape,torch.cat([X,X],dim=0).shape)

tensor([[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.]]) torch.Size([3, 4]) torch.Size([6, 4])

Reference example

View more examples

Reference materials