Multiclass Classification

Learning objectives

Understanding the relationship between class scores, probabilities, and ground truth labels helps you interpret multiclass classification errors and incorrect labels.

Comparing multiclass logits and predictions

Starting conditions

Generate 6 samples per class, 18 in total, around the provided centers (-2,0),(2,0),(0,2). Use seed=7.

Procedure

1. Check the input and label shapes for Linear(2,3) and CrossEntropy.
2. Record the 3 logits, 3 probabilities, and prediction for each of the first 6 samples.
3. Plot a scatter plot of the data and a loss curve.

Results to retain

• A 6-row output table and plots of the data and loss.

import numpy as np
import matplotlib.pyplot as plt
import torch
from torch import nn
rng=np.random.default_rng(7)
centers=np.array([[-2.,0.],[2.,0.],[0.,2.]])
X_np=np.concatenate([c+rng.normal(0,.25,(6,2)) for c in centers]);y_np=np.repeat(np.arange(3),6)
X=torch.tensor(X_np,dtype=torch.float32);y=torch.tensor(y_np,dtype=torch.long)
def fit_multi(labels=y):
    torch.manual_seed(7);m=nn.Linear(2,3);opt=torch.optim.SGD(m.parameters(),lr=.1);history=[]
    for _ in range(200):
        loss=nn.functional.cross_entropy(m(X),labels);opt.zero_grad();loss.backward();opt.step();history.append(loss.item())
    return m,history
model,history=fit_multi();logits=model(X).detach();prob=torch.softmax(logits,dim=1);pred=prob.argmax(dim=1)
print(prob[:3],pred[:3])

Record results and self-check

Samples:

Ground truth:

3 logits:

3 probabilities:

Prediction:

Points to check

• Do probabilities sum to 1 for each sample, and do logits and probabilities have the same argmax?
• Did you use integer indices for class labels?

Results by class and ambiguous inputs

Starting conditions

Pass query=[(-2,0),(2,0),(0,2),(0,0)] to the trained model.

Procedure

1. Create a 3×3 confusion matrix for the 18 training samples.
2. Record the probabilities and predictions for the 4 query points in a table.
3. Compare the probability distributions for the cluster centers and (0,0).

Results to retain

• A confusion matrix and a query table.

query=torch.tensor([[-2.,0.],[2.,0.],[0.,2.],[0.,0.]])
qprob=torch.softmax(model(query),dim=1).detach()
print(qprob)
# TODO: Complete the confusion matrix and table.

Record results and self-check

query:

P0:

P1:

P2:

Prediction:

Interpretation:

Points to check

• Does the confusion matrix sum to 18?
• Did you avoid interpreting the maximum probability as an absolute guarantee of the model's reliability?

Data with one changed label

Starting conditions

Change only sample 0's label from 0 to 1, and retrain under the same conditions.

Procedure

1. First mark the changed label and its position.
2. Compare the baseline and modified models in terms of loss, errors against original labels, and errors against modified labels.
3. Explain the training difficulty by distinguishing issues with the iteration count from issues with the data.

Results to retain

• A comparison table of the two models and a brief interpretation.

changed_y=y.clone();changed_y[0]=1
changed_model,changed_history=fit_multi(changed_y)
# TODO: Evaluate the same predictions against both sets of Labels.

Record results and self-check

Model:

Final loss:

Error count against original labels:

Error count against modified labels:

Points to check

• Did you keep the input data and initialization unchanged?
• Did you specify which labels were used as the evaluation reference?