NiiL

Multiclass Classification and Logits

English translation

Create your own copy to run, edit, and record results.

Self-study material

1. Check CSV inputs

The first 16 CSV columns are features, and the last is the Class Index. Labels must be integers from 0 to C−1. Scale each feature as (value−mean)/standard deviation and use the same mean and standard deviation for new data.

A. Baseline example · Read and run first

import sys, numpy as np, matplotlib.pyplot as plt
import torch
from torch import nn
import torch.nn.functional as F
torch.manual_seed(42)
torch.set_num_threads(2)
print('Python:', sys.version.split()[0], '| PyTorch:', torch.__version__)
from pathlib import Path
from urllib.request import urlretrieve
p=Path('data_multinomial_classification.csv')
if not p.exists():
    urlretrieve('https://nii-lab.github.io/assets/courses/ai-applications/multiclass/data_multinomial_classification.csv',p)
a=np.loadtxt(p,delimiter=',',dtype=np.float32)
x_raw=torch.tensor(a[:,:16]); labels=torch.tensor(a[:,-1],dtype=torch.long)
classes=torch.unique(labels);print('data:',a.shape,'classes:',classes)
mean=x_raw.mean(0);scale=x_raw.std(0).clamp_min(1e-6)
x=(x_raw-mean)/scale
y=labels
assert torch.equal(classes,torch.arange(len(classes)))
print(x.shape,y.shape,y.dtype)

B. Hands-on activities

  1. Run the data-loading cell. Print the shape, dtype, and first five rows of x_raw and x separately. Label the outputs before and after preprocessing.
  2. In the same cell, print the shape, dtype, and first five values of labels and y. Find where Labels are converted to the form required by Loss.
  3. Explain why you must reuse the training data's mean and scale to predict new samples, relating this to the scale of each Feature fed to the model.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Use the example above to write code; answer explanation-only items in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

2. Run baseline training

The number of Linear outputs is the Class count. CrossEntropyLoss takes these Logits and target Indices. For comparisons, fix the epoch count, as in train_classifier(lr=…, epochs=1000).

A. Baseline example · Read and run first

def train_classifier(lr=0.1,epochs=2000):
    torch.manual_seed(42);m=nn.Linear(16,len(classes));opt=torch.optim.SGD(m.parameters(),lr=lr)
    criterion=nn.CrossEntropyLoss();losses=[];accuracies=[]
    for _ in range(epochs):
        m.train();logits=m(x);loss=criterion(logits,y)
        opt.zero_grad();loss.backward();opt.step()
        with torch.no_grad():
            logits=m(x);prediction=logits.argmax(1)
            losses.append(criterion(logits,y).item());accuracies.append((prediction==labels).float().mean().item())
    return m,losses,accuracies
model,losses,accuracies=train_classifier()
print('Training accuracy:',accuracies[-1])
fig,axes=plt.subplots(1,2,figsize=(10,3));axes[0].plot(losses);axes[1].plot(accuracies)
for ax,yl in zip(axes,['Training loss','Training accuracy']):ax.set(xlabel='Epoch',ylabel=yl);ax.grid()
plt.show()

B. Hands-on activities

  1. Call train_classifier with Learning Rates η∈{0.001,0.01,0.1}\eta\in\{0.001,0.01,0.1\}, each for 1000 epochs. Keep the Seed and model initialization inside the function.
  2. Tabulate the final Loss and Training Accuracy returned by each call for each Learning Rate. Compare the two returned histories at the same epoch when examining changes in Loss and Accuracy.
  3. If Loss decreases while Accuracy remains unchanged, select and mark one such interval. Explain it using the difference between the Probability of the correct Class and the final Class decision. If no such interval exists, state that.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Use the example above to write code; answer explanation-only items in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

3. Interpret the output

Probabilities express how plausible each Class is, while the final Class is the single selected answer. Print probabilities alongside the selection to inspect both.

A. Baseline example · Read and run first

with torch.no_grad():
    logits=model(x);probability=logits.softmax(1);prediction=logits.argmax(1)
print('first predictions:',prediction[:10]);print('labels:',labels[:10]);print(probability[:10])
wrong=(prediction!=labels).nonzero().flatten()[:3]
print('wrong indices:',wrong)
for i in wrong:print(i.item(),prediction[i].item(),labels[i].item(),probability[i])
assert torch.allclose(probability.sum(1),torch.ones(len(x)),atol=1e-5)

B. Hands-on activities

  1. Find the prediction-results code cell in the practice notebook that contains logits=model(x). Keep the model trained in the preceding activity.
  2. Record sample indices 0–9 as 10 rows, with four columns: logits, probabilities, predicted label, and target label. Read the values from logits[:10], probability[:10], prediction[:10], and labels[:10], respectively.
  3. In a separate cell, print the return value of the torch.allclose call inside the provided assert statement; record the probability-sum check across all samples as true or false. Then compare logits.argmax(1) with probability.argmax(1) element by element and record whether they agree.
  4. Select up to 3 misclassified samples and record each sample index, target label, and predicted label. If there are none, record ‘none’.

Your prediction may describe a shape or an increasing/decreasing trend rather than an exact number. Use the example above to write code; answer explanation-only items in the Text Cell below.

# ✍ Write your own: add your code below this cell.
# You may add Code Cells if the code gets long.

C. Your record · Double-click this Text Cell to edit

Practice through repetition · Reason before running

Summarize the three conditions compared in the preceding activities in this table. You do not need to repeat experiments already completed. If comparisons are insufficient, supplement them by changing just one condition.

Changed condition Predicted shape/value/result Actual result Cause of any error
⟦Enter your response 1⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧
⟦Enter your response 2⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧
⟦Enter your response 3⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ ⟦Enter your response⟧ 
# ✍ Code for repeated experiments

Optional exercise · Find the cause of an error and fix the code

When Class indices are passed as a Float Tensor

  1. After training the model, add a Code Cell at the bottom of the notebook and run the following code.

    with torch.no_grad():
    debug_logits = model(x)
debug_target = y.float()
print("original Target:", y.shape, y.dtype)
print("Float Target:", debug_target.shape, debug_target.dtype)
try:
    F.cross_entropy(debug_logits, debug_target)
except (RuntimeError, ValueError) as error:
    print(type(error).__name__, str(error))
print("integer Target Loss:", F.cross_entropy(debug_logits, y).item())

  2. Add a Text Cell below the output and record these two points.

    • The shape and dtype of both Targets, and the dtype required by the error message.
    • Why Loss cannot be computed when a Target holding one Class index per sample is supplied as a Float Tensor, even with the same indices.

    y.float() creates a new Tensor. The original y remains unchanged, so the last line computes Loss with integer Targets.

Check before saving

  1. Check the response fields, outputs, and plots, then run all cells in order in a fresh runtime.
  2. Save to your Drive, then download using .ipynb.

Include code, actual outputs and plots, and your own interpretation. Submit through KLAS; check the semester schedule and latest announcements for the deadline.

Filename: Name_StudentID_Week8.ipynb.