Course materials · AI Applications

Multiclass Classification and Logits

English translation

AI Applications · Week 8

This week's goal · Connect output dimensions to labels and distinguish logit, softmax, and argmax.

Run: Save a copy to your Drive in Colab, then run Code Cells from the top using ▶ or Shift+Enter. Double-click Text Cells marked ⟦Enter your response⟧ to edit them.

Extend Binary to Multiclass

In Week 7, one score determined two Classes. Now one score per Class among C Classes gives output (N,C). For example, the largest value in Logit [1,2,4] has Class Index 2. If the target is 2, the training Label is one number, serving a different role from the three Model outputs. This CSV has 16 features per sample, so the input shape is (N,16). The model uses these features to compute Class scores.

1. Why one output is insufficient

For C classes, output C scores per sample. Input (N,16) gives linear-layer output (N,C). Having one CSV label column does not imply a model output dimension of one.

Why one output is insufficient

A label is one target class index. Prepare it as a long Tensor of shape (N,) with values from 0 to C-1. First check unique values in the actual data and map indices if needed.

2. From scores to probabilities and decisions

softmax(dim=1) transforms one sample's class scores into values summing to 1. argmax(dim=1) selects the class index with the largest score. Apply them to the class axis rather than the Batch axis.

From scores to probabilities and decisions

Softmax preserves the Class with the largest score. Thus, argmax selects the same Class from Logits and probabilities. Ties are resolved by the implementation's selection rule.

3. Feed logits to Loss

CrossEntropyLoss takes logits and target indices. Do not apply softmax separately before training. Record both loss and Accuracy and inspect which samples were misclassified.

Feed logits to Loss

Even with the same Accuracy, loss can change if the score assigned to the correct class improves. Errors in output range or dtype can be found quickly by passing a few data samples through the model before training on the full data.

4. Steps for examining misclassifications

Print predictions, targets, and probabilities for the first 10 samples and find indices where prediction!=label. Choose up to three misclassifications and record which classes were confused. If there are none, state that.

Steps for examining misclassifications

Accuracy in this exercise checks training results on the provided data. If no separate evaluation data was used, label it as Accuracy on the training data.

Check your understanding

  1. What values should go into the code, and what does each row and column of the resulting array represent? If calculating physical quantities, have you also checked the units?
  2. Can you identify which parts to rerun when you change one condition?
  3. Do the results match your expectations? If they differ, can you explain why using the output values or plots?

Official PyTorch Tutorials

Practice materials