Multinomial Classification

Imports

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as func
import torch.optim as opt

#from google.colab import drive
#drive.mount('/content/drive')

# Fix the Seed
torch.manual_seed(1)

<torch._C.Generator at 0x7fc3da535b90>

1 Data loading

# Enter Data manually
x_train = torch.FloatTensor([[1, 2, 1, 1], [2, 1, 3, 2], [3, 1, 3, 4], [4, 1, 5, 5],
               [1, 7, 5, 5], [1, 2, 5, 6], [1, 6, 6, 6], [1, 7, 7, 7]])
y_train = torch.LongTensor([2, 2, 2, 1, 1, 1, 0, 0])

print(x_train)
print(x_train.shape)
print(y_train)
print(y_train.shape)

# Initialize the Model (input dim, output dim)
model = nn.Linear(4, 3)

tensor([[1., 2., 1., 1.],
        [2., 1., 3., 2.],
        [3., 1., 3., 4.],
        [4., 1., 5., 5.],
        [1., 7., 5., 5.],
        [1., 2., 5., 6.],
        [1., 6., 6., 6.],
        [1., 7., 7., 7.]])
torch.Size([8, 4])
tensor([2, 2, 2, 1, 1, 1, 0, 0])
torch.Size([8])

2 Hypothesis, Cost and Optimization

• Assume the following equations hold. $$y_{hypo} = H(x) = Softmax( Linear(x) )$$ $$Linear(x) = W \cdot x + b$$ $$Softmax(X) = [y_1, ..., y_C], \;\; y_i = \frac{e^{x_i}}{\sum_{j=1}^C e^{x_j}}$$

• Use cross entropy as the cost function. $$- \sum_{i=1}^{C} y_{train, i}log(y_{hypo, i})$$

• Note that PyTorch's cross entropy loss function already includes softmax.

  • Therefore, use a linear model directly.
  • Pass class indices as y_train (one-hot encoding is unnecessary).

# Configure the Optimizer (set learning rate = 1)
optimizer = opt.SGD(model.parameters(), lr=0.1)

# Repeat
for epoch in range(1000):

  # Compute Cost / mse_loss(predicted value, true value)
  y_hypo = model(x_train)
  cost = func.cross_entropy(y_hypo, y_train)

  # Update the model using cost
  optimizer.zero_grad()
  cost.backward()
  optimizer.step()

  # Print intermediate results every 10 iterations
  if epoch % 100 == 99:
    correct_prediction = y_hypo.argmax(dim=1) == y_train
    accuracy = correct_prediction.sum().item() / len(correct_prediction)
    print('Epoch {:4d}/{} Cost: {:.6f} Accuracy {:2.2f}%'.format(
            epoch, 1000, cost.item(), accuracy * 100,
    ))

Epoch   99/1000 Cost: 0.695567 Accuracy 75.00%
Epoch  199/1000 Cost: 0.614224 Accuracy 75.00%
Epoch  299/1000 Cost: 0.556557 Accuracy 75.00%
Epoch  399/1000 Cost: 0.505776 Accuracy 87.50%
Epoch  499/1000 Cost: 0.457821 Accuracy 87.50%
Epoch  599/1000 Cost: 0.411135 Accuracy 87.50%
Epoch  699/1000 Cost: 0.364998 Accuracy 87.50%
Epoch  799/1000 Cost: 0.319251 Accuracy 87.50%
Epoch  899/1000 Cost: 0.275260 Accuracy 87.50%
Epoch  999/1000 Cost: 0.244028 Accuracy 100.00%

3 Assignment

• Multinomial classification using a data file
  • Train using data_multinomial_classification.csv.
    • 16 inputs, 1 output

$$[x_1, ... x_{16}, y]$$

• Achieve an accuracy of at least 95%.