Logistic Regression

Imports

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as func
import torch.optim as opt

#from google.colab import drive
#drive.mount('/content/drive')

# Fix the Seed
torch.manual_seed(1)

<torch._C.Generator at 0x7f375ac5eb70>

1 Data loading

# Enter Data manually
x_train = torch.FloatTensor([[1, 2], [2, 3], [3, 1], [4, 3], [5, 3], [6, 2]])
y_train = torch.FloatTensor([[0], [0], [0], [1], [1], [1]])

print(x_train)
print(x_train.shape)
print(y_train)
print(y_train.shape)

# Design the Model
class Logistic(nn.Module):
    def __init__(self, num_inputs, num_outputs):
        super().__init__()
        self.linear = nn.Linear(num_inputs, num_outputs)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        return self.sigmoid(self.linear(x))

# Initialize the Model (input dim, output dim)
model = Logistic(2, 1)

tensor([[1., 2.],
        [2., 3.],
        [3., 1.],
        [4., 3.],
        [5., 3.],
        [6., 2.]])
torch.Size([6, 2])
tensor([[0.],
        [0.],
        [0.],
        [1.],
        [1.],
        [1.]])
torch.Size([6, 1])

2 Hypothesis, Cost and Optimization

• Assume the following equations hold.
  • For 1 input (scalar): $$y_{hypo} = H(x) = \frac{1}{1+e^{-\alpha x}}$$
  • For multiple inputs (vector): $$y_{hypo} = H(X) = \frac{1}{1+e^{-W^TX}}$$
• For example, with 3 data points: $$y = \frac{1}{1 + e^{-\begin{bmatrix} w_1 && w_2 && w_3 \end{bmatrix} \cdot \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}}}$$
  • Find $W$ that satisfies this.
• Use binary cross entropy as the cost function. $$-[y_{train}log(y_{hypo}) + (1-y_{train})log(1-y_{hypo})]$$

# Configure the Optimizer (set learning rate = 1)
optimizer = opt.SGD(model.parameters(), lr=1)

# Repeat
for epoch in range(100):

  # Compute Cost / mse_loss(predicted value, true value)
  y_hypo = model(x_train)
  cost = func.binary_cross_entropy(y_hypo, y_train)

  # Update the model using cost
  optimizer.zero_grad()
  cost.backward()
  optimizer.step()

  # Print intermediate results every 10 iterations
  if epoch % 10 == 9:
    prediction = y_hypo >= torch.FloatTensor([0.5])
    correct_prediction = prediction.float() == y_train
    accuracy = correct_prediction.sum().item() / len(correct_prediction)
    print('Epoch {:4d}/{} Cost: {:.6f} Accuracy {:2.2f}%'.format(
            epoch, 100, cost.item(), accuracy * 100,
    ))

Epoch    9/100 Cost: 0.579565 Accuracy 50.00%
Epoch   19/100 Cost: 0.497444 Accuracy 66.67%
Epoch   29/100 Cost: 0.420540 Accuracy 66.67%
Epoch   39/100 Cost: 0.356995 Accuracy 83.33%
Epoch   49/100 Cost: 0.302327 Accuracy 83.33%
Epoch   59/100 Cost: 0.253374 Accuracy 83.33%
Epoch   69/100 Cost: 0.209217 Accuracy 100.00%
Epoch   79/100 Cost: 0.173890 Accuracy 100.00%
Epoch   89/100 Cost: 0.153032 Accuracy 100.00%
Epoch   99/100 Cost: 0.141287 Accuracy 100.00%

3 Assignment

• Logistic regression using a data file
  • Train using data_logistic_regression.csv.
    • 8 inputs, 1 output $$[x_1, ... x_8, y]$$
  • Achieve an accuracy of at least 75%.