Logistic Regression

Learning objectives

Distinguish the model that calculates probabilities from the criterion that triggers the final alarm to adjust the costs of misses and false alarms.

Logits and probabilities

Starting conditions

The inputs are the provided 12×2 synthetic features; samples whose two features sum to more than 1 belong to Class 1.

Procedure

1. Run the code while distinguishing the values passed to BCEWithLogitsLoss from those after applying Sigmoid.
2. For each sample, record the two features, ground truth, logit, probability, and decision at .5 in a table.
3. Plot the loss curve and show the classes in the 2-dimensional data.

Results to retain

• A 12-row output table and plots of the loss and data.

import numpy as np
import matplotlib.pyplot as plt
import torch
from torch import nn
X=torch.tensor([[-2.,-1.],[-1.,-2.],[-1.,0.],[0.,-1.],[1.,1.],[1.,2.],[2.,1.],[2.,2.],[-2.,1.],[1.,-2.],[0.,2.],[2.,0.]])
y=(X.sum(dim=1)>1).float().reshape(-1,1)
def fit_binary(weights=None):
    torch.manual_seed(7);m=nn.Linear(2,1);opt=torch.optim.SGD(m.parameters(),lr=.1);history=[]
    for _ in range(200):
        losses=nn.functional.binary_cross_entropy_with_logits(m(X),y,reduction='none')
        loss=(losses if weights is None else losses*weights).mean()
        opt.zero_grad();loss.backward();opt.step();history.append(loss.item())
    return m,history
model,history=fit_binary();prob=torch.sigmoid(model(X)).detach().reshape(-1).numpy();truth=y.numpy().reshape(-1).astype(int)
print(prob)

Record results and self-check

Samples:

x1:

x2:

Ground truth:

Logit:

Probability:

Decision:

Points to check

• Did you avoid applying Sigmoid twice before calculating loss?
• Are probabilities within 0~1 and separate from the decision column?

Thresholds and error types

Starting conditions

Apply Threshold=.35,.5,.65 to the probabilities from the same trained model.

Procedure

1. Calculate TP, FP, FN, and TN separately.
2. Calculate Accuracy, Precision, and Recall, and define how to display cases with a denominator of 0.
3. Choose a threshold for a sensor alarm scenario where you want to reduce missed detections, and explain your reasoning.

Results to retain

• A table of 3 conditions and 3 sentences explaining your choice.

for threshold in [.35,.5,.65]:
    predicted=prob>=threshold
    print(threshold,np.sum((predicted==1)&(truth==1)),np.mean(predicted==truth))
# TODO: Compute the remaining confusion matrices and metrics.

Record results and self-check

Threshold:

TP:

FP:

FN:

TN:

Accuracy:

Precision:

Recall:

Points to check

• Did you keep the model fixed without retraining while comparing thresholds?
• Does the sum of the confusion matrix equal the sample count?

When error costs change

Starting conditions

Assign a weight of 3 to the training loss of Class 1 samples and compare with the baseline model. Keep the inputs, initial seed, and epochs the same.

Procedure

1. Set weights to 3 for Class 1 and 1 for Class 0.
2. Compare decisions at .5 for the weighted model with decisions at .35 for the baseline model.
3. Explain how weighting the training loss differs from changing the decision threshold.

Results to retain

• A comparison table of the two approaches and a brief interpretation.

weights=torch.where(y==1,3.,1.)
weighted_model,_=fit_binary(weights)
weighted_prob=torch.sigmoid(weighted_model(X)).detach().numpy().reshape(-1)
# TODO: Compare the errors and probabilities of the two approaches.

Record results and self-check

Method:

Threshold:

FP:

FN:

Accuracy:

Observations of probability changes:

Points to check

• Did you distinguish changes to training from changes to decisions alone?
• Did you avoid assuming that improving one metric means reducing every type of error?