Logistic Regression

Learning objectives

After obtaining probabilities, use a Threshold to decide whether to prioritize reducing missed detections or false alarms.

Logits, Probabilities, and Decisions

Starting conditions

$\mathrm{logit}=\begin{bmatrix}-2&-0.5&0&0.5&2\end{bmatrix}$, $\mathrm{Label}=\begin{bmatrix}0&0&1&1&1\end{bmatrix}$.

Procedure

1. Calculate probabilities with Sigmoid and record decisions at Threshold=.5.
2. Compare TP, FP, FN, TN, and Accuracy at Threshold=.3,.5,.7.

Results to retain

• A probability/decision table and a Confusion Matrix for each Threshold.

import numpy as np
import matplotlib.pyplot as plt
logit=np.array([-2,-.5,0,.5,2]);y=np.array([0,0,1,1,1]);p=1/(1+np.exp(-logit))
print(p,p>=.5)

Record results and self-check

Threshold:

TP:

FP:

FN:

TN:

Accuracy:

Points to check

• Did you avoid treating probabilities and final labels as the same values?

Selecting Error Types

Starting conditions

Choose one of the three Thresholds for a sensor alarm scenario that prioritizes reducing missed detections (FN).

Procedure

1. Reuse the TP, FP, FN, TN, and Accuracy table for each threshold $0.3,0.5,0.7$ from the previous activity. Use the same logits and labels.
2. Find the Threshold candidates with the fewest FNs. If candidates tie, compare their FPs, select one value, and record the FN and FP counts supporting your choice.
3. Compare the decisions from your selected value and Threshold 0.5 for each sample, and explain which samples have different alarms.

Results to retain

• The selected Threshold and the reason for choosing it based on FN and FP.

thresholds=[.3,.5,.7]

Record results and self-check

Threshold:

FN:

FP:

Selected value and reasoning:

Points to check

• If candidates tied, did you record the reason for your choice?

Checking Other Inputs

Starting conditions

Compare a model that always predicts 0 on 100 synthetic Labels, 10% of which are Class 1.

Procedure

1. Create a Label array with 10 Class 1 samples and 90 Class 0 samples, and a prediction array filled with 0.
2. Before running the code, calculate TP, FP, FN, TN, and Accuracy. Calculate the same values in code and compare them. Also record the number of missed Class 1 samples.
3. Even if accuracy is high, can you conclude that the model reduces missed detections? Explain using your recorded FN.

Results to retain

• Expected and actual Confusion counts, and interpretation of Accuracy and missed detections.

# Use the preceding activity's data and functions to check the specified new examples.

Record results and self-check

TP, FP, FN, TN before running:

Actual TP, FP, FN, TN:

Accuracy:

Missed Class 1 count and interpretation:

Points to check

• Did you distinguish overall Accuracy from the number of missed Class 1 samples?