Logistic Regression

After obtaining probabilities, use a Threshold to decide whether to prioritize reducing missed detections or false alarms.

Starting conditions

$\mathrm{logit}=\begin{bmatrix}-2&-0.5&0&0.5&2\end{bmatrix}$, $\mathrm{Label}=\begin{bmatrix}0&0&1&1&1\end{bmatrix}$.

Procedure

1. Calculate probabilities with Sigmoid and record decisions at Threshold=.5.
2. Compare TP, FP, FN, TN, and Accuracy at Threshold=.3,.5,.7.

Results to retain

• A probability/decision table and a Confusion Matrix for each Threshold.

Starting conditions

Choose one of the three Thresholds for a sensor alarm scenario that prioritizes reducing missed detections (FN).

Procedure

1. Reuse the TP, FP, FN, TN, and Accuracy table for each threshold $0.3,0.5,0.7$ from the previous activity. Use the same logits and labels.
2. Find the Threshold candidates with the fewest FNs. If candidates tie, compare their FPs, select one value, and record the FN and FP counts supporting your choice.
3. Compare the decisions from your selected value and Threshold 0.5 for each sample, and explain which samples have different alarms.

Results to retain

• The selected Threshold and the reason for choosing it based on FN and FP.

Starting conditions

Compare a model that always predicts 0 on 100 synthetic Labels, 10% of which are Class 1.

Procedure

1. Create a Label array with 10 Class 1 samples and 90 Class 0 samples, and a prediction array filled with 0.
2. Before running the code, calculate TP, FP, FN, TN, and Accuracy. Calculate the same values in code and compare them. Also record the number of missed Class 1 samples.
3. Even if accuracy is high, can you conclude that the model reduces missed detections? Explain using your recorded FN.

Results to retain

• Expected and actual Confusion counts, and interpretation of Accuracy and missed detections.