Logistic Regression
English translation
Distinguish the model that calculates probabilities from the criterion that triggers the final alarm to adjust the costs of misses and false alarms.
Before practice
01 Are logits and probabilities the same values?
Description
Probabilities after Sigmoid and the real-valued scores before it differ in their ranges and their use as loss inputs.
02 Are adjusting the threshold and retraining the same operation?
Description
A threshold changes decisions for the same probabilities; retraining changes the parameters that produce those probabilities.
1. Logits and probabilities
Starting conditions
The inputs are the provided 12×2 synthetic features; samples whose two features sum to more than 1 belong to Class 1.
Procedure
- Run the code while distinguishing the values passed to BCEWithLogitsLoss from those after applying Sigmoid.
- For each sample, record the two features, ground truth, logit, probability, and decision at .5 in a table.
- Plot the loss curve and show the classes in the 2-dimensional data.
Results to retain
- A 12-row output table and plots of the loss and data.
Items to record and points to check
- Samples
-
- x1
-
- x2
-
- Answer
-
- Logit
-
- Probability
-
- Decision
-
Points to check
- Did you avoid applying Sigmoid twice before calculating loss?
- Are probabilities within 0~1 and separate from the decision column?
2. Thresholds and error types
Starting conditions
Apply Threshold=.35,.5,.65 to the probabilities from the same trained model.
Procedure
- Calculate TP, FP, FN, and TN separately.
- Calculate Accuracy, Precision, and Recall, and define how to display cases with a denominator of 0.
- Choose a threshold for a sensor alarm scenario where you want to reduce missed detections, and explain your reasoning.
Results to retain
- A table of 3 conditions and 3 sentences explaining your choice.
Items to record and points to check
- Threshold
-
- TP
-
- FP
-
- FN
-
- TN
-
- Accuracy
-
- Precision
-
- Recall
-
Points to check
- Did you keep the model fixed without retraining while comparing thresholds?
- Does the sum of the confusion matrix equal the sample count?
3. When error costs change
Starting conditions
Assign a weight of 3 to the training loss of Class 1 samples and compare with the baseline model. Keep the inputs, initial seed, and epochs the same.
Procedure
- Set weights to 3 for Class 1 and 1 for Class 0.
- Compare decisions at .5 for the weighted model with decisions at .35 for the baseline model.
- Explain how weighting the training loss differs from changing the decision threshold.
Results to retain
- A comparison table of the two approaches and a brief interpretation.
Items to record and points to check
- Method
-
- Threshold
-
- FP
-
- FN
-
- Accuracy
-
- Observations of probability changes
-
Points to check
- Did you distinguish changes to training from changes to decisions alone?
- Did you avoid assuming that improving one metric means reducing every type of error?