Logistic Regression · Learning

Topics covered

After obtaining probabilities, use a Threshold to decide whether to prioritize reducing missed detections or false alarms.

Score, probability, and decision

Applying sigmoid to logit z gives p=1/(1+exp(−z)). Probability and the final decision are different. Lowering the threshold yields more positive predictions but can also increase false alarms.

$$p=\frac{1}{1+e^{-z}}$$

Values passed to the loss function

Pass logits before Sigmoid to BCEWithLogitsLoss. Evaluate the training loss separately from the costs of misses and false alarms in deployment.

Worked example

$\mathrm{logit}=\begin{bmatrix}-2&-0.5&0&0.5&2\end{bmatrix}$, $\mathrm{Label}=\begin{bmatrix}0&0&1&1&1\end{bmatrix}$.

import numpy as np
import matplotlib.pyplot as plt
logit=np.array([-2,-.5,0,.5,2]);y=np.array([0,0,1,1,1]);p=1/(1+np.exp(-logit))
print(p,p>=.5)

[0.11920292 0.37754067 0.5        0.62245933 0.88079708] [False False  True  True  True]

Reference example

View more examples

Reference materials