Linear Regression · Learning

Topics covered

Examine predictions outside the observed range and the effect of an outlier on the regression equation.

Observations and Prediction Equations

Least squares reduces the sum of squared differences between predictions and observations. Outliers can have a large influence, so inspect residuals as well as the fitted line.

Evaluation on New Inputs

Predictions inside and outside the observed range have different evidential bases. In practice, split data first, then fit preprocessing and models only on training data. A Pipeline helps combine these steps in order.

Worked example

The data are $x=\begin{bmatrix}0&1&2&3&4&5\end{bmatrix}$, $y=\begin{bmatrix}1&3.2&4.8&7.1&8.9&11.2\end{bmatrix}$. The model is y_hat=w*x+b.

import numpy as np
import matplotlib.pyplot as plt
x=np.arange(6.);y=np.array([1,3.2,4.8,7.1,8.9,11.2]);X=np.c_[x,np.ones_like(x)]
coef=np.linalg.lstsq(X,y,rcond=None)[0]
print(coef)

[2.01142857 1.0047619 ]

Reference example

View more examples

Reference materials