Linear Regression

Learning objectives

Examine predictions outside the observed range and the effect of an outlier on the regression equation.

Straight-Line Models and Error

Starting conditions

The data are $x=\begin{bmatrix}0&1&2&3&4&5\end{bmatrix}$, $y=\begin{bmatrix}1&3.2&4.8&7.1&8.9&11.2\end{bmatrix}$. The model is y_hat=w*x+b.

Procedure

1. Calculate predictions, residuals, and MSE for w=2,$b=1$.
2. Compare w,b obtained using np.linalg.lstsq with the baseline model in the same table.

Results to retain

• Prediction and scatter plots, and a model comparison table.

import numpy as np
import matplotlib.pyplot as plt
x=np.arange(6.);y=np.array([1,3.2,4.8,7.1,8.9,11.2]);X=np.c_[x,np.ones_like(x)]
coef=np.linalg.lstsq(X,y,rcond=None)[0]
print(coef)

Record results and self-check

Model:

w:

b:

MSE:

Prediction for $x=6$:

Points to check

• Did you keep a consistent sign convention for the residual y_hat-y?

The Influence of One Sample

Starting conditions

Create data with the last y changed to 18 and compare changes in coefficients, MSE, and predictions at $x=6$.

Procedure

1. Keep x unchanged and create a copy with only the last y changed to 18. Apply np.linalg.lstsq separately to the original and modified data.
2. Record the two models' $w,b$, MSE on their respective training data, and predictions at $x=6$ in two rows. Plot the original points, modified points, and both prediction lines on the same axes.
3. Using the table and plot, explain how changing one sample affected the coefficients and predictions for new inputs.

Results to retain

• Two fitted lines and a table comparing coefficients, MSE, and predictions for new inputs.

y2=y.copy();y2[-1]=18

Record results and self-check

Training data:

w·b:

MSE:

Prediction at x=6:

Points to check

• Did you preserve the fitting method and all samples except the one changed?

Checking Other Inputs

Starting conditions

Plot predictions at $x\in\{6,8,10\}$ and distinguish them from predictions within the observed range.

Procedure

1. Use the two models trained above to calculate predictions for $x\in\{6,8,10\}$. Record both models' predictions in the same table.
2. Draw prediction lines over a range covering the original observations and the three new inputs. Use different symbols for observed points and new predictions. For the new inputs, record predictions and whether they lie within the observed range, rather than ground truth values.
3. Explain how the difference between the two models' predictions changes farther outside the observed range.

Results to retain

• A prediction table for the three new inputs and a prediction plot that distinguishes the observed range.

# Use the preceding activity's data and functions to check the specified new examples.

Record results and self-check

New input x:

Model predictions for the original data:

Model predictions for the modified data:

Inside the observed range or not; difference between the two predictions:

Points to check

• Did you avoid interpreting predictions for new inputs without ground truth as actual errors?