Week 6 · Find a line by reducing prediction error

Each row of A contains an input value and the constant 1.

1. In the data cell, print A and A_prime side by side and mark the selected columns.
2. Explain the constant column's role in $\hat y=wx$ and $\hat y=wx+b$. Choose the expression corresponding to this model's bias=False.
3. Compare the Target of the sample with input $x=0$ to the values the model can produce, and state whether there is an error that training cannot eliminate.

1. Call train_linear with Learning Rates $\eta\in\{0.001,0.01,0.1\}$, each for 1000 epochs. Keep the same Seed and model initialization inside the function.
2. Plot the three Loss curves together and tabulate the final Weight and MSE for each Learning Rate. Explain the rate of Loss decrease with reference to the curves.
3. In train_linear, immediately after loss.backward(), add code that prints model.weight.grad only during the first epoch. Relate the Gradient sign to the Weight update direction and explain why zero_grad() is needed before the next epoch.

1. Print the values and Determinant of A_prime.T @ A_prime and determine whether its Inverse exists.
2. In a three-row table, record torch.linalg.inv(A_prime.T @ A_prime) @ A_prime.T @ B_prime, torch.linalg.pinv(A_prime) @ B_prime, and the Weight trained with Learning Rate 0.01 for 1000 epochs. Also record the absolute difference between the directly computed solution and the training result.
3. For $x\in\{0,1,2,3\}$, record Input, Target, prediction, and prediction−Target in one table.

Include code, actual outputs and plots, and your own interpretation. Submit through KLAS; check the semester schedule and latest announcements for the deadline.

Filename: Name_StudentID_Week6.ipynb.

When model and input dtypes differ

1. After training the model, add a Code Cell at the bottom of the notebook and run the following code.

   print("model:", model.weight.dtype)
   for debug_input in [A_prime.double(), A_prime.float()]:
       print("Input:", debug_input.shape, debug_input.dtype)
       try:
           with torch.no_grad():
               print("Output:", model(debug_input).shape)
       except (RuntimeError, ValueError) as error:
           print(type(error).__name__, str(error))

2. In a Text Cell, tabulate the dtype of each input and whether execution succeeded, and explain the cause in one sentence in terms of the model's dtype. Input values and shapes are identical in both calls.