[Week 7] My First Predictive Model: Linear Regression Exercise 🤖

Welcome to Week 7!
After 4 weeks of Python basics, we will now build a Machine Learning model ourselves.

Remember the Linear Regression theory from Week 1?
It finds the optimal line (W, b) by minimizing Cost.

Today, using the scikit-learn library,
we will implement this entire process in code.

🛠️ Module 1: The Big Picture of a Machine Learning Project

In Week 1, we compared several candidate lines
to find the optimal W and b that minimize Cost.

scikit-learn handles this complex optimization process
with a single simple command, model.fit().

Standard Machine Learning Workflow:
The standard process we will follow in all future Machine Learning projects.
Remember these 5 steps.

1. Prepare Data
2. Split the Data
3. Train the Model
4. Evaluate Model
5. (Application) Predict

🚀 Module 2: Linear Regression Modeling Step by Step

We will model California housing price data.

"How does the average number of rooms affect house prices?"

Let's build a predictive model that answers this question, step by step.

Step 1: Prepare the Data

First, load the tools (libraries) and data needed for analysis.

# Import the required libraries.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

# Load the data built into scikit-learn.
housing = fetch_california_housing()

The data has been loaded.
Now specify the inputs (X) and targets (y) we will use from this dataset.

• Inputs (X): 'Average room count (AveRooms)'
• Target (y): 'Median house value (MedHouseVal)'

[Supplement] Data Shapes Required by scikit-learn Models

Before feeding data into a model, we need to understand scikit-learn's basic design principles.

1. Basic Equation (Scalar → Scalar)
The familiar equation of a line is $y = wx + b$.
One input ($x$) produces one output ($y$).

• x = 10 (10 hours of study) → y = 95 (95 points)

2. Handling Multiple Data Points (Vector → Vector)
Real data contain multiple values.
Thus, the input ($x$) and output ($y$) become Vectors (1-dimensional arrays) holding multiple values.

• X = [10, 5, 8] → y = [95, 70, 88]

3. Handling Multiple Features (Matrix → Vector)
What happens to the input if we add 'hours of sleep' alongside 'hours of study'?
The input becomes a Matrix (2-dimensional array).

• X (Input Matrix):
  • [[10, 8], (Student 1: study 10, sleep 8)
  • [ 5, 6], (Student 2: study 5, sleep 6)
  • [ 8, 9]] (Student 3: study 8, sleep 9)
• y (output vector):
  • [95, 70, 88]

In scikit-learn:
Models are designed for the most common case 3 (Matrix → Vector).
The model input X must always be a 2-dimensional array with Shape (number of data points, number of features).

• (n_samples, n_features)

This is why, even when using only one feature,
we need a 2-dimensional array with n_features equal to 1.

Following the explanation above, specify
the inputs (X) and targets (y) from the dataset in the correct shapes.

# housing.data[:, 2] -> A 1D Vector with shape (20640,)
# housing.data[:, 2:3] -> A 2D Matrix with shape (20640, 1)

# Extract the 'AveRooms' feature (Index 5) as a 2D array
X = housing.data[:, 2:3]
y = housing.target

Before feeding data into a model,
check whether our hypothesis ("Room count and price have a linear relationship") is reasonable
by inspecting the data visually.

This is called Exploratory Data Analysis (EDA),
and visualization is one of its most basic methods.

# Examine the relationship in the data using a scatter plot
plt.figure(figsize=(6, 4))
plt.scatter(X, y, alpha=0.1)

plt.title("Average number of rooms vs. price")
plt.xlim(0,20)
plt.xlabel("AveRooms")
plt.ylabel("MedHouseVal)")
plt.grid(True)
plt.show()

print("선형 관계일까??")

<Figure size 600x400 with 1 Axes>

선형 관계일까??

Step 2: Split Data

Now prepare for modeling.
As we learned in Week 1,
evaluating a model's true ability requires splitting data into 'practice problems' and a 'real exam'.
Use the train_test_split function.

# Split the data in an 8:2 ratio
# random_state=42 : Fix the random seed for reproducibility
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

print(f"연습 문제 개수: {len(X_train)}개")
print(f"실전 시험 개수: {len(X_test)}개")

연습 문제 개수: 16512개
실전 시험 개수: 4128개

Step 3: Train the Model

Data preparation is complete.
Now load scikit-learn's LinearRegression model
and train it on the 'practice problem' data.

The model.fit() function performs this training.

# 1. Create a Linear Regression model object (instance)
model = LinearRegression()

# 2. Train the model on 'practice problems' (fit)
# This code automatically finds the optimal W and b.
model.fit(X_train, y_train)

print("모델 학습이 완료되었습니다!")

모델 학습이 완료되었습니다!

When training is complete,
the model has found the optimal slope ($W$) and intercept ($b$).

You can inspect the learned results using model.coef_ and model.intercept_.

# Inspect the optimal W and b found by the trained model
W = model.coef_[0]
b = model.intercept_

print(f"학습된 기울기(W): {W:.4f}")
print(f"학습된 절편(b): {b:.4f}")
print(f"\n=> 최종 모델: y = {W:.4f} * x + {b:.4f}")

학습된 기울기(W): 0.0768
학습된 절편(b): 1.6548

=> 최종 모델: y = 0.0768 * x + 1.6548

💯 Module 3: How Good Is This Model?

• R-squared (R², coefficient of determination): A common metric for evaluating regression models.

• Meaning: Indicates what percentage of the data's variability the model explains.

  • $R^2 = 1.0$ means perfect explanation (100%)
  • $R^2 = 0.0$ means no explanation (0%)

• model.score(): Calculates R² on the 'real exam' (test set) data to evaluate generalization.

• model.predict(): Use the trained model to obtain predictions for new data.

Step 4: Evaluate Performance

The model has found an optimal formula.
We must now evaluate its usefulness with an objective score.

Evaluate the regression model using R-squared (coefficient of determination) on the 'actual exam' data.

In general, if the magnitude of the coefficient of determination is at least 0.3,
the regression model is considered reasonably well fitted.

# Evaluate the model's R-squared score on the 'real exam' data
r2_score = model.score(X_test, y_test)

print(f"모델의 R-squared 점수: {r2_score:.4f}")
print(f"\n=> 우리 모델은 방 개수만으로")
print(f"   주택 가격 변동의 약 {r2_score*100:.2f}%를 설명합니다.")

모델의 R-squared 점수: 0.0138

=> 우리 모델은 방 개수만으로
   주택 가격 변동의 약 1.38%를 설명합니다.

Now that we have checked the evaluation score,
plot the line learned by the model to see how well it explains the data.

# Visualize the final results
plt.figure(figsize=(6, 4))

# Scatter plot of the actual data
plt.scatter(X_train, y_train, alpha=0.1, label='Train Data')
plt.scatter(X_test, y_test, alpha=0.1,
            label='Test Data', color='red')

# The optimal line learned by the model
plt.plot(X, model.predict(X), color='black',
         linewidth=3, label='Learned Regression Line')
plt.xlim(0, 20)
plt.ylim(0, 5.5)
plt.title("Linear Regression Model Result")
plt.xlabel("AveRoom)")
plt.ylabel("MedHouseVal")
plt.legend()
plt.grid(True)
plt.show()

<Figure size 600x400 with 1 Axes>

Step 5: Predict

The ultimate purpose of a well-trained and validated model is prediction on new data.

Use model.predict() to predict future values.

# What if a house has an average of 5 rooms?
# Model inputs must always be 2D arrays.
new_house_rooms = np.array([[5]])
predicted_price = model.predict(new_house_rooms)

print(f"방 5개 집의 예측 가격: ${predicted_price[0] * 100000:,.2f}")
print(f"(단위 변환 전 값: {predicted_price[0]:.2f})")

방 5개 집의 예측 가격: $203,854.02
(단위 변환 전 값: 2.04)