[Week 12] First Steps in Deep Learning: Beyond Classical Models with MLP 🚀

Welcome to Week 12!
So far, we have learned powerful classical Machine Learning models
such as Decision Trees and Random Forests.
These models learned which questions to ask (rules).

Today, we finally meet the most basic Deep Learning model:
the MLP (Multilayer Perceptron).

An MLP solves nonlinear problems by 'stacking multiple layers' of simple perceptrons,
which briefly appeared in the Week 2 history lesson as a cause of the 'AI winter'.
Implement this model using MLPClassifier from scikit-learn,
and revisit the Iris problem from Week 10 to examine its performance.

🧠 Module 1: What Is an MLP (Multilayer Perceptron)?

Think of MLP as a 'company making complex decisions'.

1. Input Layer — 'New Employees'

• Role: Receives raw external data (e.g., the 4 Iris features).\
• Like new employees who receive customers' original requests (data) and pass them to the next department.\
• Implementation: One node per feature (e.g., 4 for the Iris problem).

2. Hidden Layer(s) — 'Middle Managers'

• Role: The core of an MLP. It takes input data and extracts and combines complex features.\
• First hidden layer (junior managers): Examine the raw data passed on by new employees
  and make simple judgments, such as "Is the petal at most 1cm long?" or "Is the width at least 0.5cm?"\
• Second hidden layer (department/team managers): Combine the junior managers' simple judgments
  to make more complex, abstract judgments, such as "The petals are very short and narrow."
• Implementation: The number of these layers and nodes (neurons) per layer are Hyperparameters that we choose.
  (e.g., hidden_layer_sizes=(10, 5) -> 10 junior managers and 5 department managers)

3. Output Layer — 'Chief Executive Officer (CEO)'

• Role: Combines all information processed by the hidden layers to reach a final conclusion (prediction).\
• Analogy: After reviewing all the complex reports from department/team managers,
  make the final decision: "This is Setosa" or "This is Versicolor".
• Implementation: There is one node for each class in a classification problem (e.g., 3 for Iris).

Neurons and Nonlinear Activation Functions (ReLU)

Each MLP node (neuron) essentially does two things.

1. Linear Calculation: As in Logistic Regression, multiply inputs ($x$) by weights ($W$) and add a bias ($b$).

• $z = W_1x_1 + W_2x_2 + ... + b$

2. Nonlinear Activation: Pass the calculated $z$ value through a nonlinear function.

• This Is the Core of Deep Learning!
• Without an Activation Function (i.e., with only linear operations), stacking many linear layers
  still produces one large linear calculation.
  (e.g., 1st-degree function * 1st-degree function = 1st-degree function)
• Nonlinear functions between layers
  allow deeper networks to form more complex, curved decision boundaries.

ReLU (Rectified Linear Unit):

• The most widely used nonlinear activation function.
• $f(z) = max(0, z)$
• Meaning: If the calculated value $z$ is less than 0 (e.g., irrelevant information), ignore it by outputting 0;
  if greater than 0 (e.g., meaningful information), pass it through unchanged.\
• Analogy: "If this information helps the decision (positive), report it; otherwise (negative), ignore it!"

Learning Process: Backpropagation

How Does an MLP Learn? (Review of Week 2 History)

1. Feed data into the input layer and calculate through to the output layer (Forward Propagation).
2. Compare the output layer's prediction with the true answer to calculate the Error.
3. Propagate this error backward from the CEO to the new employees.
4. Each node (neuron) calculates its contribution to the error
   and slightly adjusts its decision criteria (weight W and bias b).
5. Repeat this process over all data hundreds or thousands of times (Epochs) to minimize the error.

🌸 Module 2: Revisiting the Iris Problem with an MLP

We will now solve the Iris classification problem from Week 10,
previously addressed with Decision Trees and Random Forests, using MLP.

[Important] Data Preprocessing for Deep Learning Models: Scaling

Decision Trees split on feature values (e.g., 5cm), making them less sensitive to scale,
while Neural Network models such as MLP are very sensitive to input data scale.

• Reason: Consider the calculation $z = W_1x_1 + W_2x_2 + b$.
  • If $x_1$ ranges over (0 ~ 1) and $x_2$ over (0 ~ 1000),
    the model will rely almost entirely on $x_2$, ignoring the influence of $x_1$.
    This makes learning the weights (W) very unstable and inefficient.
• Solution: Use StandardScaler to Standardize all features to mean 0 and variance 1,
  so that they contribute to learning on an equal scale.

# Import required libraries
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import (classification_report,
                             accuracy_score)

# 1. Load and prepare data
iris = load_iris()
X = iris.data   # Use all 4 features
y = iris.target
feature_names = iris.feature_names
target_names = iris.target_names

# 2. Split data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

# 3. Scale data (important!)
# Create a StandardScaler object
scaler = StandardScaler()

# [Important] Fit scaler only on *Train data*!
# (Information from the 'real exam' data must not leak into training)
scaler.fit(X_train)

# Use the fitted scaler to transform both train/test data
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)

print("--- 스케일링 전 Train 데이터 (일부) ---")
print(np.round(X_train[:5], 2))
print("\n--- 스케일링 후 Train 데이터 (일부) ---")
print(np.round(X_train_scaled[:5], 2))

--- 스케일링 전 Train 데이터 (일부) ---
[[5.1 2.5 3.  1.1]
 [6.2 2.2 4.5 1.5]
 [5.1 3.8 1.5 0.3]
 [6.8 3.2 5.9 2.3]
 [5.7 2.8 4.1 1.3]]

--- 스케일링 후 Train 데이터 (일부) ---
[[-0.9  -1.22 -0.44 -0.14]
 [ 0.38 -1.88  0.4   0.38]
 [-0.9   1.64 -1.29 -1.17]
 [ 1.08  0.32  1.19  1.41]
 [-0.2  -0.56  0.18  0.12]]

Creating and Training an MLP Model

Now create MLPClassifier and train it on the scaled data.
The key is to design the model structure using the hidden_layer_sizes parameter.

# 1. Create an MLP model
# hidden_layer_sizes=(10, 5)
# : A first hidden layer with 10 neurons,
# and a second hidden layer with 5 neurons
# (Company analogy: 10 junior managers, 5 department managers)
# activation='relu' : Use ReLU as the nonlinear function
# max_iter=1000 : Repeat training (Backpropagation) up to 1000 times
# random_state=42 : Fix the seed for reproducible results
# solver='adam' : Optimization algorithm (default, efficient)
model_mlp = MLPClassifier(
    hidden_layer_sizes=(10, 5),
    activation='relu',
    max_iter=1000,
    random_state=42,
    solver='adam'
)

# 2. Train the model on 'scaled practice problems' (fit)
model_mlp.fit(X_train_scaled, y_train)

print("MLP 모델 학습 완료!")

MLP 모델 학습 완료!

Performance Evaluation and Comparison

Evaluate the trained MLP using scaled actual exam data.

# Make predictions on 'scaled real exam' data
y_pred_mlp = model_mlp.predict(X_test_scaled)

# Evaluate performance
accuracy_mlp = accuracy_score(y_test, y_pred_mlp)
report_mlp = classification_report(y_test, y_pred_mlp,
                                   target_names=target_names)

print("--- MLPClassifier 성능 평가 (Test Data) ---")
print(f"정확도: {accuracy_mlp:.4f}")
print("\nClassification Report:")
print(report_mlp)

--- MLPClassifier 성능 평가 (Test Data) ---
정확도: 0.9556

Classification Report:
              precision    recall  f1-score   support

      setosa       1.00      1.00      1.00        15
  versicolor       0.93      0.93      0.93        15
   virginica       0.93      0.93      0.93        15

    accuracy                           0.96        45
   macro avg       0.96      0.96      0.96        45
weighted avg       0.96      0.96      0.96        45

✨ Wrap-up

Today, we learned about MLPs (Multilayer Perceptrons), a basic Deep Learning model.

• Core principle: Combine multiple layers of neurons with nonlinear activation functions (ReLU)
  to learn complex, smooth nonlinear decision boundaries.
• Essential preprocessing: Since MLPs are sensitive to input feature scales, StandardScaler is used
  for crucial data Scaling.
• Performance: Comparable to or better than classical models (e.g., Random Forest),
  and especially effective for more complex problems (images and language).

Next time, we will use MLP to address the more complex problem of classifying high-dimensional image data.