[Week 11] Facing Unlabeled Data: Unsupervised Learning 🔍

Hello! Welcome to Week 11.
So far, we have covered learning with given 'answers (y)',
namely Supervised Learning (e.g., Linear/Logistic Regression and Decision Trees).

Today, we study another major branch of Machine Learning: Unsupervised Learning.
Given data without correct answers (labels),
it discovers hidden structure or patterns within the data itself.

Today, two representative Unsupervised Learning techniques,

1. K-Means Clustering: A technique for finding hidden groups in data\
2. We will study PCA, a technique for reducing data dimensionality for visualization,
   and implement it using scikit-learn.

🧺 Module 1: K-Means Clustering

What Is Clustering?

• Grouping data without ground truth labels by similarity.\
• Analogy: Given a drawer of mixed red, blue, and yellow socks,
  you can naturally divide them into three piles by similar color and shape,
  even without a 'red' answer label on each sock. This is clustering.

An Intuitive Understanding of K-Means:

• K-Means is the best-known algorithm for finding K groups (clusters).

• How It Works (Analogy: 'Choosing Group Project Leaders'):

  1. Initialize
     Randomly place K leaders (Centroids) around a classroom.
  2. Assign
     Each student (data point) joins the nearest leader's group.
  3. Update
     Each leader moves to the center (mean position) of their group.\
  • [Repeat]
    Since the leaders have moved, regroup using step (2).
  • [Converge]
    Repeat steps 2–3 until leaders stop moving (stabilize).

K-Means Exercise 1: Exploring the Principle with Synthetic Data

Apply K-Means using scikit-learn.
First, generate synthetic data for clustering practice using make_blobs.

# Import required libraries
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs # Data generator for clustering exercises
from sklearn.cluster import KMeans # K-Means model
from sklearn.decomposition import PCA # PCA
from sklearn.preprocessing import StandardScaler # Data scaler
from sklearn.datasets import load_iris # Iris dataset

# 1. Prepare data (generate unlabeled data)
# n_samples=300: Generate 300 samples
# centers=3: Generate 3 groups (centers)
# cluster_std=2.0: How spread out the groups are
# random_state=42: Fix the seed for reproducibility
# X is 2D data with shape (300, 2); y_true is the ground truth (0,1,2)
X, y_true = make_blobs(n_samples=300, centers=3,
                       cluster_std=2.0, random_state=42)

# 2. Explore data (visualization)
# For unsupervised learning, assume the ground truth (y_true) is unknown
# and display all data in gray (c='gray').
plt.figure()
plt.scatter(X[:, 0], X[:, 1], c='gray', s=50, alpha=0.7)
plt.title("Unlabeled Data")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.grid(True)
plt.show()

print("우리의 목표: 이 회색 점들을 3개의 그룹으로 분류하는 것입니다.")

<Figure size 640x480 with 1 Axes>

우리의 목표: 이 회색 점들을 3개의 그룹으로 분류하는 것입니다.

K-Means Exercise 2: Training the Model and Checking Results

Now create and train the KMeans model.
The key parameter is n_clusters (K).

# 1. Create a K-Means model
# n_clusters=3 : Set the desired number of groups (K) to 3
# random_state=42 : Fix the leaders' initial random positions
model_kmeans = KMeans(n_clusters=3, random_state=42)

# 2. Train the model (fit)
# For unsupervised learning, provide only X without ground truth (y)!
# The 'assignment' and 'update' steps repeat internally during this process.
model_kmeans.fit(X)

# 3. Inspect training results
# model.labels_ : Group number (0, 1, 2) assigned to each data point
y_pred = model_kmeans.labels_
# model.cluster_centers_ : Final positions of the 3 leaders (Centroids) after training
centroids = model_kmeans.cluster_centers_

print("K-Means 학습 완료!")
print(f"예측된 그룹 라벨 (앞 10개): {y_pred[:10]}")
print(f"최종 중심점 위치:\n{centroids}")

K-Means 학습 완료!
예측된 그룹 라벨 (앞 10개): [1 1 0 2 1 2 0 2 0 0]
최종 중심점 위치:
[[-2.75726773  9.07285344]
 [-6.88811639 -7.0878587 ]
 [ 4.85432791  2.04801886]]

Visualize how the trained model grouped the data
using model.labels_.

# Visualize K-Means clustering results
plt.figure()
# Plot the data with colors (c) set to the predicted groups (y_pred)
plt.scatter(X[:, 0], X[:, 1],
            c=y_pred, s=50, cmap='viridis', alpha=0.7)

# Overlay final centroids as red 'x' markers
plt.scatter(centroids[:, 0], centroids[:, 1],
            c='red', s=200, marker='X', label='Centroids')

plt.title("K-Means Clustering Result (K=3)")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.legend()
plt.grid(True)
plt.show()

print("정답(레이블) 없이도 데이터의 분포만으로 그룹을 성공적으로 찾아냈습니다.")

<Figure size 640x480 with 1 Axes>

정답(레이블) 없이도 데이터의 분포만으로 그룹을 성공적으로 찾아냈습니다.

🧐 Module 2: Finding the Best K (Elbow Method)

We set n_clusters=3 because we knew there were 3 true groups.
In real Unsupervised Learning, however, we do not know the appropriate K.

How Do We Choose K?

• Inertia: Measures how far each data point is from its group's Centroid
  (the sum of squared distances).
  • Smaller Inertia means points cluster more tightly around their centers.
• Properties of Inertia:
  • K=1 (1 group): All data share one center, so Inertia is maximal.
  • K=N (as many groups as data points): Each point is its own center, so Inertia is 0.
    (But this is not meaningful.)

Elbow Method:

• Increase K from 1 to 10, calculate Inertia for each K, and plot it.
• The graph shows a sharp bend (Elbow),
  where the benefit of increasing K (Inertia reduction) drops sharply.
  This can be considered a candidate for the optimal K.

# List for storing Inertia while varying K from 1 to 10
inertia_list = []
k_range = range(1, 11)

for k in k_range:
    # 1. Create and train K-Means with the current k
    model = KMeans(n_clusters=k, random_state=42)
    model.fit(X)
    # 2. Append the trained model's Inertia to the list
    # The model.inertia_ attribute stores the trained Inertia value.
    inertia_list.append(model.inertia_)
print("K값에 따른 Inertia 계산 완료!")

# Plot Inertia (y-axis) against K (x-axis) as a line plot
plt.figure()
plt.plot(k_range, inertia_list, marker='o', linestyle='--')
plt.title("Elbow Method")
plt.xlabel("K")
plt.ylabel("Inertia")
plt.grid(True)
plt.show()

print("결과 해석:")
print("- K=1에서 2로, 2에서 3으로 갈 때는 Inertia가 급격히 감소합니다.")
print("- K=3에서 4로 갈 때부터는 감소폭이 완만해집니다.")
print("=> 따라서, 이 그래프의 '팔꿈치(Elbow)' 지점은 K=3이며,")
print("   최적의 클러스터 개수는 3개라고 판단할 수 있습니다.")

K값에 따른 Inertia 계산 완료!

<Figure size 640x480 with 1 Axes>

결과 해석:
- K=1에서 2로, 2에서 3으로 갈 때는 Inertia가 급격히 감소합니다.
- K=3에서 4로 갈 때부터는 감소폭이 완만해집니다.
=> 따라서, 이 그래프의 '팔꿈치(Elbow)' 지점은 K=3이며,
   최적의 클러스터 개수는 3개라고 판단할 수 있습니다.

📉 Module 3: Dimensionality Reduction and PCA

Now we move to the second Unsupervised Learning topic: Dimensionality Reduction.

What Is Dimensionality?

• This is the number of Features in the data.\
• Iris data: 4 features (sepal length/width, petal length/width)
  -> 4-dimensional data

Why reduce dimensions?

1. Visualization: Humans can perceive up to 3 dimensions.\

• Data with 4 or 100 dimensions cannot be visualized directly.
• Compressing data to 2 or 3 dimensions lets us see its distribution.

2. Curse of Dimensionality: As dimensionality increases (too many features),
   the space containing the data grows very large while the data become sparse,
   which can reduce model performance (e.g., unnecessary noise features).

Principal Component Analysis (PCA):

• The most representative dimensionality reduction technique.
• Key idea: Find the directions in which data are most spread out (have the highest variance)
  as new 'axes (Principal Components)',
  then reduce dimensionality by projecting the original data onto these axes.
• Analogy: To represent a 3D object (e.g., a hand) in 2D, shine light onto the palm
  to find the shadow that best reveals its shape.

[Important] Data Preprocessing for PCA: Scaling

PCA works based on the data's variance.
If features (columns) have different measurement scales (e.g., height (150~190) vs. weight (40~100)),
the higher-variance 'height' feature will have an excessively large influence on PCA.

Therefore, before PCA, to treat all features as equally important,
use Standardization (Standard Scaling) to give all features mean 0 and variance 1.
This preprocessing is essential.

Use StandardScaler from scikit-learn.

# 1. Load the Iris data
iris = load_iris()
# Use all 4 features (4D)
X_iris = iris.data
y_iris = iris.target

print(f"원본 데이터 Shape: {X_iris.shape}")

# 2. Scale data using StandardScaler
# Create a StandardScaler object
scaler = StandardScaler()

# In one step, have scaler learn the mean and standard deviation (fit)
# and transform the data using those values (transform)
X_scaled = scaler.fit_transform(X_iris)

print(f"\n스케일링 후 데이터 (앞 5줄):\n{np.round(X_scaled[:5], 2)}")
print(f"\n스케일링 후 전체 평균 (≈0): {X_scaled.mean():.4f}")
print(f"스케일링 후 전체 표준편차 (≈1): {X_scaled.std():.4f}")

원본 데이터 Shape: (150, 4)

스케일링 후 데이터 (앞 5줄):
[[-0.9   1.02 -1.34 -1.32]
 [-1.14 -0.13 -1.34 -1.32]
 [-1.39  0.33 -1.4  -1.32]
 [-1.51  0.1  -1.28 -1.32]
 [-1.02  1.25 -1.34 -1.32]]

스케일링 후 전체 평균 (≈0): -0.0000
스케일링 후 전체 표준편차 (≈1): 1.0000

PCA Exercise 1: Reduce Dimensionality from 4D → 2D

Now reduce the scaled, 4-dimensional Iris data
to 2 dimensions using PCA.

# 1. Create a PCA model
# n_components=2 : Compress 4 features into 2 principal components (2D)
pca = PCA(n_components=2)

# 2. Train the PCA model (fit) and transform the data (transform)
# Take scaled 4D data (X_scaled) as input
# and return data transformed into 2D principal component space (X_pca)
X_pca = pca.fit_transform(X_scaled)

print("PCA 변환 완료!")
print(f"원본 데이터 Shape: {X_scaled.shape}")
print(f"PCA 변환 후 Shape: {X_pca.shape}")

PCA 변환 완료!
원본 데이터 Shape: (150, 4)
PCA 변환 후 Shape: (150, 2)

PCA Exercise 2: Interpreting Results

Reducing dimensions with PCA causes information loss.
Check how much information is retained.

explained_variance_ratio_:

• For each Principal Component (PC1, PC2, …) found by PCA,
  it gives the percentage of the original data's variance (information) explained.

# Inspect PCA's 'explained variance' ratios
variance_ratio = pca.explained_variance_ratio_

print(f"설명된 분산 비율 (PC1, PC2): {variance_ratio}")

# Total variance (information) explained by the two principal components
total_variance = np.sum(variance_ratio)
print(f"\n총 분산 비율: {total_variance * 100:.2f}%")

print("\n=> 4차원 데이터를 2차원으로 압축했지만,")
print(f"   약 {total_variance * 100:.2f}%를 보존하고 있습니다!")

설명된 분산 비율 (PC1, PC2): [0.72962445 0.22850762]

총 분산 비율: 95.81%

=> 4차원 데이터를 2차원으로 압축했지만,
   약 95.81%를 보존하고 있습니다!

PCA Exercise 3: 2D Visualization

Plot the 2-dimensional data transformed by PCA (X_pca) as a scatter plot.
Color the points by their original Iris species (y_iris)
to check whether the species remain distinguishable in 2-dimensional space.

# Visualize the 2D data transformed by PCA
plt.figure()

# Use column 0 of X_pca for the x-axis and column 1 for the y-axis
# Use the original ground truth (y_iris) for colors (c)
plt.scatter(X_pca[:, 0], X_pca[:, 1],
            c=y_iris, cmap='viridis', alpha=0.7)

plt.title("PCA Result")
plt.xlabel("Principal Component 1")
plt.ylabel("Principal Component 2")
plt.colorbar(label='Iris Species')
plt.grid(True)
plt.show()

print("결과 해석:")
print("- 데이터의 분포가 2차원 평면에도 꽤 잘 나타납니다.")
print("- 각 붓꽃 품종(색깔)이 2차원 공간에서도",
      "뚜렷하게 구분되는 것을 볼 수 있습니다.")
print("=> PCA가 데이터의 핵심 구조를 잘 보존하며",
      "차원을 축소했음을 의미합니다.")

<Figure size 640x480 with 2 Axes>

결과 해석:
- 데이터의 분포가 2차원 평면에도 꽤 잘 나타납니다.
- 각 붓꽃 품종(색깔)이 2차원 공간에서도 뚜렷하게 구분되는 것을 볼 수 있습니다.
=> PCA가 데이터의 핵심 구조를 잘 보존하며 차원을 축소했음을 의미합니다.

✨ Wrap-up

Today we learned to handle data without ground truth using Unsupervised Learning.

• K-Means Clustering: A technique for finding hidden groups in data.
  • Repeat 'assignment' and 'update' to find optimal Centroids.
  • The elbow method can estimate the optimal K.
• PCA (Principal Component Analysis): Reduces the dimensionality of high-dimensional data.
  • Finds new axes (Principal Components) that maximize data variance.
  • Scaling must come first,
    and explained_variance_ratio_ shows the amount of information retained.
  • Very useful for visualizing high-dimensional data.

Next time, we will work with MLP (Multilayer Perceptron), the foundation of the models learned so far
and a basic Deep Learning model.