Introduction to Machine Learning

Learning objectives

The splitting criterion changes predictions. Use a simple Threshold to examine the differences among inputs, true labels, and decisions.

Rules and Data

Starting conditions

$x=\begin{bmatrix}1&2&3&4&5&6&7&8\end{bmatrix}$, $y=\begin{bmatrix}0&0&0&0&1&1&1&1\end{bmatrix}$. x is a sensor feature, and y contains Labels for two states.

Procedure

1. Tabulate predictions from the rule x>4 and the true labels.
2. Change the Threshold to 3,4,5 and record misclassified samples and Accuracy.

Results to retain

• An 8-row prediction table and a table of 3 threshold conditions.

import numpy as np
import matplotlib.pyplot as plt
x=np.arange(1,9);y=(x>4).astype(int)
pred=(x>4).astype(int)
print(np.mean(pred==y))

Record results and self-check

Threshold:

Number of correct predictions:

Accuracy:

Misclassified x values:

Points to check

• Did you separate input features, true labels, and predictions into different columns?

Data with one changed label

Starting conditions

Preserve the original y, change one Label using y2=y.copy(); y2[3]=1, and compare Thresholds 3,4,5 again.

Procedure

1. Keep x and Thresholds $3,4,5$ unchanged, and create the modified version using y2=y.copy(); y2[3]=1. Recompute the x > threshold predictions for each Threshold.
2. For both the original and modified labels, calculate the misclassified sample indices, their count, and accuracy. Record one table row per threshold.
3. Identify the input, prediction, and true label of the changed sample, and explain which Thresholds had their classification correctness affected by that single Label change.

Results to retain

• A table of misclassifications and Accuracy by Threshold before and after the Label change.

y2=y.copy();y2[3]=1

Record results and self-check

Threshold:

Label condition:

Misclassified indices and count:

Accuracy:

Points to check

• Did you change only the labels while keeping inputs and thresholds unchanged?

Checking Other Inputs

Starting conditions

Apply the selected threshold unchanged to $x=\begin{bmatrix}0.5&2.5&4.5&6.5\end{bmatrix}$, $y=\begin{bmatrix}0&0&1&1\end{bmatrix}$.

Procedure

1. Choose one of the three thresholds above and record your choice first. Apply the same x > threshold rule to the four new inputs.
2. Record inputs, predictions, provided Labels, and matches in four rows, and calculate Accuracy. Place it beside the Accuracy for the original 8 samples and Labels from Activity 1 using the same Threshold, and explain the difference.

Results to retain

• A prediction table for four new inputs and an Accuracy comparison between the baseline and new data.

# Use the preceding activity's data and functions to check the specified new examples.

Record results and self-check

Threshold used:

New inputs and ground truth:

Predictions and matches:

Accuracy on baseline and new inputs:

Points to check

• Did you keep the selected Threshold unchanged for new inputs?