Decision Tree and Random Forest

Learning objectives

Choose model complexity by checking whether a tree that fits the training data well also performs well on new data.

Tree depth and generalization

Starting conditions

Split the Iris data with test_size=.3, random_state=7, stratify=y. Use depths 1,3,None.

Procedure

1. Train three DecisionTreeClassifier models on the same data split.
2. Record Train Accuracy, Test Accuracy, and the number of Leaves, and read the decision path for one Test sample.

Results to retain

• A table of 3 depth conditions and an explanation of one sample's decision path.

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X,y=load_iris(return_X_y=True)
Xtr,Xte,ytr,yte=train_test_split(X,y,test_size=.3,random_state=7,stratify=y)
model=DecisionTreeClassifier(max_depth=3,random_state=7).fit(Xtr,ytr)
print(model.score(Xte,yte))

Record results and self-check

max_depth:

Number of Leaves:

Train Accuracy:

Test Accuracy:

Points to check

• Did you avoid repeatedly tuning depth on the Test data?

Comparison with Random Forest

Starting conditions

Compare with RandomForest using n_estimators=50, max_depth=3 on the same split.

Procedure

1. Use the same train/test split as before and train RandomForestClassifier(n_estimators=50,max_depth=3,random_state=7).
2. Record the train and test accuracy of the depth-3 Decision Tree and Random Forest from the previous activity in the same table.
3. Provide one shared Test sample to both models and record its true label and each prediction. Explain the difference between a single Tree's result and combining results from multiple Trees.

Results to retain

• A Tree–Forest comparison table using the same split and predictions for shared Test samples.

from sklearn.ensemble import RandomForestClassifier
forest=RandomForestClassifier(n_estimators=50,max_depth=3,random_state=7)

Record results and self-check

Model and settings:

Train Accuracy:

Test Accuracy:

Selected Test Index, true label, and prediction:

Points to check

• Did you compare both models using the same train/test data?

Checking Other Inputs

Starting conditions

Split once more using split seed=11 and check whether the model ranking holds.

Procedure

1. Keep test_size=.3 and stratify=y unchanged, and change only the split's random_state to 11.
2. Retrain the depth-3 Tree and the Forest with 50 Trees from the previous activity on the new Train set. Keep the other model settings unchanged.
3. Tabulate split seed, model name, Train Accuracy, and Test Accuracy. Explain whether the ordering of Test Accuracy remains the same for seeds 7 and 11.

Results to retain

• A table of split seeds and accuracy by model, and changes in the comparison results.

# Use the preceding activity's data and functions to check the specified new examples.

Record results and self-check

Split Seed:

Model:

Train·Test Accuracy:

Comparison results for the two models:

Points to check

• Did you change only the data split Seed while keeping the model settings unchanged?