A Decision Tree That Ignores Half Its Features — And Still Hits 98% Accuracy

The prompt was one sentence:

"From iris data, give me a decision tree for predicting the type of flower species."

The dataset is the classic one — Fisher's Iris measurements, 150 flowers, three species, four physical measurements each. VerbaGPT loaded it, split it 70/30 (stratified, so all three species stay balanced in both sets), trained a depth-3 decision tree, scored it, and rendered both a confusion matrix and the tree structure itself. No feature selection was requested. The model did it anyway.

The Result

97.78% accuracy on the held-out test set:

Species	Precision	Recall	F1-Score
Setosa	1.00	1.00	1.00
Versicolor	1.00	0.93	0.97
Virginica	0.94	1.00	0.97
Overall			0.98

Only one flower out of 45 in the test set was misclassified — a single versicolor predicted as virginica. But the more interesting number isn't the accuracy. It's the feature importance table underneath it:

Feature	Importance
petal_length_cm	55.09%
petal_width_cm	44.91%
sepal_width_cm	0.00%
sepal_length_cm	0.00%

Sepal length and sepal width — half of the four measurements available — contribute exactly zero to this model. Not "a little." Zero.

Why a Feature Can Be Useful and Still Score 0%

It's tempting to read "0% importance" as "this feature is useless." That's not what it means. Sepal measurements do correlate with species in this dataset — a classic scatterplot of sepal length vs. width shows visible clustering by species. But a decision tree's feature importance measures something narrower: how much a feature reduces impurity at the splits the tree actually made. Once petal length separates setosa from the rest with a single cut, and petal width separates versicolor from virginica with a second cut, there's no error left for sepal measurements to clean up. They're redundant, not irrelevant.

This is the tree's actual logic, read straight off the diagram above:

• Split 1: petal_length_cm ≤ 2.45 → perfectly isolates setosa (35 of 35 training samples, gini = 0.0).
• Split 2: petal_width_cm ≤ 1.55 → separates most of versicolor from virginica.
• Split 3: a further petal-length or petal-width cut resolves the handful of remaining borderline cases.

Three splits, two features, zero mentions of sepals — and the tree still gets 44 of 45 test flowers right.

The Broader Lesson

This generalizes past flowers. Feature importance rankings from tree-based models (decision trees, random forests, gradient boosting) tell you what mattered given the other features present — not what's correlated with your target in isolation. Two correlated features will often show one at high importance and the other near zero, because the model only needs one of them once it's used. Dropping the "0% importance" feature from a report is usually safe. Concluding it has no relationship to the outcome at all is not — swap out the feature that beat it, and the zero can flip to the dominant signal.

Data source: Fisher's Iris dataset (1936) — 150 samples, 3 species, 4 features. One of VerbaGPT's built-in sample datasources.

Ask your own question about this dataset → The ML Demo datasource is live and open — no signup required for a few questions.