Decision Stump

What is Decision Stump? Decision Stump Explained

A decision stump, also known as a one-level decision tree, is a simple machine-learning model used for binary classification. It is the most basic form of a decision tree and consists of a single decision node and two leaf nodes. The decision node applies a simple rule to split the data based on a single feature, and the leaf nodes represent the predicted class labels.

Here’s how a decision stump works:

Feature Selection: The decision stump selects a single feature from the dataset as the splitting criterion.

Splitting Rule: The decision node applies a threshold or condition on the selected feature to partition the data into two subsets, typically based on a binary decision. For example, if the feature is numerical, the rule could be “If the feature value is less than or equal to a threshold, go to the left leaf; otherwise, go to the right leaf." If the feature is categorical, the rule might be “If the feature value is equal to a specific category, go to the left leaf; otherwise, go to the right leaf."

Leaf Nodes: The two leaf nodes represent the predicted class labels for the subsets of data generated by the splitting rule. Each leaf node contains the majority class label of the corresponding subset. In other words, if most of the data points in a subset belong to class A, that leaf node will predict class A, and if most of the data points belong to class B, the other leaf node will predict class B.

Prediction: To make predictions for new instances, the decision stump follows the splitting rule and assigns the corresponding class label based on which leaf node the instance ends up in.

Decision stumps are simple models that can be trained quickly and are easy to interpret. However, they are limited in their ability to capture complex relationships in the data since they can only make decisions based on a single feature. Decision stumps are often used as building blocks in ensemble methods like boosting, where multiple decision stumps are combined to create a more powerful classifier.