Creating a new feature from existing ones using a decision tree

Question

Is it possible to create a new feature out of two, or more than two existing features using a decision tree?

If so, how, and can it produce features with good information value that can better help the model?

Answer 1

The Decision Tree itself doesn't create the third variable. You would create the third variable yourself, a task commonly referred to as feature engineering. There are numerous and perhaps infinite possibilities, for example,

x3 = x1 + x2

x3 = x1 / x2 (as long as x2 can't be zero)

x3 = x1 * exp(x2)

...

As you explore this wonderful world of feature engineering, you may find that some types of combinations work better with decision trees than others... but in general there is no correct answer; just experiment.

Just a tip to get you started - Decision Trees naturally handle collinearity quite well, because as soon as 1 node is split on x, the variables that were collinear with x are suddenly less useful within the split. So transformations that are highly correlated with x1 or x2 directly might not help very much.