Learning Decision Trees with Reinforcement Learning

Abstract

We encode the problem of learning the optimal decision tree of a given depth as an integer optimization problem. We show experi-mentally that our method (DTIP) can be used to learn good trees up to depth 5 from data sets of size up to 1000. In addition to being efficient, our new formulation allows for a lot of flexibility. Experiments show that we can use the trees learned from any existing decision tree algorithms as starting solutions and improve the trees using DTIP. Moreover, the pro-posed formulation allows us to easily create decision trees with different optimization objectives instead of accuracy and error, and constraints can be added explicitly during the tree construction phase. We show how this flexibility can be used to learn discrimination-aware classifica-tion trees, to improve learning from imbalanced data, and to learn trees that minimise false positive/negative errors.