Decision Trees

Abstract

Decision tree is a fundamentally different approach towards machine learning compared to other options like neural networks or support vector machines. The other approaches deal with the data that is strictly numerical that may increase or decrease monotonically. The equations that define these approaches are designed to work only when the data is numerical. However, the theory of decision trees does not rely on the assumption of numerical data. In this chapter, we will study the theory of decision trees along with some advanced topics in decision trees, like ensemble methods. We will focus on bagging and boosting as two main types of ensemble methods and learn how they work and what their advantages and disadvantages.