Supervised Learning: Decision Trees, Rule Algorithms, and Their Hybrids

Abstract

In this Chapter, we describe representative algorithms of the two key supervised inductive machine learning techniques, namely, decision trees and rule algorithms. We present both basic and more advanced versions of decision trees, such as the ID3, C4.5, ID5R, and 1RD algorithms. Note that all decision tree algorithms are based on the fundamental concept learning algorithm originally proposed by Hunt. On the other hand, most rule algorithms have their origins in the set-covering problem first tackled by Michalski. The rule algorithms are represented by the DataSqueezer algorithm and the hybrid algorithms by the CLIP4 algorithm that combines the best characteristics of decision trees and rule algorithms.