Fast and accurate decision trees for natural language processing tasks

Abstract

Decision trees have long been used in many machine-learning tasks; they have a clear structure that provides insight into the training data and are simple to conceptually understand and implement. We present an optimized tree-computation algorithm based on the original ID3 algorithm. We introduce a tree-pruning method that uses the development set to delete nodes from overfitted models, as well as a result-caching method for speedup. Our algorithm is 1 to 3 orders of magnitude faster than a naive implementation and yields accurate results on our test datasets.