The construction of scalable decision tree based on fast splitting and j-max pre pruning on large datasets

Abstract

The decision tree is one of the most important algorithms in the classification which offers a comprehensible model of data. In building a tree we may encounter a memory limitation. The present study aims to implement an incremental scalable approach based on fast splitting, and employs a pruning technique to construct the decision tree on a large dataset to reduce the complexity of the tree. The proposed algorithm constructs the decision tree without storing the entire dataset in the primary memory via employing a minimum number of parameters. Furthermore, the J-max Pre pruning method was used to reduce the complexity with acceptable results. Experimental results show that this approach can create a balance between the accuracy and complexity of the tree and overcome the difficulties of the complexity of the tree. In spite of the appropriate accuracy and time, the proposed algorithm could produce a decision tree with less complexity on a large dataset.