Random Forests: The Wisdom of Crowds in Action

Abstract

Random Forests are an extremely effective ensemble learning technique that utilizes the combined decision-making process of many decision trees for improved prediction. Based on "the wisdom of crowds" principle, Random Forests face the challenge of improving upon a single classifier by incorporating ways of randomness in both the data sampling through the bootstrapping technique and in the selection of features at each decision tree. By introducing this randomness, Random Forests reduce the variation of the original classifier and improve on generalization, while still avoiding overfitting and ensuring a level of robustness. In order to perform either a classification or regression task, the Random Forests algorithm makes a prediction by averaging across an ensemble of independently trained trees, with classification making predictions by majority vote and regression by averaging. Random Forests have shown to be more accurate, scalable, and provide resiliency against noise across a number of different application domains, such as medical diagnosis, finance, bioinformatics, and text mining. In addition, Random Forests provide an easy-to-interpret measure of variable importance for practitioners to understand which features are important to the prediction. Random Forests also employ high-dimensional data well and can generally produce reliable predictions even when the number of predictors exceeds the number of observations. However, maximizing the number of trees, addressing the dilemma of interpretability and complexity, and addressing the cost of computation on very large datasets are still challenges that Random Forests present. Overall, Random Forests continues to benchmark standardized ensemble technique and the ability of a collective learning technique to improve performance in predictive problems.