Comprehensive Analysis of Grid and Randomized Search on Dataset Performance

Abstract

This paper presents a comprehensive comparison of grid search and randomized search, the two main hyperparameter search methods used in machine learning. The paper analyses the performance of these two methods in terms of efficiency, scalability and applicability on different machine learning models and datasets. In the paper, it is emphasized that grid search provides a comprehensive search since it searches all hyperparameter combinations on a regular grid, but it creates high computational cost. On the other hand, while random search provides faster results by selecting random samples from the hyperparameter space, it has the disadvantage of not providing complete coverage. Practical suggestions and decision-making processes are also presented for which search method should be preferred in real-world applications. In conclusion, the paper summarizes the situations where grid search and random search can be advantageous according to factors such as the complexity of the model, the size of the hyperparameter space and the available computational resources and aims to provide a comprehensive guide for practitioners.