Submodular batch selection for training deep neural networks

14Citations
Citations of this article
37Readers
Mendeley users who have this article in their library.

Abstract

Mini-batch gradient descent based methods are the de facto algorithms for training neural network architectures today. We introduce a mini-batch selection strategy based on submodular function maximization. Our novel submodular formulation captures the informativeness of each sample and diversity of the whole subset. We design an efficient, greedy algorithm which can give high-quality solutions to this NP-hard combinatorial optimization problem. Our extensive experiments on standard datasets show that the deep models trained using the proposed batch selection strategy provide better generalization than Stochastic Gradient Descent as well as a popular baseline sampling strategy across different learning rates, batch sizes, and distance metrics.

Cite

CITATION STYLE

APA

Joseph, K. J., Vamshi Teja, R., Singh, K., & Balasubramanian, V. N. (2019). Submodular batch selection for training deep neural networks. In IJCAI International Joint Conference on Artificial Intelligence (Vol. 2019-August, pp. 2677–2683). International Joint Conferences on Artificial Intelligence. https://doi.org/10.24963/ijcai.2019/372

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free