Carpe Diem, Seize the Samples Uncertain "at the Moment" for Adaptive Batch Selection

6Citations
Citations of this article
18Readers
Mendeley users who have this article in their library.
Get full text

Abstract

The accuracy of deep neural networks is significantly affected by how well mini-batches are constructed during the training step. In this paper, we propose a novel adaptive batch selection algorithm called Recency Bias that exploits the uncertain samples predicted inconsistently in recent iterations. The historical label predictions of each training sample are used to evaluate its predictive uncertainty within a sliding window. Then, the sampling probability for the next mini-batch is assigned to each training sample in proportion to its predictive uncertainty. By taking advantage of this design, Recency Bias not only accelerates the training step but also achieves a more accurate network. We demonstrate the superiority of Recency Bias by extensive evaluation on two independent tasks. Compared with existing batch selection methods, the results showed that Recency Bias reduced the test error by up to 20.97% in a fixed wall-clock training time. At the same time, it improved the training time by up to 59.32% to reach the same test error.

Cite

CITATION STYLE

APA

Song, H., Kim, M., Kim, S., & Lee, J. G. (2020). Carpe Diem, Seize the Samples Uncertain “at the Moment” for Adaptive Batch Selection. In International Conference on Information and Knowledge Management, Proceedings (pp. 1385–1394). Association for Computing Machinery. https://doi.org/10.1145/3340531.3411898

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free