Improving text classification on small-sized, imbalanced datasets with selective few-random-shot augmentation

Abstract

Small and highly imbalanced text datasets often produce brittle classifiers because scarce minority examples provide limited coverage of class semantics, while skewed class distributions bias decision boundaries toward the majority class. We propose Selective Few-Random-Shot Augmentation (SFRSA), a generate-then-select framework for low-resource imbalanced text classification. SFRSA first over-generates minority-class candidates through few-random-shot LLM prompting, then selects a fixed-budget subset by optimizing the trade-off between relevance to minority examples and diversity among selected samples in embedding space. To operationalize this idea, we develop and compare three selection modules: centroid-guided clustering (CGC), determinantal point process (DPP) selection, and a training-aware influence-utility DPP (IU-DPP) variant. Across three benchmark datasets under training sizes of 500, 1000, and 2000 and imbalance ratios of 4:1 and 9:1, SFRSA consistently improves minority-sensitive performance over classic text augmentation methods and standard LLM-based generation baselines, with the largest gains appearing in the most data-scarce settings. A similarity–diversity analysis links the selection objective to downstream gains, showing that SFRSA preserves minority semantics while avoiding near-duplicate generations. The method is model-agnostic and can be integrated into existing NLP pipelines to improve classification in low-resource, imbalanced settings.