Investigating robustness dynamics of shallow machine learning techniques: beyond basic natural variations in image classification

Abstract

Despite the widespread adoption of deep learning models, shallow machine learning (SML) algorithms are still used for image classification due to simplicity, interpretability and efficiency. This study aims to bridge this gap by investigating the robustness dynamics of SML techniques under more complex scenarios, such as adversarial perturbations and geometric transformation. Five popular classification algorithms, including k-nearest neighbour and support vector machines, were employed to build classification models. Methodology involves investigating the robustness of proposed  methods, first, on original and corrupted data by utilising benchmark datasets across several image domains. To strengthen the investigation, the models were trained using a new low-rank representation (LRR) strategy. This hybrid model simultaneously addresses two key limitations in classical LRR models: overcoming the sequential learning process and effectively capturing both local and global data structures. By introducing a dual regularisation mechanism, it integrates a k-nearest neighbour graph to preserve local consistency, while a global low-rank constraint ensures coherent data representation. Experimental results reveal significant drops in accuracy of most SML methods, especially under adversarial attacks and geometric transformations, but LRR approach mitigates these effects to a notable extent, boosting performance across data variations. The results also show that the proposed  method outperforms state-of-the-art LRR techniques in most experiments.