Saliency-based metric and FaceKeepOriginalAugment: a novel approach for enhancing fairness and Diversity

Abstract

Data augmentation is essential for enhancing computer vision performance, with the KeepOriginalAugment method standing out for intelligently incorporating salient and less prominent regions. Despite its success in image classification, its potential in addressing biases is unexplored. We introduce FaceKeepOriginalAugment, extending KeepOriginalAugment to address geographical, gender, and stereotypical biases in computer vision models. By balancing data diversity and information preservation, our approach enables models to leverage both salient and non-salient regions, fostering diversity and debiasing. We explore strategies for salient region placement and augmentation selection, quantifying diversity using Image Similarity Score (ISS) across datasets like FFHQ, WIKI, IMDB, LFW, and UTK Faces. We assess FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, using the Image-Image Association Score (IIAS) in CNNs and vision transformers (ViTs). Results show FaceKeepOriginalAugment effectively promotes fairness and inclusivity by reducing gender bias and enhancing fairness. Additionally, we introduce a Saliency-Based Diversity and Fairness Metric to quantify diversity and fairness while addressing data imbalance across datasets.