Enhancing Generalizability of Deep Learning Polyp Segmentation Using Online Spatial Interpolation and Hue Transformation

Abstract

Polyps, which are precursors to colon cancer, can be detected early to reduce mortality rates. However, the limited availability of public datasets and the variability of polyp shapes, textures, and colors restrict the generalizability of existing deep learning models. To overcome this challenge, researchers often employ data augmentation techniques or generative models to increase the number of training samples, regardless of the downstream learning task (i.e., polyp segmentation). In this study, we propose a deep learning framework that combines an image transformation layer with a segmentation model, where the transformed images serve as input for the segmentation model. The image transformation layer comprises a random hue shifting function and an autoencoder. The autoencoder removes textures while preserving key polyp features, transforming the input images. To control the intensity of the transformation, we employ a simple interpolation between the original and transformed images. During training, the image transformation layer generates multiple levels of texture and color variations for each input image in every epoch, effectively regularizing the segmentation model. By exposing the segmentation model to different texture and color levels within the same training batch, we encourage the model to update its weights based on intrinsic features present in both the original images and their corresponding transformed versions. This approach enhances the generalizability of deep learning models on unseen test sets. Experimental results using various configurations consistently demonstrate significant improvements in polyp Intersection over Union (IoU) ranging from 1.8% to 16.4% across different test sets.