Task-Relevant Feature Replenishment for Cross-Centre Polyp Segmentation

Abstract

Colonoscopy images from different centres usually exhibit appearance variations, making the models trained on one domain unable to generalize well to another. To tackle this issue, we propose a novel Task-relevant Feature Replenishment based Network (TRFR-Net) for cross-centre polyp segmentation via retrieving task-relevant knowledge for sufficient discrimination capability with style variations alleviated. Specifically, we first design a domain-invariant feature decomposition (DIFD) module placed after each encoding block to extract domain-shared information for segmentation. Then we develop a task-relevant feature replenishment (TRFR) module to distill informative context from the residual features of each DIFD module and dynamically aggregate these task-relevant parts, providing extra information for generalized segmentation learning. To further bridge the domain gap leveraging structural similarity, we devise a Polyp-aware Adversarial Learning (PPAL) module to align prediction feature distribution, where more emphasis is imposed on the polyp-related alignment. Experimental results on three public datasets demonstrate the effectiveness of our proposed algorithm. The code is available at: https://github.com/CathyS1996/TRFRNet.