Widening the Focus: Biomedical Image Segmentation Challenges and the Underestimated Role of Patch Sampling and Inference Strategies

Abstract

Image analysis challenges have considerably influenced the recent years in natural and biomedical computer vision. With several important architectures and training strategies having emerged from image analysis challenges, they are often interpreted as contests in model design and training, and much effort is put into optimization of these aspects. This paper is to widen the focus beyond model architecture and training pipeline design by shedding a light on inference efficiency and the underestimated role of patch sampling strategies. A notable influence of the patch overlap on the challenge scores for successful MICCAI challenges of the previous year is found, in contrast to this parameter being systematically reported in rarely any challenge paper. These edge-overlap effects are shown to be etiologically related to varying dataset-specific intra-patch accuracies. Finally, novel strategies for inference-time patch sampling – other than strided cropping and including Monte Carlo - and uncertainty-based strategies – are proposed and examined, where special focus is put on effects that overarch the single-dataset level and, amongst other effects, an improved performance in the low patch number regimen is achieved. Drawing on these findings, practical guidance is provided to the reader, and potential challenge participant, on how inference strategies can be optimized experimentally. Moreover, implications on the on-going best practice debate with respect to challenge design and reporting are discussed. In the hope it may stipulate interest in the undervalued topic of optimized sampling strategies, our inference framework and the source codes for the patch sampling strategies are made publicly available (https://github.com/IPMI-ICNS-UKE/inference-patch-sampling).