Small training dataset convolutional neural networks for application-specific super-resolution microscopy

Abstract

Significance: Machine learning (ML) models based on deep convolutional neural networks have been used to significantly increase microscopy resolution, speed (signal-to-noise ratio), and data interpretation. The bottleneck in developing effective ML systems is often the need to acquire large datasets to train the neural network. This paper demonstrates how adding a “dense encoder-decoder” block can be used to effectively train a neural network that produces super-resolution images from conventional microscopy diffraction-limited images trained using a small dataset (15 field-of-views). Aim: ML helps to retrieve super-resolution information from a diffraction-limited image when trained with a massive training dataset. The aim of this work is to demonstrate a neural network that estimates super-resolution images from diffraction-limited images using modifications that enable training with a small dataset. Approach: We employ “Dense Encoder-Decoder" (called DenseED) blocks in existing super-resolution ML network architectures. DenseED blocks use a dense layer that concatenates features from the previous convolutional layer to the next convolutional layer. DenseED blocks in fully convolutional networks (FCNs) estimate the super-resolution images when trained with a small training dataset (15 field-of-views) of human cells from the Widefield2SIM dataset and in fluorescent-labeled fixed bovine pulmonary artery endothelial cells (BPAE samples). Results: Conventional ML models without DenseED blocks trained on small datasets fail to accurately estimate super-resolution images while models including the DenseED blocks can. The average peak signal-to-noise ratio (PSNR) and resolution improvements achieved by networks containing DenseED blocks are ≈3.2 dB and 2×, respectively. We evaluated various configurations of target image generation methods (e.g, experimentally captured target and computationally generated target) that are used to train FCNs with and without DenseED blocks and showed including DenseED blocks in simple FCNs outperforms compared to simple FCNs without DenseED blocks. Conclusions: DenseED blocks in neural networks show accurate extraction of super-resolution images even if the ML model is trained with a small training dataset of 15 field-of-views. This approach shows that microscopy applications can use DenseED blocks to train on smaller datasets that are application-specific imaging platforms and there is a promise for applying this to other imaging modalities such as MRI/X-ray, etc.