Machine-learning enables image reconstruction and classification in a “see-through” camera

Abstract

We demonstrate that image reconstruction can be achieved via a convolutional neural network for a “see-through” computational camera comprised of a transparent window and CMOS image sensor. Furthermore, we compared classification results using a classifier network for the raw sensor data against those with the reconstructed images. The results suggest that similar classification accuracy is likely possible in both cases with appropriate network optimizations. All networks were trained and tested for the MNIST (6 classes), EMNIST, and the Kanji49 datasets.