The endoscopogram: A 3D model reconstructed from endoscopic video frames

Abstract

Endoscopy enables high resolution visualization of tissue texture and is a critical step in many clinical workflows,including diagnosis and radiation therapy treatment planning for cancers in the nasopharynx. However,an endoscopic video does not provide explicit 3D spatial information,making it difficult to use in tumor localization,and it is inefficient to review. We introduce a pipeline for automatically reconstructing a textured 3D surface model,which we call an endoscopogram,from multiple 2D endoscopic video frames. Our pipeline first reconstructs a partial 3D surface model for each input individual 2D frame. In the next step (which is the focus of this paper),we generate a single highquality 3D surface model using a groupwise registration approach that fuses multiple,partially overlapping,incomplete,and deformed surface models together. We generate endoscopograms from synthetic,phantom,and patient data and show that our registration approach can account for tissue deformations and reconstruction inconsistency across endoscopic video frames.