Automatic Neural Reconstruction from Petavoxel of Electron Microscopy Data

Abstract

Connectomics is the study of the dense structure of the neurons in the brain and their synapses, providing new insights into the relation between brain's structure and its function. Recent advances in Electron Microscopy enable high-resolution imaging (4nm per pixel) of neural tissue at a rate of roughly 10 terapixels in a single day, allowing neuroscientists to capture large blocks of neural tissue in a reasonable amount of time. The large amounts of data require novel computer vision based algorithms and scalable software frameworks to process this data. We describe RhoANA [1], our dense Automatic Neural Annotation framework, which we have developed in order to automatically align, segment and reconstruct a 1mm 3 brain tissue (~2 peta-pixels). Our workflow is depicted in Fig. 1. A Zeiss MultiSEM 505 (Fig 1-a) captures the images (Fig 1-b) of a neural tissue block that is sliced into thin, 30nm thick, sections using an automatic microtome. The images of each section are stitched together and the entire volume is 3D aligned using our stitching and registration tool (Fig 1-c). Each cell membrane is detected using a membrane classification algorithm, and the cells are reconstructed into 3D objects (Fig 1-d, 1-e). Finally, we use our web-based tool to manually proofread the output, and ensure reconstruction correctness (Fig 1-f). The rest of the paper details the software tools that are used after the image acquisition by the microscope. The Zeiss MultiSEM 505 simultaneously captures 61 image tiles (multi-beam field-of-view, or mFoV) at a time in a hexagonal shape (Fig 2). Multiple mFoVs are captured to cover the targeted region in each section (Fig 2). To give immediate feedback, we have developed a web-based application that allows viewing a section after it is acquired by the microscope. Our application can also overlay additional information on each mFoV (e.g., image quality score), and perform basic image processing operations. The first stage of the pipeline performs 2D stitching and 3D registration. When collecting multiple mFoVs at the nanometer resolution, the microscope estimation of image location is imprecise. The stitching process extracts and matches features from the boundaries of each pair of adjacent tiles, and finds a per-tile rigid transformation that minimizes the distances of these features. 3D registration finds a per-section elastic (non-affine) transformation that compensates for non-linear distortions, and is based on TrakEM2's alignment tool [2]. The registration overlays a grid of points on each section that need to be matched to the adjacent section. A small image patch is taken around each of the points, and a corresponding patch is searched for in a restricted area of the adjacent section. Finally, an optimization process minimizes the deformation by simulating an elastic system of springs connected between the grid points, and outputs a per-tile transformation that can be used to render the tile. To overcome artifacts that deform parts of the sections (e.g., dirt) and make the alignment more robust, the registration process also compares sections that are 1 or 2 sections apart. 536