Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality

Citations of this article
Mendeley users who have this article in their library.
Get full text


We present Bionic Tracking, a novel method for solving biological cell tracking problems with eye tracking in virtual reality using commodity hardware. Using gaze data, and especially smooth pursuit eye movements, we are able to track cells in time series of 3D volumetric datasets. The problem of tracking cells is ubiquitous in developmental biology, where large volumetric microscopy datasets are acquired on a daily basis, often comprising hundreds or thousands of time points that span hours or days. The image data, however, is only a means to an end, and scientists are often interested in the reconstruction of cell trajectories and cell lineage trees. Reliably tracking cells in crowded three-dimensional space over many time points remains an open problem, and many current approaches rely on tedious manual annotation or curation. In the Bionic Tracking approach, we substitute the usual 2D point-and-click interface for annotation or curation with eye tracking in a virtual reality headset, where users follow cells with their eyes in 3D space in order to track them. We detail the interaction design of our approach and explain the graph-based algorithm used to connect different time points, also taking occlusion and user distraction into account. We demonstrate Bionic Tracking using examples from two different biological datasets. Finally, we report on a user study with seven cell tracking experts, highlighting the benefits and limitations of Bionic Tracking compared to point-and-click interfaces.




Günther, U., Harrington, K. I. S., Dachselt, R., & Sbalzarini, I. F. (2020). Bionic Tracking: Using Eye Tracking to Track Biological Cells in Virtual Reality. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 12535 LNCS, pp. 280–297). Springer Science and Business Media Deutschland GmbH.

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free