Data management and archiving in a large microscopy-and-imaging, multi-user facility: Problems and solutions

Abstract

Advancements in microscopy and imaging have pushed the boundaries of what was once thought possible in many fields of research. New techniques, coupled with the application of new technologies, allow researchers to answer increasingly complex questions by probing deeper and with greater accuracy. While, these new techniques provide far greater specificity and increased sensitivity in regards to both resolution and frequency, the amount of data generated is swelling to a point where conventional data-management systems struggle to keep pace; this is especially true for large microscopy-and-imaging shared-user facilities. Sub-optimal data management can severely hinder the ability of a researcher to determine experimental results accurately or efficiently, and will inevitably limit the functionality of the research facility itself. This review discusses the source of the problem: how data are produced by systems available today, and the information's specificity and relative importance; techniques for management of these data to maximize functionality of the facility; and practices that can be detrimental in the research core environment.