Climbing the Data Mountain: Processing of SFX Data

Abstract

Serial femtosecond crystallography experiments produce mountains of data that require FEL facilities to provide many petabytes of storage space and large compute clusters for timely processing of user data. The route to reach the summit of the data mountain requires peak finding, indexing, integration, refinement, and phasing. Those who reach the summit get a crystal clear view of the “radiation damage-free” structure of a protein that is most consistent with the observed measurements. Data processing plays a critical role in the ability to measure accurate structure factor intensities from individual diffraction snapshots and combine them in three-dimensional space. Current developments in SFX aim to take into account the huge complexity of SFX experiments, modeling variations in the beam and crystals, uncertainties in geometry, partiality, mosaicity, and figures of merit that are unique to SFX.