Real-time data compression system for data-intensive scientific applications using fpga architecture

Abstract

Particle accelerators are continually advancing and offer insights into the world of molecules, atoms, and particles on the ever shorter length and timescales. A variety of detectors, which are connected to different front-end electronics are installed in various kinds of Data Acquisition (DAQ) systems, to collect a huge amount of raw data. This goes along with a rapid and highly accurate transformation of analog quantities into discrete values for electronic storage and processing with exponentially increasing amounts of data. Therefore, data reduction or compression is an important feature for the DAQ systems to reduce the size of the data transmission path between the detectors and the computing units or storage devices. The flexibility of the Field Programmable Gate Arrays (FPGAs) allows the implementation of real-time data compression algorithms inside these DAQ systems. In this contribution, we will present our developed real-time data compression technique for continuous data recorded by high-speed imaging detectors at the terahertz source facility at ELBE particle accelerator. The hardware implementation of the algorithm proved its real-time suitability by compressing one hundred thousand consecutive input signals without introducing dead time.