“Superluminal” FITS File Processing on Multiprocessors: Zero Time Endian Conversion Technique

  • Eguchi S
1Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

The FITS is the standard file format in astronomy, and it has been extended to agree with astronomical needs of the day. However, astronomical datasets have been inflating year by year. In case of ALMA telescope, a ~ TB scale 4-dimensional data cube may be produced for one target. Considering that typical Internet bandwidth is a few 10 MB/s at most, the original data cubes in FITS format are hosted on a VO server, and the region which a user is interested in should be cut out and transferred to the user (Eguchi et al., 2012). The system will equip a very high-speed disk array to process a TB scale data cube in a few 10 seconds, and disk I/O speed, endian conversion and data processing one will be comparable. Hence to reduce the endian conversion time is one of issues to realize our system. In this paper, I introduce a technique named "just-in-time endian conversion", which delays the endian conversion for each pixel just before it is really needed, to sweep out the endian conversion time; by applying this method, the FITS processing speed increases 20% for single threading, and 40% for multi-threading compared to CFITSIO. The speed-up by the method tightly relates to modern CPU architecture to improve the efficiency of instruction pipelines due to break of "causality", a programmed instruction code sequence.

Cite

CITATION STYLE

APA

Eguchi, S. (2013). “Superluminal” FITS File Processing on Multiprocessors: Zero Time Endian Conversion Technique. Publications of the Astronomical Society of the Pacific, 125(927), 565–579. https://doi.org/10.1086/671105

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free