Huge data but small programs: Visualization design via multiple embedded DSLs

Abstract

Although applications of functional programming are diverse, most examples deal with modest amounts of data - no more than a few megabytes. This paper describes how Haskell has been used to address a challenging astrophysics visualization problem, where the complete uncompressed dataset is nearly a terabyte. Our solution makes extensive use of three novel domain-specific languages: to specify data resources, to abstract over rendering operations, and most significantly, to design the desired visualization. The result is a powerful framework for time-varying multi-field visualization. This approach represents a significant departure from standard practices in the visualization field, and has application well beyond the original problem. That our solution consists of less than 4.5K lines of code is itself a notable result. This paper motivates and describes the overall architecture of our solution, and technical features of the DSLs that are used in place of the traditional visualization pipeline. © 2009 Springer Berlin Heidelberg.