Skip to content
Conference proceedings

Parallel I/O, analysis, and visualization of a trillion particle simulation

Byna S, Chou J, R??bel O, Prabhat, Karimabadi H, Daughter W, Roytershteyn V, Bethel E, Howison M, Hsu K, Lin K, Shoshani A, Uselton A, Wu K ...see all

International Conference for High Performance Computing, Networking, Storage and Analysis, SC (2012)

  • 25

    Readers

    Mendeley users who have this article in their library.
  • 13

    Citations

    Citations of this article.
  • N/A

    Views

    ScienceDirect users who have downloaded this article.
Sign in to save reference

Abstract

Petascale plasma physics simulations have recently entered the regime of simulating trillions of particles. These unprecedented simulations generate massive amounts of data, posing significant challenges in storage, analysis, and visualization. In this paper, we present parallel I/O, analysis, and visualization results from a VPIC trillion particle simulation running on 120,000 cores, which produces ~ 30TB of data for a single timestep. We demonstrate the successful application of H5Part, a particle data extension of parallel HDF5, for writing the dataset at a significant fraction of system peak I/O rates. To enable efficient analysis, we develop hybrid parallel FastQuery to index and query data using multi-core CPUs on distributed memory hardware. We show good scalability results for the FastQuery implementation using up to 10,000 cores. Finally, we apply this indexing/query-driven approach to facilitate the first-ever analysis and visualization of the trillion particle dataset.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Get full text

Authors

  • Surendra Byna

  • Jerry Chou

  • Oliver R??bel

  • Prabhat

  • Homa Karimabadi

  • William S. Daughter

Cite this document

Choose a citation style from the tabs below