A high throughput workflow environment for cosmological simulations

Abstract

This is a preprint of a paper in the proceedings of the XSEDE12 conference, held July 16-19, 2012 in Chicago, IL. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or any other granting agency.The next generation of wide-area sky surveys offer the power to place extremely precise constraints on cosmological parameters and to test the source of cosmic acceleration. These observational programs will employ multiple techniques based on a variety of statistical signatures of galaxies and large-scale structure. These techniques have sources of systematic error that need to be understood at the percent-level in order to fully leverage the power of next-generation catalogs. Simulations of large-scale structure provide the means to characterize these uncertainties. We are using XSEDE resources to produce multiple synthetic sky surveys of galaxies and large-scale structure in support of science analysis for the Dark Energy Survey. In order to scale up our production to the level of fifty 1010-particle simulations, we are working to embed production control within the Apache Airavata workflow environment. We explain our methods and report how the workflow has reduced production time by 40% compared to manual management.This work is partially supported by the Extreme Science and Engineering Discovery Environment (XSEDE) Extended Collaborative Support Service (ECSS) Program funded by the National Science Foundation through the award: OCI-1053575, and by NSF OCI Award 1032742, SDCI NMI Improvement: Open Gateway Computing Environments|Tools for Cyberinfrastructure-Enabled Science and Education. AEE acknowledges support from NSF AST-0708150 and NASA NNX07AN58G