Computing Just What You Need: Online Data Analysis and Reduction at Extreme Scales
Document Type
Conference Proceeding
Publication Date
1-1-2017
Abstract
A growing disparity between supercomputer computation speeds and I/O rates makes it increasingly infeasible for applications to save all results for offline analysis. Instead, applications must analyze and reduce data online so as to output only those results needed to answer target scientific question(s). This change in focus complicates application and experiment design and introduces algorithmic, implementation, and programming model challenges that are unfamiliar to many scientists and that have major implications for the design of various elements of supercomputer systems. We review these challenges and describe methods and tools that we are developing to enable experimental exploration of algorithmic, software, and system design alternatives.
Identifier
85028716568 (Scopus)
ISBN
[9783319642024]
Publication Title
Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics
External Full Text Location
https://doi.org/10.1007/978-3-319-64203-1_1
e-ISSN
16113349
ISSN
03029743
First Page
3
Last Page
19
Volume
10417 LNCS
Grant
DE-AC02-06CH11357
Fund Ref
U.S. Department of Energy
Recommended Citation
Foster, Ian; Ainsworth, Mark; Allen, Bryce; Bessac, Julie; Cappello, Franck; Choi, Jong Youl; Constantinescu, Emil; Davis, Philip E.; Di, Sheng; Di, Wendy; Guo, Hanqi; Klasky, Scott; Van Dam, Kerstin Kleese; Kurc, Tahsin; Liu, Qing; Malik, Abid; Mehta, Kshitij; Mueller, Klaus; and Munson, Todd, "Computing Just What You Need: Online Data Analysis and Reduction at Extreme Scales" (2017). Faculty Publications. 10083.
https://digitalcommons.njit.edu/fac_pubs/10083
