Bootstrapping in-situ workflow auto-Tuning via combining performance models of component applications

Document Type

Conference Proceeding

Publication Date

11-14-2021

Abstract

In an in-situ workflow, multiple components such as simulation and analysis applications are coupled with streaming data transfers the multiplicity of possible configurations necessitates an auto-Tuner for workflow optimization. Existing auto-Tuning approaches are computationally expensive because many configurations must be sampled by running the whole workflow repeatedly in order to train the autotuner surrogate model or otherwise explore the configuration space. To reduce these costs, we instead combine the performance models of component applications by exploiting the analytical workflow structure, selectively generating test configurations to measure and guide the training of a machine learning workflow surrogate model. Because the training can focus on well-performing configurations, the resulting surrogate model can achieve high prediction accuracy for good configurations despite training with fewer total configurations. Experiments with real applications demonstrate that our approach can identify significantly better configurations than other approaches for a fixed computer time budget.

Identifier

85119951786 (Scopus)

ISBN

[9781450384421]

Publication Title

International Conference for High Performance Computing Networking Storage and Analysis Sc

External Full Text Location

https://doi.org/10.1145/3458817.3476197

e-ISSN

21674337

ISSN

21674329

Fund Ref

U.S. Department of Energy

This document is currently not available here.

Share

COinS