Error-controlled Progressive Retrieval of Scientific Data under Derivable Quantities of Interest

Document Type

Conference Proceeding

Publication Date

1-1-2024

Abstract

The unprecedented amount of scientific data has introduced heavy pressure on the current data storage and transmission systems. Progressive compression has been proposed to mitigate this problem, which offers data access with on-demand precision. However, existing approaches only consider precision control on primary data, leaving uncertainties on the quantities of interest (QoIs) derived from it. In this work, we present a progressive data retrieval framework with guaranteed error control on derivable QoIs. Our contributions are three-fold. (1) We carefully derive the theories to strictly control QoI errors during progressive retrieval. Our theory is generic and can be applied to any QoIs that can be composited by the basis of derivable QoIs proved in the paper. (2) We design and develop a generic progressive retrieval framework based on the proposed theories, and optimize it by exploring feasible progressive representations. (3) We evaluate our framework using five real-world datasets with a diverse set of QoIs. Experiments demonstrate that our framework can faithfully respect any user-specified QoI error bounds in the evaluated applications. This leads to over 2.02 × performance gain in data transfer tasks compared to transferring the primary data while guaranteeing a QoI error that is less than 1E-5.

Identifier

85194010883 (Scopus)

ISBN

[9798350352917]

Publication Title

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

External Full Text Location

https://doi.org/10.1109/SC41406.2024.00092

e-ISSN

21674337

ISSN

21674329

Grant

OAC-2311757

Fund Ref

Advanced Scientific Computing Research

This document is currently not available here.

Share

COinS