Compression Ratio Modeling and Estimation across Error Bounds for Lossy Compression

Document Type

Article

Publication Date

7-1-2020

Abstract

Scientific simulations on high-performance computing (HPC) systems generate vast amounts of floating-point data that need to be reduced in order to lower the storage and I/O cost. Lossy compressors trade data accuracy for reduction performance and have been demonstrated to be effective in reducing data volume. However, a key hurdle to wide adoption of lossy compressors is that the trade-off between data accuracy and compression performance, particularly the compression ratio, is not well understood. Consequently, domain scientists often need to exhaust many possible error bounds before they can figure out an appropriate setup. The current practice of using lossy compressors to reduce data volume is, therefore, through trial and error, which is not efficient for large datasets which take a tremendous amount of computational resources to compress. This paper aims to analyze and estimate the compression performance of lossy compressors on HPC datasets. In particular, we predict the compression ratios of two modern lossy compressors that achieve superior performance, SZ and ZFP, on HPC scientific datasets at various error bounds, based upon the compressors' intrinsic metrics collected under a given base error bound. We evaluate the estimation scheme using twenty real HPC datasets and the results confirm the effectiveness of our approach.

Identifier

85081632664 (Scopus)

Publication Title

IEEE Transactions on Parallel and Distributed Systems

External Full Text Location

https://doi.org/10.1109/TPDS.2019.2938503

e-ISSN

15582183

ISSN

10459219

First Page

1621

Last Page

1635

Issue

7

Volume

31

Grant

DE-AC05-00OR22725

Fund Ref

Office of Science

This document is currently not available here.

Share

COinS