|Compression for scientific data - Thursday 1:30 PM - 5:00 PM:
Large-scale numerical simulations, observations and experiments are generating very large datasets that are difficult to analyze, store and transfer. Data compression is an attractive and efficient technique to significantly reduce the size of scientific datasets. This tutorial reviews the state of the art in lossless and lossy compression of scientific datasets, discusses in detail two lossy compressors (SZ and ZFP), introduces compression error assessment metrics, including error distribution analysis, and the Z-checker tool to analyze the difference between initial and decompressed datasets. The tutorial will offer a comprehensive hands on session on SZ and ZFP as well as Z-checker. The tutorial addresses the following questions: Why lossy compression? How does compression work? How to measure and control compression error? The tutorial uses examples of real world compressors and scientific datasets to illustrate the different compression techniques and their performance. The tutorial is given by two of the leading teams in this domain and targets an audience of beginners and advanced researchers and practitioners in scientific computing and data analytics. This 3h tutorial is improved from the tutorials given at the ISC17 and SC17 on the same topic by the same team.