A Case for Enrichment in Data Management Systems

Document Type

Article

Publication Date

6-1-2022

Abstract

We describe ENRICHDB, a new DBMS technology designed for emerging domains (e.g., sensor-driven smart spaces and social media analytics) that require incoming data to be enriched using expensive functions prior to its usage. To support online processing, today, such enrichment is performed outside of DBMSs, as a static data processing workflow prior to its ingestion into a DBMS. Such a strategy could result in a significant delay from the time when data arrives and when it is enriched and ingested into the DBMS, especially when the enrichment complexity is high. Also, enriching at ingestion could result in wastage of resources if applications do not use/require all data to be enriched. ENRICHDB's design represents a significant departure from the above, where we explore seamless integration of data enrichment all through the data processing pipeline-at ingestion, triggered based on events in the background, and progressively during query processing. The cornerstone of ENRICHDB is a powerful enrichment data and query model that encapsulates enrichment as an operator inside a DBMS enabling it to co-optimize enrichment with query processing. This paper describes this data model and provides a summary of the system implementation.

Identifier

85135462921 (Scopus)

Publication Title

SIGMOD Record

External Full Text Location

https://doi.org/10.1145/3552490.3552497

ISSN

01635808

First Page

38

Last Page

43

Issue

2

Volume

51

Grant

1527536

Fund Ref

National Science Foundation

This document is currently not available here.

Share

COinS