research-article

Getting Rid of Data

Author:

Tova MiloAuthors Info & Claims

Journal of Data and Information Quality (JDIQ), Volume 12, Issue 1

Article No.: 1, Pages 1 - 7

https://doi.org/10.1145/3326920

Published: 11 November 2019 Publication History

Abstract

We are experiencing an amazing data-centered revolution. Incredible amounts of data are collected, integrated, and analyzed, leading to key breakthroughs in science and society. This well of knowledge, however, is at a great risk if we do not dispense with some of the data flood. First, the amount of generated data grows exponentially and already at 2020 is expected to be more than twice the available storage. Second, even disregarding storage constraints, uncontrolled data retention risks privacy and security, as recognized, e.g., by the recent EU Data Protection reform. Data disposal policies must be developed to benefit and protect organizations and individuals.

Retaining the knowledge hidden in the data while respecting storage, processing, and regulatory constraints is a great challenge. The difficulty stems from the distinct, intricate requirements entailed by each type of constraint, the scale and velocity of data, and the constantly evolving needs. While multiple data sketching, summarization, and deletion techniques were developed to address specific aspects of the problem, we are still very far from a comprehensive solution. Every organization has to battle the same tough challenges with ad hoc solutions that are application-specific and rarely sharable.

In this article, we will discuss the logical, algorithmic, and methodological foundations required for the systematic disposal of large-scale data, for constraints enforcement and for the development of applications over the retained information. In particular, we will overview relevant related work, highlighting new research challenges and potential reuse of existing techniques.

References

[1]

E. Ainy, P. Bourhis, S. B. Davidson, D. Deutch, and T. Milo. 2015. Approximated summarization of data provenance. In Proceedings of the CIKM. 483--492.

[2]

Y. Amsterdamer, S. B. Davidson, D. Deutch, T. Milo, J. Stoyanovich, and V. Tannen. 2011. Putting lipstick on pig: Enabling database-style workflow provenance. PVLDB 5, 4 (2011), 346--357.

Digital Library

[3]

Yael Amsterdamer, Yael Grossman, Tova Milo, and Pierre Senellart. 2013. Crowd mining. In Proceedings of the SIGMOD. 241--252.

Digital Library

[4]

M. Besta and T. Hoefler. 2018. Survey and taxonomy of lossless graph compression and space-efficient graph representations. Retrieved from CoRR abs/1806.01799 (2018).

[5]

A. Calì, D. Calvanese, and M. Lenzerini. 2013. Data integration under integrity constraints. In Seminal Contributions to Information Systems Engineering, 25 Years of CAiSE. 335--352.

[6]

S. Chaudhuri, B. Ding, and S. Kandula. 2017. Approximate query processing: No silver bullet. In Proceedings of the SIGMOD.

[7]

C. Chen, B. Golshan, A. Y. Halevy, W. C. Tan, and A. Doan. 2018. BigGorilla: An open-source ecosystem for data preparation and integration. IEEE Data Eng. Bull. 41, 2 (2018), 10--22.

[8]

J. Cheney, L. Chiticariu, and W. C. Tan. 2009. Provenance in databases: Why, how, and where. Found. Trends Datab. 1, 4 (2009), 379--474.

Digital Library

[9]

G. Cormode. 2017. Data sketching. Commun. ACM 60, 9 (2017), 48--55.

Digital Library

[10]

D. Deutch and T. Milo. 2012. Business Processes: A Database Perspective. Morgan 8 Claypool Publishers.

[11]

D. Deutch and T. Milo. 2012. A structural/temporal query language for business processes. J. Comput. Syst. Sci. 78, 2 (2012), 583--609.

Digital Library

[12]

A. Doan, A. Y. Halevy, and Z. G. Ives. 2012. Principles of Data Integration. Morgan Kaufmann.

[13]

GDPR. (2016). General Data Protection Regulation (GDPR). Retrieved from https://en.wikipedia.org/wiki/General_Data_Protection_Regulation.

[14]

B. Glavic and G. Alonso. 2009. Perm: Processing provenance and data on the same data model through query rewriting. In Proceedings of the ICDE. 174--185.

[15]

T. J. Green and V. Tannen. 2017. The semiring framework for database provenance. In Proceedings of the PODS. 93--99.

[16]

Ihab F. Ilyas. 2016. Effective data cleaning with continuous evaluation. IEEE Data Eng. Bull. 39, 2 (2016), 38--46.

[17]

H. V. Jagadish, J. Gehrke, A. Labrinidis, Y. Papakonstantinou, J. M. Patel, R. Ramakrishnan, and C. Shahabi. 2014. Big data and its technical challenges. Commun. ACM 57, 7 (2014), 86--94.

Digital Library

[18]

M. L. Kersten and L. Sidirourgos. 2017. A database system with amnesia. In Proceedings of the CIDR.

[19]

C. Koch, D. Lupei, and V. Tannen. 2016. Incremental view maintenance for collection programming. In Proceedings of the PODS. 75--90.

[20]

T. Kraska, A. Beutel, E. Chi, J. Dean, and N. Polyzotis. 2018. The case for learned index structures. In Proceedings of the SIGMOD. 489--504.

[21]

Y. Liu, T. Safavi, A. Dighe, and D. Koutra. 2018. Graph summarization methods and applications: A survey. ACM Comput. Surv. 51, 3 (2018), 62:1--62:34.

[22]

R. J. Miller. 2017. The future of data integration. In Proceedings of the SIGKDD.

Digital Library

[23]

Tova Milo. 2017. The smart crowd—Learning from the ones who know. In Proceedings of the ICDT. 3:1--3:1.

[24]

Aditya G. Parameswaran, Akash Das Sarma, and Vipul Venkataraman. 2016. Optimizing open-ended crowdsourcing: The next frontier in crowdsourced data management. IEEE Data Eng. Bull. 39, 4 (2016), 26--37.

[25]

M. H. Rehman, C. S. Liew, A. Abbas, P. P. Jayaraman, T. Y. Wah, and S. U. Khan. 2016. Big data reduction methods: A survey. Data Sci. Eng. 1, 4 (2016), 265--284.

[26]

retention [n.d.]. Data Retention. Retrieved from https://en.wikipedia.org/wiki/Data_retention.

[27]

L. Rizzatti. 2016. Digital data storage is undergoing mind-boggling growth. EETimes Magazine (Sept. 14, 2016).

[28]

J. Skyt, C. S. Jensen, and T. Bach Pedersen. 2008. Specification-based data reduction in dimensional data warehouses. Inf. Syst. 33, 1 (2008), 36--63.

Digital Library

Cited By

Mersy GKrishnan SChien AEilam TPorter GAnderson TJosephson CPark J(2023)Toward a Life Cycle Assessment for the Carbon Footprint of DataProceedings of the 2nd Workshop on Sustainable Computer Systems10.1145/3604930.3605724(1-9)Online publication date: 9-Jul-2023
https://dl.acm.org/doi/10.1145/3604930.3605724
Yan LXu NLi GBhowmick SChoi BXu J(2022)SENSORProceedings of the VLDB Endowment10.14778/3554821.355486615:12(3650-3653)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554866
Teofili TFirmani DKoudas NMerialdo PSrivastava D(2022)CERTEMProceedings of the VLDB Endowment10.14778/3554821.355486415:12(3642-3645)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554864
Show More Cited By

Index Terms

Getting Rid of Data
1. Information systems
  1. Data management systems

Recommendations

Ready for data analytics?: data collection and creation in local governments
dg.o '18: Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age

A good understanding of local government data, particularly the contexts in which the data are collected and created, is essential in order for data guardians and users to effectively manage that data and to draw accurate and rich information from them. ...
Extended RBAC-based design and implementation for a secure data warehouse

This paper first discusses security issues for data warehousing. In particular, issues on building a secure data warehouse, secure data warehousing technologies as well as design issues are discussed. Our design of a secure data warehouse that enforces ...
A conceptual framework for managing very diverse data for complex, interdisciplinary science

Much attention has been given to the challenges of handling massive data volumes in modern data-intensive science. This paper examines an equally daunting challenge - the diversity of interdisciplinary data, notably research data, and the need to ...

Comments

Information & Contributors

Information

Published In

cover image Journal of Data and Information Quality

Journal of Data and Information Quality Volume 12, Issue 1

ON THE HORIZON, CHALLENGE PAPER, REGULAR PAPERS, and EXPERIENCE PAPER

March 2020

110 pages

ISSN:1936-1955

EISSN:1936-1963

DOI:10.1145/3372130

Editor:
Tiziana Catarci
Sapienza University of Rome, Rome, Italy

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 November 2019

Accepted: 01 April 2019

Revised: 01 April 2019

Received: 01 March 2019

Published in JDIQ Volume 12, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
679
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)2

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mersy GKrishnan SChien AEilam TPorter GAnderson TJosephson CPark J(2023)Toward a Life Cycle Assessment for the Carbon Footprint of DataProceedings of the 2nd Workshop on Sustainable Computer Systems10.1145/3604930.3605724(1-9)Online publication date: 9-Jul-2023
https://dl.acm.org/doi/10.1145/3604930.3605724
Yan LXu NLi GBhowmick SChoi BXu J(2022)SENSORProceedings of the VLDB Endowment10.14778/3554821.355486615:12(3650-3653)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554866
Teofili TFirmani DKoudas NMerialdo PSrivastava D(2022)CERTEMProceedings of the VLDB Endowment10.14778/3554821.355486415:12(3642-3645)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554864
Drien OFreiman MAmsterdamer Y(2022)ActivePDBProceedings of the VLDB Endowment10.14778/3554821.355486315:12(3638-3641)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554863
Huang KYe QZhao JZhao XHu HZhou X(2022)VINCENTProceedings of the VLDB Endowment10.14778/3554821.355486215:12(3634-3637)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554862
Davidson SGershtein SMilo TNovgorodov SShoshan M(2022)PHOcusProceedings of the VLDB Endowment10.14778/3554821.355486115:12(3630-3633)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.14778/3554821.3554861
Vu TEldawy AHristidis VTsotras V(2022)Incremental partitioning for efficient spatial data analyticsProceedings of the VLDB Endowment10.14778/3494124.349415015:3(713-726)Online publication date: 4-Feb-2022
https://dl.acm.org/doi/10.14778/3494124.3494150
Dosso DDavidson SSilvello G(2022)Credit distribution in relational scientific databasesInformation Systems10.1016/j.is.2022.102060109(102060)Online publication date: Dec-2022
https://doi.org/10.1016/j.is.2022.102060
Petersohn DTang DDurrani RMelik-Adamyan AGonzalez JJoseph AParameswaran A(2021)Flexible rule-based decomposition and metadata independence in modinProceedings of the VLDB Endowment10.14778/3494124.349415215:3(739-751)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.14778/3494124.3494152
Koutras CPsarakis KSiachamis GIonescu AFragkoulis MBonifati AKatsifodimos A(2021)Valentine in actionProceedings of the VLDB Endowment10.14778/3476311.347636614:12(2871-2874)Online publication date: 28-Oct-2021
https://dl.acm.org/doi/10.14778/3476311.3476366
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents