Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Optimal Update Repair with Maximum Likelihood and Minimum Cost

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14850))

Included in the following conference series:

  • 280 Accesses

Abstract

In this paper, we study the problem of update repair under integrity constraints. For a set of inconsistent data, update repair modifies the attribute values of inconsistent tuples such that the modified data no longer violate the constraints. There usually exist multiple update repairs and it is difficult to determine which one is the optimal. Most previous works prefer the one with the minimum cost (a.k.a minimum update repair) to avoid excessive modifications to the original data. However, the repair quality of minimum update repair is often far from satisfactory. We intuitively notice that the erroneous attribute values usually have low co-occurrence with the correct ones. Therefore we propose to take both the quantitative statistics and the repair cost into consideration to repair the data. Specifically, we first formalize the update repair with maximum likelihood and minimum cost problem and analyze its hardness. Then we propose an efficient approach to generate candidate repairs based on the minimum update set. Finally, a probability model together with an efficient inference approach is proposed to compute the likelihood of each candidate. Extensive experiments on real-world datasets show that our proposal can achieve higher precision and recall compared with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/HoloClean/holoclean/blob/master/testdata/hospital.csv

  2. 2.

    https://github.com/HoloClean/holoclean/blob/master/testdata/flight.csv

  3. 3.

    https://www.kaggle.com/datasets/nickhould/craft-cans

  4. 4.

    https://db.unibas.it/projects/bart/

  5. 5.

    https://github.com/HoloClean/holoclean

  6. 6.

    https://github.com/BigDaMa/raha

  7. 7.

    https://github.com/daqcri/NADEEF

References

  1. Afrati, F.N., Kolaitis, P.G.: Repair checking in inconsistent databases: algorithms and complexity. In: Proceedings of the 12th International Conference on Database Theory, pp. 31–41 (2009)

    Google Scholar 

  2. Wijsen, J.: Database repairing using updates. In: SIGMOD, vol. 30 (2005)

    Google Scholar 

  3. Neutatz, F., Mahdavi, M., Abedjan, Z.: ED2: two-stage active learning for error detection. In: CIKM ’19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (2019)

    Google Scholar 

  4. Kolahi, S., Lakshmanan, L.V.S.: On approximating optimum repairs for functional dependency violations. In: Proceedings of the 12th International Conference on Database Theory, pp. 53–62 (2009)

    Google Scholar 

  5. Rekatsinas, T., et al.: HoloClean: holistic data repairs with probabilistic inference. In: arXiv preprint arXiv:1702.00820 (2017)

  6. Mahdavi, M., Abedjan, Z.: Baran: effective error correction via a unified context representation and transfer learning. VLDB Endow. 13(12), 1948–1961 (2020)

    Article  Google Scholar 

  7. Bohannon, P., Fan, W., Flaster, M., Rastogi, R.: A cost-based model and effective heuristic for repairing constraints by value modification. In: SIGMOD (2005)

    Google Scholar 

  8. Cong, G., Fan, W., Geerts, F., Jia, X., Ma, S.: Improving data quality: consistency and accuracy. In: ACM (2007)

    Google Scholar 

  9. Chu, X., et al.: KATARA: reliable data cleaning with knowledge bases and crowdsourcing. VLDB Endow. 8(12), 1952–1955 (2015)

    Article  Google Scholar 

  10. Kolahi, S., Lakshmanan, L.V.S.: On approximating optimum repairs for functional dependency violations. In: Proceedings of the 12th International Conference on Database Theory, pp. 53–62 (2009)

    Google Scholar 

  11. Dallachiesa, M., et al.: NADEEF: a commodity data cleaning system. In: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, pp. 541–552 (2013)

    Google Scholar 

  12. Khayyat, Z., et al.: Bigdansing: a system for big data cleansing. In: Proceedings of the 2015 ACM SIGMOD International Conference on Management Of Data, pp. 1215–1230 (2015)

    Google Scholar 

  13. Livshits, E., Kimelfeld, B., Roy, S.: Computing optimal repairs for functional dependencies. In: TODS, pp. 1–46 (2020)

    Google Scholar 

  14. Miao, D., et al.: The computation of optimal subset repairs. Proc. VLDB Endow. 13(12), 2061–2074 (2020)

    Google Scholar 

  15. Ge, C., et al.: A hybrid data cleaning framework using Markov logic networks. IEEE Trans. Knowl. Data Eng. 34(5), 2048–2062 (2020)

    Google Scholar 

  16. Chu, X., Ilyas, I. F., Papotti, P.: Holistic data cleaning: putting violations into context. In: ICDE, pp. 458–469 (2013)

    Google Scholar 

  17. Rezig, E.K., et al.: Horizon: scalable dependency-driven data cleaning. Proc. VLDB Endow. 14(11), 2546–2554 (2021)

    Google Scholar 

  18. Bar-Yehuda, R., Even, S.: A linear-time approximation algorithm for the weighted vertex cover problem. J. Algorithms 2(2), 198–203 (1981)

    Google Scholar 

Download references

Acknowledgment

The work is supported by the National Natural Science Foundation of China(No.62102271). Rui Zhu is corresponding author of this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rui Zhu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, W., Zhang, A., Zong, C., Zhu, R., Qiu, T. (2024). Optimal Update Repair with Maximum Likelihood and Minimum Cost. In: Onizuka, M., et al. Database Systems for Advanced Applications. DASFAA 2024. Lecture Notes in Computer Science, vol 14850. Springer, Singapore. https://doi.org/10.1007/978-981-97-5552-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5552-3_20

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5551-6

  • Online ISBN: 978-981-97-5552-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics