Article

Mining for patterns in contradictory data

Authors:

Johann-Christoph FreytagAuthors Info & Claims

IQIS '04: Proceedings of the 2004 international workshop on Information quality in information systems

Pages 51 - 58

https://doi.org/10.1145/1012453.1012463

Published: 18 June 2004 Publication History

Abstract

Information integration is often faced with the problem that different data sources represent the same set of the real-world objects, but give conflicting values for specific properties of these objects. Within this paper we present a model of such conflicts and describe an algorithm for efficiently detecting patterns of conflicts in a pair of overlapping data sources. The contradiction patterns we can find are a special kind of association rules, describing regularities in conflicts occurring together with certain attribute values, paris of attribute values, or with other conflicts. Therefore, we adapt existing association rule mining algorithms for mining contradiction patterns. Such patterns are an important tool for human experts that try to find and resolve problems in data quality using domain knowledge. We present the results of applying our method on a real world data set from the life science domain and show how it helps to generate clean data for integrated data warehouses.

References

[1]

Naumann, F., Quality-Driven Query Answering for Integrated Information Systems. Lecture Notes in Computer Science, LNCS 2261, Springer, 2002

Digital Library

[2]

Olson, J. E., Data Quality - The Accuracy Dimension. Morgan Kaufmann Publishers, ISBN1-55860-891-5, 2003

Digital Library

[3]

Rother, K., Müller, H., Trissl, S., Koch, I., Steinke, T., Preissner, R., Frömmel, C., Leser, U., COLUMBA: Multidimensional Data Integration of Protein Annotations. International Workshop on Data Integration in Life Sciences (DILS 2004), Leipzig, Germany

[4]

Benson, D. A., Karsch-Mizrachi, I., Lipman, D. J., Ostell, J., Wheeler, D. L. GenBank. Nucleic Acids Res., Vol. 31(1) (2003) 23--37

[5]

Kulikova, T., Aldebert, P., Althorpe, N., Baker, W., Bates, K., Browne, P., van den Broek, A., Cochrane, G., Duggan, K., Eberhardt, R., Faruque, N., Garcia-Pastor, M., Harte, N., Kanz, C., Leinonen, R., Lin, Q., Lombard, V., Lopez, R., Mancuso, R., McHale, M., Nardone, F., Silventoinen, V., Stoehr, P., Stoesser, G., Tuli, M. A., Tzouvara, K., Vaughan, R., Wu, D., Zhu, W., Apweiler. R., The EMBL Nucleotide Sequence Database. Nucl. Acids. Res. 2004 32:D27-D30.

[6]

Tateno, Y., Imanishi, T., Miyazaki, S., Fukami-Kobayashi, K., Saitou, N., Sugawara, H., Gojobori, T., DNA Data Bank of Japan (DDBJ) for genome scale research in life science. Nucl. Acids. Res. 2002 30:27--30

[7]

Bernstein, F. C., Koetzle, T. F., Willliams, G. J. B., Meyer, E. F. Jr., Brice, M. D., Rodgers, J. R., Kennard, O., Shimanouchi, T., Tasumi, M., The Protein Data Bank: a computer-based archival file for macromolecular structures. J. Mol. Biol., Vol. 112 (1977) 535--542

[8]

Bhat, T. N., Bourne, P., Feng, Z., Gilliland, G., Jain, S., Ravichandran, V., Schneider, B., Schneider, K., Thanki, N., Weissig, H., Westbrook, J., Berman, H. M., The PDB data uniformity project. Nucleic Acid Research, Vol. 29(1) (2001) 214--218

[9]

Boutselakis, H., Dimitropoulos, D., Fillon, J., Golovin, A., Henrick, K., Hussain, A., Ionides, J., John, M., Keller, P. A., Krissinel, E., McNeil, P., Naim, A., Newman, R., Oldfield, T., Pineda, J., Rachedi, A., Copeland, J., Sitnov, A., Sobhany, S., Suarez-Urunea, A., Swaminathan, J., Tagari, M., Tate, J., Tromm, S., Velankar, S., Vranken, W.; E-MSD: the European Bioinformatics Institute Macromolecular Structure Database. Nucleic Acid Research, Vol. 31(1) (2003) 458--462

[10]

Dasu, T., Johnson, T., Exploratory Data Mining and Data Cleaning. Wiley-Interscience, ISBN 0-471-26851-8, 2003

Digital Library

[11]

Naumann, F., Haeussler, M., Declarative Data Merging with Conflict Resolution. Proceedings of the International Conference on Information Quality (IQ 2002), Cambridge, MA, 2002

[12]

Müller, H., Naumann, F., Freytag, J. C., Data Quality in Genome Databases. Proceedings of the International Conference on Information Quality (IQ 2003), Cambridge, MA, 2003

[13]

Hui Xiong, Pang-Ning Tan, Kumar, V., Mining Strong Affinity Association Patterns in Data Sets with Skewed Support Distribution. Third IEEE International Conference on Data Mining, Melbourne, Florida, November 19--22, 2003

Digital Library

[14]

Agrawal, R., Srikant, R. Fast Algorithms for Mining Association Rules. Proc. Of the 20th Int'l Conf. On Very Large Data Bases, Santiago, Chile, Sept. 1994

Digital Library

[15]

Srikant, R, Agrawal, R Mining Quantiative Association Rules in Large Relational Tables. Proc. of the ACM-SIGMOD 1996 Conference on Management of Data, Montreal, Canada, June 1996

Digital Library

Cited By

Nwagwu HOkereke GNwobodo C(2017)Mining and visualising contradictory dataJournal of Big Data10.1186/s40537-017-0100-94:1Online publication date: 30-Oct-2017
https://doi.org/10.1186/s40537-017-0100-9
Bauckmann JAbedjan ZLeser UMüller HNaumann FChen XLebanon GWang HZaki M(2012)Discovering conditional inclusion dependenciesProceedings of the 21st ACM international conference on Information and knowledge management10.1145/2396761.2398580(2094-2098)Online publication date: 29-Oct-2012
https://dl.acm.org/doi/10.1145/2396761.2398580
Müller HFreytag JLeser U(2012)Improving data quality by source analysisJournal of Data and Information Quality10.1145/2107536.21075382:4(1-38)Online publication date: 2-Mar-2012
https://dl.acm.org/doi/10.1145/2107536.2107538
Show More Cited By

Mining for patterns in contradictory data
1. Information systems
  1. Information systems applications

Recommendations

Efficient algorithms for mining constrained frequent patterns from uncertain data
U '09: Proceedings of the 1st ACM SIGKDD Workshop on Knowledge Discovery from Uncertain Data

Mining of frequent patterns is one of the popular knowledge discovery and data mining (KDD) tasks. It also plays an essential role in the mining of many other patterns such as correlation, sequences, and association rules. Hence, it has been the subject ...
Data Mining Patterns: New Methods and Applications
Post sequential patterns mining: a new method for discovering structural patterns
Intelligent information processing II

In this paper we present a novel data mining technique, known as Post Sequential Patterns Mining, which can be used to discover Structural Patterns. A Structural Pattern is a new pattern, which is composed of sequential patterns, branch patterns or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IQIS '04: Proceedings of the 2004 international workshop on Information quality in information systems

June 2004

81 pages

ISBN:1581139020

DOI:10.1145/1012453

Program Chairs:
Felix Naumann
Humboldt Universität zu Berlin,Germany
,
Monica Scannapieco
Universit´ di Roma "La Sapienza",Italy

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

IQIS04

Sponsor:

SIGMOD

IQIS04: International Workshop on Information Quality in Information Systems 2004

18 06 2004

Paris, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
668
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)3

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nwagwu HOkereke GNwobodo C(2017)Mining and visualising contradictory dataJournal of Big Data10.1186/s40537-017-0100-94:1Online publication date: 30-Oct-2017
https://doi.org/10.1186/s40537-017-0100-9
Bauckmann JAbedjan ZLeser UMüller HNaumann FChen XLebanon GWang HZaki M(2012)Discovering conditional inclusion dependenciesProceedings of the 21st ACM international conference on Information and knowledge management10.1145/2396761.2398580(2094-2098)Online publication date: 29-Oct-2012
https://dl.acm.org/doi/10.1145/2396761.2398580
Müller HFreytag JLeser U(2012)Improving data quality by source analysisJournal of Data and Information Quality10.1145/2107536.21075382:4(1-38)Online publication date: 2-Mar-2012
https://dl.acm.org/doi/10.1145/2107536.2107538
Moussouni FBerti-Équille LRozé GLoréal OGuérin E(2007)QDexProceedings of the 2007 international conference on Web information systems engineering10.5555/1781503.1781506(5-16)Online publication date: 3-Dec-2007
https://dl.acm.org/doi/10.5555/1781503.1781506
Berti-Équille L(2007)Measuring and Modelling Data Quality for Quality-Awareness in Data MiningQuality Measures in Data Mining10.1007/978-3-540-44918-8_5(101-126)Online publication date: 2007
https://doi.org/10.1007/978-3-540-44918-8_5
Müller HFreytag JLeser UYu PTsotras VFox ELiu B(2006)Describing differences between databasesProceedings of the 15th ACM international conference on Information and knowledge management10.1145/1183614.1183702(612-621)Online publication date: 6-Nov-2006
https://dl.acm.org/doi/10.1145/1183614.1183702
Scannapieco MBerti-Équille L(2006)Report from the First and Second International Workshops on Information Quality in Information SystemsACM SIGMOD Record10.1145/1147376.114738435:2(50-52)Online publication date: 1-Jun-2006
https://dl.acm.org/doi/10.1145/1147376.1147384

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents