article

Monadic datalog and the expressive power of languages for Web information extraction

Authors:

Christoph KochAuthors Info & Claims

Journal of the ACM (JACM), Volume 51, Issue 1

Pages 74 - 113

https://doi.org/10.1145/962446.962450

Published: 01 January 2004 Publication History

Abstract

Research on information extraction from Web pages (wrapping) has seen much activity recently (particularly systems implementations), but little work has been done on formally studying the expressiveness of the formalisms proposed or on the theoretical foundations of wrapping. In this paper, we first study monadic datalog over trees as a wrapping language. We show that this simple language is equivalent to monadic second order logic (MSO) in its ability to specify wrappers. We believe that MSO has the right expressiveness required for Web information extraction and propose MSO as a yardstick for evaluating and comparing wrappers. Along the way, several other results on the complexity of query evaluation and query containment for monadic datalog over trees are established, and a simple normal form for this language is presented. Using the above results, we subsequently study the kernel fragment Elog⁻ of the Elog wrapping language used in the Lixto system (a visual wrapper generator). Curiously, Elog⁻ exactly captures MSO, yet is easier to use. Indeed, programs in this language can be entirely visually specified.

References

[1]

Abiteboul, S., Hull, R., and Vianu, V. 1995. Foundations of Databases. Addison-Wesley, Reading, Mass.]]

Digital Library

[2]

Abiteboul, S., and Vianu, V. 1999. Regular path queries with constraints. J. Comput. Syst. Sci. 58, 3, 428--452.]]

Digital Library

[3]

Aho, A. V., Beeri, C., and Ullman, J. D. 1979. The theory of joins in relational databases. ACM Trans. Datab. Syst. 4, 3, 297--314.]]

Digital Library

[4]

Atzeni, P., and Mecca, G. 1997. Cut and paste. In Proceedings of the 16th ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS'97) (Tucson, Az.). ACM, New York.]]

Digital Library

[5]

Baumgartner, R., Flesca, S., and Gottlob, G. 2001a. Declarative information extraction, web crawling, and recursive wrapping with lixto. In Proceedings of LPNMR'01 (Vienna, Austria).]]

Digital Library

[6]

Baumgartner, R., Flesca, S., and Gottlob, G. 2001b. Visual web information extraction with lixto. In Proceedings of the 27th International Conference on Very Large Data Bases (VLDB'01).]]

Digital Library

[7]

Beeri, C., and Bernstein, P. A. 1979. Computational problems related to the design of normal form relational schemas. ACM Trans. Datab. Syst. 4, 1, 30--59.]]

Digital Library

[8]

Brüggemann-Klein, A., Murata, M., and Wood, D. 2001. Regular tree and regular hedge languages over non-ranked alphabets: Version 1, April 3, 2001. Tech. Rep. HKUST-TCSC-2001-05, Hong Kong University of Science and Technology, Hong Kong SAR, China.]]

[9]

Brüggemann-Klein, A., and Wood, D. 2000. Caterpillars: A context specification technique. Markup Lang. 2, 1, 81--106.]]

[10]

Ceri, S., Gottlob, G., and Tanca, L. 1990. Logic Programming and Databases. Springer-Verlag, Berlin, Germany.]]

Digital Library

[11]

Cosmadakis, S., Gaifman, H., Kanellakis, P., and Vardi, M. 1988. Decidable optimization problems for database logic programs. In Proceedings of the 20th Annual ACM Symposium on Theory of Computing (Chicago, Ill.). ACM, New York, 477--490.]]

Digital Library

[12]

Courcelle, B. 1990. Graph rewriting: An algebraic and logic approach. In Handbook of Theoretical Computer Science, J. van Leeuwen, Ed. Vol. 2. Elsevier Science Publishers B.V., Chap. 5, 193--242.]]

Digital Library

[13]

Doner, J. 1970. Tree acceptors and some of their applications. J. Comput. Syst. Sci. 4, 406--451.]]

Digital Library

[14]

Dowling, W. F., and Gallier, J. H. 1984. Linear-time algorithms for testing the satisfiability of propositional horn formulae. J. Logic Prog. 1, 3, 267--284.]]

[15]

Ebbinghaus, H.-D., and Flum, J. 1999. Finite Model Theory. 2nd ed. Springer-Verlag, New York.]]

[16]

Flum, J., Frick, M., and Grohe, M. 2001. Query evaluation via tree-decompositions. In Proceedings of the 8th International Conference on Database Theory (ICDT'01), J. Van den Bussche and V. Vianu, Eds. Lecture Notes in Computer Science, vol. 1973. Springer, New York, 22--38.]]

Digital Library

[17]

Frick, M., and Grohe, M. 2002. The complexity of first-order and monadic second-order logic revisited. In Proceedings of the 17th Annual IEEE Symposium on Logic in Computer Science (LICS). IEEE Computer Society Press, Los Alamitos, Calif., 215--224.]]

Digital Library

[18]

Frick, M., Grohe, M., and Koch, C. 2003. Query evaluation on compressed trees. In Proceedings of the 18th Annual IEEE Symposium on Logic in Computer Science (LICS) (Ottawa, Ont., Canada). IEEE Computer Society Press, Los Alamitos, Calif.]]

Digital Library

[19]

Gécseg, F., and Steinby, M. 1997. Tree languages. In Handbook of Formal Languages, G. Rozenberg and A. Salomaa, Eds. Vol. 3. Springer-Verlag, New York, Chap. 1, 1--68.]]

Digital Library

[20]

Gottlob, G., Grädel, E., and Veith, H. 2002. Datalog LITE: A deductive query language with linear-time model checking. ACM Trans. Computat. Logic 3, 1, 42--79.]]

Digital Library

[21]

Gottlob, G., and Koch, C. 2002. Monadic queries over tree-structured data. In Proceedings of the 17th Annual IEEE Symposium on Logic in Computer Science (LICS) (Copenhagen, Denmark). IEEE Computer Society Press, Los Alamitos, Calif., 189--202.]]

Digital Library

[22]

Gottlob, G., Koch, C., and Pichler, R. 2002. Efficient algorithms for processing XPath queries. In Proceedings of the 28th International Conference on Very Large Data Bases (VLDB'02) (Hong Kong, China).]]

Digital Library

[23]

Hopcroft, J. E., and Ullman, J. D. 1979. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley Publishing Company, Reading, Mass.]]

Digital Library

[24]

Laender, A. H. F., Ribeiro-Neto, B., and S. da Silva, A. 2002. DEByE---Data Extraction By Example. Data Knowl. Eng. 40, 2 (Feb.), 121--154.]]

Digital Library

[25]

Liu, L., Pu, C., and Han, W. 2000. XWRAP: An XML-enabled wrapper construction system for web information sources. In Proceedings of the 16th IEEE International Conference on Data Engineering (ICDE) (San Diego, Calif.). ACM, New York, 611--621.]]

Digital Library

[26]

Lixto. http://www.lixto.com.]]

[27]

Ludäscher, B., Himmeröder, R., Lausen, G., May, W., and Schlepphorst, C. 1998. Managing semistructured data with florid: A deductive object-oriented perspective. Inf. Syst. 23, 8, 1--25.]]

Digital Library

[28]

Maier, D., Mendelzon, A. O., and Sagiv, Y. 1979. Testing implications of data dependencies. ACM Trans. Datab. Syst. 4, 4, 455--469.]]

Digital Library

[29]

Miklau, G., and Suciu, D. 2003. Containment and equivalence for an fragment of XPath. J. ACM 51, 1 (Jan.), 2--45.]]

Digital Library

[30]

Minoux, M. 1988. LTUR: A simplified linear-time unit resolution algorithm for horn formulae and computer implementation. Inf. Proc. Lett. 29, 1, 1--12.]]

Digital Library

[31]

Neven, F. 2002. On the power of walking for querying tree-structured data. In Proceedings of the 21st ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS'02). Madison, Wisconsin.]]

Digital Library

[32]

Neven, F., and Schwentick, T. 2000. Expressive and efficient pattern languages for tree-structured data. In Proceedings of the ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS'00) (Dallas, Tex.), ACM, New York, 145--156.]]

Digital Library

[33]

Neven, F., and Schwentick, T. 2002. Query automata on finite trees. Theoret. Comput. Sci. 275, 633--674.]]

Digital Library

[34]

Neven, F., and Schwentick, T. 2003. XPath containment in the presence of disjunction, DTDs, and variables. In Proceedings of the 9th International Conference on Database Theory (ICDT'03). 315--329.]]

Digital Library

[35]

Neven, F., and van den Bussche, J. 2002. Expressiveness of structured document query languages based on attribute grammars. J. ACM 49, 1 (Jan.), 56--100.]]

Digital Library

[36]

Papadimitriou, C. H. 1994. Computational Complexity. Addison-Wesley, Reading, Mass.]]

[37]

Papakonstantinou, Y., Gupta, A., Garcia-Molina, H., and Ullman, J. 1995. A query translation scheme for rapid implementation of wrappers. In Proceedings of the 4th International Conference on Deductive and Object-oriented Databases (DOOD'95) (Singapore), Springer-Verlag, New York, 161--186.]]

Digital Library

[38]

Sahuguet, A., and Azavant, F. 2001. Building intelligent web applications using lightweight wrappers. Data Knowl. Eng. 36, 3, 283--316.]]

Digital Library

[39]

Szilard, A., Yu, S., Zhang, K., and Shallit, J. 1992. Characterizing regular languages with polynomial densities. In Proceedings of the 17th International Symposium on Mathematical Foundations of Computer Science. Lecture Notes in Computer Science, vol. 629. Springer-Verlag, Berlin, Germany, 494--503.]]

Digital Library

[40]

Thatcher, J., and Wright, J. 1968. Generalized finite automata theory with an application to a decision problem of second-order logic. Math. Syst. Theory 2, 1, 57--81.]]

[41]

Thomas, W. 1990. Automata on infinite objects. In Handbook of Theoretical Computer Science, J. van Leeuwen, Ed. Vol. 2. Elsevier Science Publishers B.V., Chap. 4, 133--192.]]

Digital Library

[42]

Thomas, W. 1997. Languages, automata, and logic. In Handbook of Formal Languages, G. Rozenberg and A. Salomaa, Eds. Vol. 3. Springer-Verlag, New York, Chap. 7, 389--455.]]

Digital Library

[43]

World Wide Web Consortium. 1999. XML Path Language (XPath) Recommendation. http://www. w3c.org/TR/xpath/.]]

[44]

Yannakakis, M. 1981. Algorithms for acyclic database schemes. In Proceedings of the 7th International Conference on Very Large Data Bases (VLDB'81). 82--94.]]

Digital Library

[45]

Yu, S. 1997. Regular languages. In Handbook of Formal Languages, G. Rozenberg and A. Salomaa, Eds. Vol. 1. Springer-Verlag, New York, Chap. 2, 41--110.]]

Digital Library

Cited By

Zhao HDeep SKoutris PRoy STannen V(2024)Evaluating Datalog over Semirings: A Grounding-based ApproachProceedings of the ACM on Management of Data10.1145/36515912:2(1-26)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3651591
Jiang HFan ZWang JLiu SLin W(2024)The determinants of product trust in live streaming E-commerce: a hybrid method integrating SEM and fsQCAAsia Pacific Journal of Marketing and Logistics10.1108/APJML-01-2024-0048Online publication date: 14-Aug-2024
https://doi.org/10.1108/APJML-01-2024-0048
Stephenson M(2023)A Differential Datalog InterpreterSoftware10.3390/software20300202:3(427-446)Online publication date: 21-Sep-2023
https://doi.org/10.3390/software2030020
Show More Cited By

Index Terms

Monadic datalog and the expressive power of languages for Web information extraction

Recommendations

Monadic datalog and the expressive power of languages for web information extraction
PODS '02: Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems

Research on information extraction from Web pages (wrapping) has seen much activity in recent times (particularly systems implementations), but little work has been done on formally studying the expressiveness of the formalisms proposed or on the ...
Datalog LITE: a deductive query language with linear time model checking

We present Datalog LITE, a new deductive query language with a linear-time model-checking algorithm, that is, linear time data complexity and program complexity. Datalog LITE is a variant of Datalog that uses stratified negation, restricted variable ...
On the expressive power of temporal concurrent constraint programming languages
PPDP '02: Proceedings of the 4th ACM SIGPLAN international conference on Principles and practice of declarative programming

The tcc paradigm is a formalism for timed concurrent constraint programming. Several tcc languages differing in their way of expressing infinite behavior have been proposed in the literature. In this paper we study the expressive power of some of these ...

Comments

Information & Contributors

Information

Published In

cover image Journal of the ACM

Journal of the ACM Volume 51, Issue 1

January 2004

113 pages

ISSN:0004-5411

EISSN:1557-735X

DOI:10.1145/962446

Issue’s Table of Contents

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2004

Published in JACM Volume 51, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

132
Total Citations
View Citations
1,979
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao HDeep SKoutris PRoy STannen V(2024)Evaluating Datalog over Semirings: A Grounding-based ApproachProceedings of the ACM on Management of Data10.1145/36515912:2(1-26)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3651591
Jiang HFan ZWang JLiu SLin W(2024)The determinants of product trust in live streaming E-commerce: a hybrid method integrating SEM and fsQCAAsia Pacific Journal of Marketing and Logistics10.1108/APJML-01-2024-0048Online publication date: 14-Aug-2024
https://doi.org/10.1108/APJML-01-2024-0048
Stephenson M(2023)A Differential Datalog InterpreterSoftware10.3390/software20300202:3(427-446)Online publication date: 21-Sep-2023
https://doi.org/10.3390/software2030020
Gottlob G(2022)Adventures with Datalog: Walking the Thin Line Between Theory and PracticeAIxIA 2022 – Advances in Artificial Intelligence10.1007/978-3-031-27181-6_34(489-500)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-27181-6_34
Vianu VLibkin LPichler RGuagliardo P(2021)Datalog UnchainedProceedings of the 40th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems10.1145/3452021.3458815(57-69)Online publication date: 20-Jun-2021
https://dl.acm.org/doi/10.1145/3452021.3458815
Suh JWilliams SFann JFogarty JBauer AHsieh G(2020)Parallel Journeys of Patients with Cancer and Depression: Challenges and Opportunities for Technology-Enabled Collaborative CareProceedings of the ACM on Human-Computer Interaction10.1145/33928434:CSCW1(1-36)Online publication date: 29-May-2020
https://dl.acm.org/doi/10.1145/3392843
Lustig CRintel SScult LSuri S(2020)Stuck in the middle with you: The Transaction Costs of Corporate Employees Hiring FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/33928424:CSCW1(1-28)Online publication date: 29-May-2020
https://dl.acm.org/doi/10.1145/3392842
Jackson CØsterlund CCrowston KHarandi MTrouille L(2020)Shifting forms of Engagement: Volunteer Learning in Online Citizen ScienceProceedings of the ACM on Human-Computer Interaction10.1145/33928414:CSCW1(1-19)Online publication date: 29-May-2020
https://dl.acm.org/doi/10.1145/3392841
Xu TFeng ASong XGao ZZeng X(2020)Chinese News Data Extraction System Based on Readability AlgorithmArtificial Intelligence and Security10.1007/978-981-15-8083-3_14(153-164)Online publication date: 13-Sep-2020
https://doi.org/10.1007/978-981-15-8083-3_14
Maier DTekle KKifer MWarren D(2018)DatalogDeclarative Logic Programming10.1145/3191315.3191317(3-100)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1145/3191315.3191317
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents