research-article

High-performance complex event processing over hierarchical data

Authors:

Barzan Mozafari,

Loris D'antoni,

Carlo ZanioloAuthors Info & Claims

ACM Transactions on Database Systems (TODS), Volume 38, Issue 4

Article No.: 21, Pages 1 - 39

https://doi.org/10.1145/2536779

Published: 04 December 2013 Publication History

Abstract

While Complex Event Processing (CEP) constitutes a considerable portion of the so-called Big Data analytics, current CEP systems can only process data having a simple structure, and are otherwise limited in their ability to efficiently support complex continuous queries on structured or semistructured information. However, XML-like streams represent a very popular form of data exchange, comprising large portions of social network and RSS feeds, financial feeds, configuration files, and similar applications requiring advanced CEP queries. In this article, we present the XSeq language and system that support CEP on XML streams, via an extension of XPath that is both powerful and amenable to an efficient implementation. Specifically, the XSeq language extends XPath with natural operators to express sequential and Kleene-* patterns over XML streams, while remaining highly amenable to efficient execution. In fact, XSeq is designed to take full advantage of the recently proposed Visibly Pushdown Automata (VPA), where higher expressive power can be achieved without compromising the computationally attractive properties of finite state automata. Besides the efficiency and expressivity benefits, the choice of VPA as the underlying model also enables XSeq to go beyond XML streams and be easily applicable to any data with both sequential and hierarchical structures, including JSON messages, RNA sequences, and software traces. Therefore, we illustrate the XSeq's power for CEP applications through examples from different domains and provide formal results on its expressiveness and complexity. Finally, we present several optimization techniques for XSeq queries. Our extensive experiments indicate that XSeq brings outstanding performance to CEP applications: two orders of magnitude improvement is obtained over the same queries executed in general-purpose XML engines.

References

[1]

Alexander, M., Fawcett, J., and Runciman, P. 2000. Nursing Practice: Hospital and Home: The Adult 2^nd Ed. Churchill Livingstone.

[2]

Alur, R. and Madhusudan, P. 2004. Visibly pushdown languages. In Proceedings of the Symposium on Theory of Computing (STOC'04).

Digital Library

[3]

Alur, R. and Madhusudan, P. 2006. Adding nesting structure to words. In Proceedings of the 10^th International Conference on Developments in Language Theory (DLT'06). Lecture Notes in Computer Science, vol. 4036, Springer, 1--13.

Digital Library

[4]

Amagasa, T., Yoshikawa, M., and Uemura, S. 2000. A data model for temporal xml documents. In Proceedings of the 11^th International Conference on Database and Expert Systems Applications (DEXA'00). 334--344.

Digital Library

[5]

Bamford, R., Borkar, V., Branter, M., Fischer, P. M., Florescu, D., et al. 2009. Xquery reloaded. Proc. VLDB Endow. 2, 2, 1342--1353.

Digital Library

[6]

Barton, C., Charles, P., Goyal, D., Raghavachari, M., Fontoura, M., et al. 2003. Streaming xpath processing with forward and backward axes. In Proceedings of the 19^th International Conference on Data Engineering (ICDE'03). 455--466.

[7]

Boncz, P., Gurst, T., van Keulen, M., Manegold, S., Rittinger, J., et al. 2006. Monetdb/xquery: A fast xquery processor powered by a relational engine. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'06).

Digital Library

[8]

Brenna, L., Gehrke, J., Hong, M., and Johansen, D. 2009. Distributed event stream processing with non-deterministic finite automata. In Proceedings of the 3^rd International Conference on Distributed Event-Based Systems (DEBS'09).

Digital Library

[9]

Cate, B. T. and Lutz, C. 2009. The complexity of query containment in expressive fragments of xpath 2.0. J. ACM 56, 6.

Digital Library

[10]

Chen, Y., Davidson, S. B., and Zheng, Y. 2006. An efficient xpath query processor for xml streams. In Proceedings of the 22^nd International Conference on Data Engineering (ICDE'06).

Digital Library

[11]

Diao, Y., Altinel, M., Franklin, M. J., Zhang, H., and Fischer, P. 2003. Path sharing and predicate evaluation for high-performance xml filtering. ACM Trans. Datab. Syst. 28,4.

Digital Library

[12]

Florescu, D., Hilliery, C., Kossman, D., Lucas, P., Riccardi, F., et al. 2003. The bea/xqrl streaming xquery processor. In Proceedings of the 29^th International Conference on Very Large Databases (VLDB'03).

Digital Library

[13]

Furche, T., Gottlob, G., Grasso, G., Schallhart, C., and Sellers, A. J. 2011. Oxpath: A language for scalable, memory-efficient data extraction from web applications. Proc. VLDB Endow. 4, 11.

[14]

Gauwin, O. and Niehren, J. 2011. Streamable fragments of forward xpath. In Proceedings of the 16^th International Conference on Implementation and Application of Automata (CIAA'11). 3--15.

Digital Library

[15]

Gauwin, O., Niehren, J., and Tison, S. 2011. Queries on xml streams with bounded delay and concurrency. Inf. Comput. 209, 3, 409--442.

Digital Library

[16]

Josifovski, V., Fontoura, M., and Barta, A. 2005. Querying xml streams. VLDB J. 14,2.

Digital Library

[17]

Kay, M. 2008. Ten reasons why saxon xquery is fast. IEEE Data Engin. Bull. 31, 4.

[18]

Knuth, D. E., Morris, J. H. Jr., and Pratt, V. R. 1977. Fast pattern matching in strings. SIAM J. Comput. 6, 1, 323--350.

Digital Library

[19]

Koch, C. 2009. Xml stream processing. In Encyclopedia of Database Systems, Springer, 3634--3637

[20]

Laptev, N. and Zaniolo, C. 2012. Optimization of massive pattern queries by dynamic configuration morphing. In Proceedings of the 28^th International Conference on Data Engineering (ICDE'12). 917--928.

Digital Library

[21]

Luckham, D. C. 2001. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley.

Digital Library

[22]

Madhusudan, P. and Viswanathan, M. 2009. Query automata for nested words. In Proceedings of the 34^th International Symposium on Mathematical Foundations of Computer Science (MFCS'09). Lecture Notes in Computer Science, vol. 5734, Springer, 561--573.

Digital Library

[23]

Marx, M. 2005. Conditional xpath. Trans. Datab. Syst. 30, 4.

Digital Library

[24]

Mozafari, B., Zeng, K., and Zaniolo, C. 2010a. From regular expressions to nested words: Unifying languages and query execution for relational and xml sequences. Proc. VLDB Endow. 3, 1.

Digital Library

[25]

Mozafari, B., Zeng, K., and Zaniolo, C. 2010b. K&ast;sql: A unifying engine for sequence patterns and xml. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'10). 1143--1146.

Digital Library

[26]

Mozafari, B., Zeng, K., and Zaniolo, C. 2012. High-performance complex event processing over xml streams. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'12). 253--264.

Digital Library

[27]

Olteanu, D., Kiesling, T., and Bry, F. 2003. An evaluation of regular path expressions with qualifiers against xml streams. In Proceedings of the 19^th International Conference on Data Engineering (ICDE'03). 702--704.

[28]

Peng, F. and Chawathe, S. S. 2003. Xpath queries on streaming data. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'03).

Digital Library

[29]

Pitcher, C. 2005. Visibly pushdown expression effects for xml stream processing. In Proceedings of the Workshop on Programming Language Technologies for XML (PLAN-X'05).

[30]

Schmidt, A., Wass, F., Kersten, M., Carey, M. J., Manoescu, I., and Busse, R. 2002. Xmark: A benchmark for xml data management. In Proceedings of the 28^th International Conference on Very Large Data Bases (VLDB'02). 974--985.

Digital Library

[31]

Snodgrass, R. T. 2009. Tsql2. In Encyclopedia of Database Systems, Springer, 3192--3197.

[32]

Stromback, L. and Schmidt, S. 2009. An extension of xquery for graph analysis of biological pathways. In Proceedings of the 1^st International Conference on Advances on Databases, Knowledge, and Data Applications (DBKDA'09). 22--27.

Digital Library

[33]

Tang, N. V. 2009. A tighter bound for the determinization of visibly pushdown automata. In Proceedings of the International Workshop on Verification of Infinite-State Systems (INFINITY'09). 62--76.

[34]

ten Cate, B. 2006. The expressivity of xpath with transitive closure. In Proceedings of the 25^th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'06). 328-337.

Digital Library

[35]

ten Cate, B. and Marx, M. 2007a. Axiomatizing the logical core of xpath 2.0. In Proceedings of the 11^th International Conference on Database Theory (ICDT'07).

Digital Library

[36]

ten Cate, B. and Marx, M. 2007b. Navigational xpath: calculus and algebra. SIGMOD Rec. 36, 2.

Digital Library

[37]

ten Cate, B. and Segoufin, L. 2008. Xpath, transitive closure logic, and nested tree walking automata. In Proceedings of the 27^th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'08). 251--260.

Digital Library

[38]

Vagena, Z., Moro, M. M., and Tsotras, V. J. 2007. Roxsum: Leveraging data aggregation and batch processing for xml routing. In Proceedings of the 23^rd IEEE International Conference on Data Engineering (ICDE'07). 1466--1470.

[39]

Wang, F., Zaniolo, C., and Zhou, X. 2008. Archis: an xml-based approach to transaction-time temporal database systems. VLDB J. 17, 6, 1445--1463.

Digital Library

[40]

Wu, E., Diao, Y., and Rizvi, S. 2006. High-performance complex event processing over streams. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'06). 407--418.

Digital Library

[41]

Zaniolo, C. 2009. Event-oriented data models and temporal queries in transaction-time databases. In Proceedings of the 16^th International Symposium on Temporal Representation and Reasoning (TIME'09). 47--53.

Digital Library

[42]

Zeng, K., Yang, M., Mozafari, B., and Zaniolo, C. 2013. Complex pattern matching in complex structures: The xseq approach. In Proceedings of the IEEE International Conference on Data Engineering (ICDE'13). 1328--1331.

Digital Library

Cited By

Alazeb AAlshehri MAlmakdi S(2021)Review on Data Science and Prediction2021 2nd International Conference on Computing and Data Science (CDS)10.1109/CDS52072.2021.00100(548-555)Online publication date: Jan-2021
https://doi.org/10.1109/CDS52072.2021.00100
Gillani SZimmermann APicard GLaforest F(2019)A query language for semantic complex event processingSemantic Web10.3233/SW-18031310:1(53-93)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.3233/SW-180313
Kolajo TDaramola OAdebiyi A(2019)Big data stream analysis: a systematic literature reviewJournal of Big Data10.1186/s40537-019-0210-76:1Online publication date: 6-Jun-2019
https://doi.org/10.1186/s40537-019-0210-7
Show More Cited By

Index Terms

High-performance complex event processing over hierarchical data
1. Information systems
  1. Data management systems
    1. Query languages
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query languages (principles)

Recommendations

High-performance complex event processing over XML streams
SIGMOD '12: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data

Much research attention has been given to delivering high-performance systems that are capable of complex event processing (CEP) in a wide range of applications. However, many current CEP systems focus on processing efficiently data having a simple ...
Refactoring legacy AJAX applications to improve the efficiency of the data exchange component

The AJAX paradigm encodes data exchange XML formats. Recently, JSON has also become a popular data exchange format. XML has numerous benefits including human-readable structures and self-describing data. However, JSON provides significant performance ...
Better than XML: Towards a lexicographic markup language
Abstract
This article takes a critical look at how XML is used in lexicography and asks the question, why do dictionary entries often end up looking so complex when encoded in XML? The main reason for the perceived complexity of XML-encoded dictionaries ...
Highlights
- Dictionaries encoded in XML are unnecessarily verbose and complex due to overuse of purely structural markup.
- Much data in lexicography is inherently headed, but headedness is difficult to represent in XML.
- JSON and YAML are no ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Database Systems

ACM Transactions on Database Systems Volume 38, Issue 4

Invited papers issue

November 2013

294 pages

ISSN:0362-5915

EISSN:1557-4644

DOI:10.1145/2539032

Editor:
Z. Meral Özsoyoğlu
ACM Transactions on Database Systems

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 December 2013

Accepted: 01 February 2013

Received: 01 October 2012

Published in TODS Volume 38, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Division of Information and Intelligent Systems

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
673
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alazeb AAlshehri MAlmakdi S(2021)Review on Data Science and Prediction2021 2nd International Conference on Computing and Data Science (CDS)10.1109/CDS52072.2021.00100(548-555)Online publication date: Jan-2021
https://doi.org/10.1109/CDS52072.2021.00100
Gillani SZimmermann APicard GLaforest F(2019)A query language for semantic complex event processingSemantic Web10.3233/SW-18031310:1(53-93)Online publication date: 1-Jan-2019
https://dl.acm.org/doi/10.3233/SW-180313
Kolajo TDaramola OAdebiyi A(2019)Big data stream analysis: a systematic literature reviewJournal of Big Data10.1186/s40537-019-0210-76:1Online publication date: 6-Jun-2019
https://doi.org/10.1186/s40537-019-0210-7
Chen XLi Q(2019)Event modeling and mining: a long journey toward explainable eventsThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-019-00545-029:1(459-482)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s00778-019-00545-0
Yan LLiao HSu HWei CChen XSong WGao QCarutasu G(2018)Parallel complex event detection based on regular tree pattern matchingProceedings of the 1st International Conference on Big Data Technologies10.1145/3226116.3226128(34-39)Online publication date: 18-May-2018
https://dl.acm.org/doi/10.1145/3226116.3226128
Yang JWang HCao Y(2018)Tractable queries on big data via preprocessing with logarithmic-size outputKnowledge and Information Systems10.1007/s10115-017-1092-756:1(141-163)Online publication date: 1-Jul-2018
https://dl.acm.org/doi/10.1007/s10115-017-1092-7
Giatrakos NArtikis ADeligiannakis AGarofalakis M(2017)Complex event recognition in the big data eraProceedings of the VLDB Endowment10.14778/3137765.313782910:12(1996-1999)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.14778/3137765.3137829
Saarikivi OVeanes MMytkowicz TMusuvathi M(2017)Fusing effectful comprehensionsACM SIGPLAN Notices10.1145/3140587.306236252:6(17-32)Online publication date: 14-Jun-2017
https://dl.acm.org/doi/10.1145/3140587.3062362
Artikis AMargara AUgarte MVansummeren SWeidlich M(2017)Complex Event Recognition LanguagesProceedings of the 11th ACM International Conference on Distributed and Event-based Systems10.1145/3093742.3095106(7-10)Online publication date: 8-Jun-2017
https://dl.acm.org/doi/10.1145/3093742.3095106
Saarikivi OVeanes MMytkowicz TMusuvathi MCohen AVechev M(2017)Fusing effectful comprehensionsProceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3062341.3062362(17-32)Online publication date: 14-Jun-2017
https://dl.acm.org/doi/10.1145/3062341.3062362
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents