short-paper

Towards a standard event processing benchmark

Authors:

Marcelo R.N. Mendes,

Pedro Bizarro,

Paulo MarquesAuthors Info & Claims

ICPE '13: Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering

Pages 307 - 310

https://doi.org/10.1145/2479871.2479913

Published: 21 April 2013 Publication History

Get Access

Abstract

There has been an increasing interest both in academia and industry for systematic methods for evaluating the performance and scalability of event processing systems. A number of performance results have been disclosed over the last years, but there is still a lack of standardized benchmarks that allow an objective comparison of the different systems. In this paper, we present our work in progress: the BiCEP benchmark suite, a set of workloads, datasets and tools for evaluating different performance aspects of event processing platforms. In particular, we introduce 'Pairs', the first of the BiCEP benchmarks, aimed at assessing the ability of CEP engines in processing progressively larger volumes of events and simultaneous queries while providing quick answers.

References

[1]

Aldridge, Irene, High-frequency trading: a practical guide to algorithmic strategies and trading. Wiley trading series. ISBN 978-0-470-56376-2.

Google Scholar

[2]

BiCEP project web site: http://bicep.dei.uc.pt

Google Scholar

[3]

Dekkers, P. Master Thesis Computer Science. Complex Event Processing. Radboud University Nijmegen, Thesis number 574, October 2007.

Google Scholar

[4]

FINCoS Framework: https://code.google.com/p/fincos/

Google Scholar

[5]

Hinze, A., Sachs, K., Buchmann, A.P. Event-based applications and enabling technologies. In Proceedings of DEBS 2009.

Digital Library

Google Scholar

[6]

Mendes, M.R.N., Bizarro, P., Marques, P. 2009. A Performance Study of Event Processing Systems. In Proceedings of the 1st TPC Technology Conference (Lyon, France, August 2009), 221--236.

Digital Library

Google Scholar

[7]

SPECjsm2007 benchmark: http://www.spec.org/jms2007/

Google Scholar

[8]

STAC Report: Aleri Order Book Consolidation on Intel Tigertown and Solaris 10. Available at: http://www.stacresearch.com/node/3844

Google Scholar

[9]

White, S., Alves, A., Rorke, D. 2008. WebLogic event server: a lightweight, modular application server for event processing. In Proceedings of DEBS 2008, 193--200.

Digital Library

Google Scholar

Cited By

View all

Karimov J(2022)Stream BenchmarksEncyclopedia of Big Data Technologies10.1007/978-3-319-63962-8_299-2(1-6)Online publication date: 24-May-2022
https://doi.org/10.1007/978-3-319-63962-8_299-2
Coenen MWagner CEchler AFrischbier S(2019)Benchmarking Financial Data Feed SystemsProceedings of the 13th ACM International Conference on Distributed and Event-based Systems10.1145/3328905.3332506(252-253)Online publication date: 24-Jun-2019
https://dl.acm.org/doi/10.1145/3328905.3332506
Giatrakos NAlevizos EArtikis ADeligiannakis AGarofalakis M(2019)Complex event recognition in the Big Data era: a surveyThe VLDB Journal10.1007/s00778-019-00557-wOnline publication date: 25-Jul-2019
https://doi.org/10.1007/s00778-019-00557-w
Show More Cited By

Index Terms

Towards a standard event processing benchmark
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
    2. Metrics

Recommendations

SPEC MPI2007—an application benchmark suite for parallel systems using MPI
International Supercomputing Conference (ISC07)

The SPEC High-Performance Group has developed the benchmark suite SPEC MPI2007 and its run rules over the last few years. The purpose of the SPEC MPI2007 benchmark and its run rules is to further the cause of fair and objective benchmarking of high-...
A scalable complex event processing system and evaluations of its performance
DEBS '12: Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems

This paper describes a scalable context delivery platform (SCTXPF) and our evaluations of it. The SCTXPF receives a large number of events from various event sources and a large number of complex event processing (CEP) rules from various services/...
A complex event processing for large-scale M2M services and its performance evaluations
DEBS '15: Proceedings of the 9th ACM International Conference on Distributed Event-Based Systems

This paper describes a scalable complex event processing (CEP) system and rule allocation algorithm for large-scale M2M services and presents the results of its performance evaluations. Platforms providing M2M services deal with various data from ...

Comments

Information & Contributors

Information

Published In

ICPE '13: Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering

April 2013

446 pages

ISBN:9781450316361

DOI:10.1145/2479871

Editor:
Seetharami Seelam
IBM T.J. Watson Research Center, USA
,
General Chairs:
Petr Tůma
Charles University, Czech Republic
,
Giuliano Casale
Imperial College London, UK
,
Program Chairs:
Tony Field
Imperial College London, UK
,
José Nelson Amaral
University of Alberta, Canada

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICPE'13

Sponsor:

ICPE'13: ACM/SPEC International Conference on Performance Engineering

April 21 - 24, 2013

Prague, Czech Republic

Acceptance Rates

ICPE '13 Paper Acceptance Rate 28 of 64 submissions, 44%;

Overall Acceptance Rate 252 of 851 submissions, 30%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
226
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Karimov J(2022)Stream BenchmarksEncyclopedia of Big Data Technologies10.1007/978-3-319-63962-8_299-2(1-6)Online publication date: 24-May-2022
https://doi.org/10.1007/978-3-319-63962-8_299-2
Coenen MWagner CEchler AFrischbier S(2019)Benchmarking Financial Data Feed SystemsProceedings of the 13th ACM International Conference on Distributed and Event-based Systems10.1145/3328905.3332506(252-253)Online publication date: 24-Jun-2019
https://dl.acm.org/doi/10.1145/3328905.3332506
Giatrakos NAlevizos EArtikis ADeligiannakis AGarofalakis M(2019)Complex event recognition in the Big Data era: a surveyThe VLDB Journal10.1007/s00778-019-00557-wOnline publication date: 25-Jul-2019
https://doi.org/10.1007/s00778-019-00557-w
Karimov J(2019)Stream BenchmarksEncyclopedia of Big Data Technologies10.1007/978-3-319-77525-8_299(1595-1600)Online publication date: 20-Feb-2019
https://doi.org/10.1007/978-3-319-77525-8_299
Sampaio Gradvohl A(2018)Metrics and Tool for Evaluating Data Stream Processing Systems2018 6th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW)10.1109/W-FiCloud.2018.00014(48-55)Online publication date: Aug-2018
https://doi.org/10.1109/W-FiCloud.2018.00014
Karimov J(2018)Stream BenchmarksEncyclopedia of Big Data Technologies10.1007/978-3-319-63962-8_299-1(1-6)Online publication date: 27-Mar-2018
https://doi.org/10.1007/978-3-319-63962-8_299-1
Alevizos ESkarlatidis AArtikis APaliouras G(2017)Probabilistic Complex Event RecognitionACM Computing Surveys10.1145/311780950:5(1-31)Online publication date: 26-Sep-2017
https://dl.acm.org/doi/10.1145/3117809
Dayarathna MLi YWen YFan R(2017)Energy consumption analysis of data stream processingSoftware—Practice & Experience10.1002/spe.245847:10(1443-1462)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1002/spe.2458
Ritter DMay NSachs KRinderle-Ma SGal AWeidlich MKalogeraki VVenkasubramanian N(2016)Benchmarking integration pattern implementationsProceedings of the 10th ACM International Conference on Distributed and Event-based Systems10.1145/2933267.2933269(125-136)Online publication date: 13-Jun-2016
https://dl.acm.org/doi/10.1145/2933267.2933269
Gradvohl A(2016)Investigating Metrics to Build a Benchmark Tool for Complex Event Processing Systems2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW)10.1109/W-FiCloud.2016.40(143-147)Online publication date: Aug-2016
https://doi.org/10.1109/W-FiCloud.2016.40
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

SPEC MPI2007—an application benchmark suite for parallel systems using MPI

A scalable complex event processing system and evaluations of its performance

A complex event processing for large-scale M2M services and its performance evaluations