research-article

A dynamic OSGi-based data stream system

Authors:

Bernhard Wolf,

Martin RosjatAuthors Info & Claims

MDS '08: Proceedings of the 5th Middleware doctoral symposium

Pages 31 - 36

https://doi.org/10.1145/1462728.1462734

Published: 01 December 2008 Publication History

Get Access

Abstract

Softwares for production systems are facing challenges due to rapidly changing requirements and huge amount of data from different sources. Data stream systems are utilized to process the large quantity of data using stored queries.

Since the downtime needs to be avoided, the data processing system must not be restarted. Consequently, a major challenge is the modification of running queries during the operation.

We developed a data stream processing middleware, that is capable of changing process in parallel with the runtime. The middleware applies a three step procedure to implement changes. The original elements are duplicated, modified and finally reintegrated into the running process. A special focus is on state preservation during the reintegration phase.

We evaluated our method by implementing a prototype. It provides a testbed for our future research and investigations.

References

[1]

J. Appavoo, K. Hui, C. Soules, R. Wisniewski, D. Da Silva, O. Krieger, M. Auslander, D. Edelsohn, B. Gamsa, G. Ganger, P. McKenney, M. Ostrowski, B. Rosenburg, M. Stumm, and J. Xenidis. Enabling autonomic behavior in systems software with hot swapping. IBM Systems Journal, 42(1):60--76, January 2003.

Digital Library

Google Scholar

[2]

A. Arasu, S. Babu, and J. Widom. The CQL Continuous Query Language: Semantic Foundations and Query Execution. The International Journal on Very Large Data Bases, 15(2):121--142, June 2006.

Digital Library

Google Scholar

[3]

B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and Issues in Data Stream Systems. In Proceedings of the Twenty-First ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pages 1--16, Madison, Wisconsin, USA, June 3--5 2002. ACM New York, NY, USA.

Digital Library

Google Scholar

[4]

M. Duller, R. Tamosevicius, G. Alonso, and D. Kossmann. XTream: Personal Data Streams. In Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pages 1088--1090, Beijing, China, June 12--14 2007. ACM New York, NY, USA.

Digital Library

Google Scholar

[5]

J. Gama and M. Gaber, editors. Learning from Data Streams: Processing Techniques in Sensor Networks. Springer Berlin Heidelberg, 2007.

Digital Library

Google Scholar

[6]

L. Golab and M. Özsu. Issues in Data Stream Management. SIGMOD Rec., 32(2):5--14, June 2003.

Digital Library

Google Scholar

[7]

G. Hjálmtÿsson and R. Gray. Dynamic C++ Classes: A lightweight mechanism to update code in a running program. In Proceedings of the USENIX Annual Technical Conference, pages 1--13, New Orleans, Louisiana, USA, June 15--19 1998. USENIX Association Berkeley, CA, USA.

Digital Library

Google Scholar

[8]

P. Kriens and B. Hargrave. Listeners Considered Harmful: The "Whiteboard" Pattern. Technical whitepaper, OSGi Alliance, August 2004.

Google Scholar

[9]

D. Luckham. The Power of Events: An Introduction to Complex Event Processing in Distributed Enterprise Systems. Addison-Wesley Professional, May 2002.

Digital Library

Google Scholar

[10]

R. Müller, G. Alonso, and D. Kossmann. SwissQM: Next Generation Data Processing in Sensor Networks. In Proceedings of the 3rd Biennial Conference on Innovative Data Systems Research (CIDR 2007), Asilomar, CA, USA, January 7--10 2007.

Google Scholar

[11]

The Eclipse Foundation. Equinox. http://www.eclipse.org/equinox, July 2008.

Google Scholar

[12]

The OSGi Alliance. OSGi Service Platform Core Specification, April 2007. Release 4, Version 4.1.

Google Scholar

[13]

W. White, M. Riedewald, J. Gehrke, and A. Demers. What is "Next" in Event Processing? In Proceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pages 263--272, Beijing, China, June 11--13 2007. SIGMOD: ACM Special Interest Group on Management of Data, ACM New York, NY, USA.

Digital Library

Google Scholar

[14]

B. Wolf and M. Neugebauer. Refining the Window Size of Sliding Window Operations in Running Data Processing Systems. In Proceedings of the 13th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA'08), Hamburg, Germany, September 15--18 2008.

Crossref

Google Scholar

Cited By

View all

Noack TSchmitt I(2013)Monitoring mobile cyber-physical systems by means of a knowledge discovery cycleIEEE 7th International Conference on Research Challenges in Information Science (RCIS)10.1109/RCIS.2013.6577715(1-12)Online publication date: May-2013
https://doi.org/10.1109/RCIS.2013.6577715
Wolf BBehrens I(2010)On-the-Fly Adaptation of Data Stream QueriesProceedings of the 2010 13th IEEE International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing10.1109/ISORC.2010.22(175-179)Online publication date: 5-May-2010
https://dl.acm.org/doi/10.1109/ISORC.2010.22
Weifeng HDi HJuan C(2010)An OSGi Based RFID Complex Event Processing SystemProceedings of the 2010 IEEE/IFIP International Conference on Embedded and Ubiquitous Computing10.1109/EUC.2010.32(162-169)Online publication date: 11-Dec-2010
https://dl.acm.org/doi/10.1109/EUC.2010.32

Index Terms

A dynamic OSGi-based data stream system
1. Computer systems organization
  1. Embedded and cyber-physical systems
  2. Real-time systems
2. Computing methodologies
  1. Modeling and simulation
    1. Simulation support systems
      1. Simulation environments

Recommendations

Generic windowing support for extensible stream processing systems

Stream processing applications process high volume, continuous feeds from live data sources, employ data-in-motion analytics to analyze these feeds, and produce near real-time insights with low latency. One of the fundamental characteristics of such ...
Frequency-based load shedding over a data stream of tuples

Usually the data generation rate of a data stream is unpredictable, and some data elements of the data stream cannot be processed in real time if the generation rate exceeds the capacity of a data stream processing algorithm. In order to overcome this ...
Elastic Stream Computing with Clouds
CLOUD '11: Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing

Stream computing, also known as data stream processing, has emerged as a new processing paradigm that processes incoming data streams from tremendous numbers of sensors in a real-time fashion. Data stream applications must have low latency even when the ...

Comments

Information & Contributors

Information

Published In

MDS '08: Proceedings of the 5th Middleware doctoral symposium

December 2008

42 pages

ISBN:9781605583617

DOI:10.1145/1462728

Program Chair:
Sam Michiels
K.U.Leuven, Belgium

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Middleware '08

Middleware '08: 9th International Middleware Conference

December 1, 2008

Leuven, Belgium

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
157
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Noack TSchmitt I(2013)Monitoring mobile cyber-physical systems by means of a knowledge discovery cycleIEEE 7th International Conference on Research Challenges in Information Science (RCIS)10.1109/RCIS.2013.6577715(1-12)Online publication date: May-2013
https://doi.org/10.1109/RCIS.2013.6577715
Wolf BBehrens I(2010)On-the-Fly Adaptation of Data Stream QueriesProceedings of the 2010 13th IEEE International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing10.1109/ISORC.2010.22(175-179)Online publication date: 5-May-2010
https://dl.acm.org/doi/10.1109/ISORC.2010.22
Weifeng HDi HJuan C(2010)An OSGi Based RFID Complex Event Processing SystemProceedings of the 2010 IEEE/IFIP International Conference on Embedded and Ubiquitous Computing10.1109/EUC.2010.32(162-169)Online publication date: 11-Dec-2010
https://dl.acm.org/doi/10.1109/EUC.2010.32

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Generic windowing support for extensible stream processing systems

Frequency-based load shedding over a data stream of tuples

Elastic Stream Computing with Clouds

Comments

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Generic windowing support for extensible stream processing systems

Frequency-based load shedding over a data stream of tuples

Elastic Stream Computing with Clouds

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations