survey

Runtime Adaptation of Data Stream Processing Systems: The State of the Art

Authors:

Valeria Cardellini,

Francesco Lo Presti,

Matteo Nardelli,

Gabriele Russo RussoAuthors Info & Claims

ACM Computing Surveys, Volume 54, Issue 11s

Article No.: 237, Pages 1 - 36

https://doi.org/10.1145/3514496

Published: 09 September 2022 Publication History

Abstract

Data stream processing (DSP) has emerged over the years as the reference paradigm for the analysis of continuous and fast information flows, which often have to be processed with low-latency requirements to extract insights and knowledge from raw data. Dealing with unbounded dataflows, DSP applications are typically long running and thus, likely experience varying workloads and working conditions over time. To keep a consistent service level in face of such variability, a lot of effort has been spent studying strategies for runtime adaptation of DSP systems and applications. In this survey, we review the most relevant approaches from the literature, presenting a taxonomy to characterize the state of the art along several key dimensions. Our analysis allows us to identify current research trends as well as open challenges that will motivate further investigations in this field.

Supplementary Material

3514496.app (3514496.app.pdf)

Supplementary appendix

Download
947.88 KB

References

[1]

Daniel J. Abadi, Yanif Ahmad, Magdalena Balazinska, Ugur Çetintemel, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag S. Maskey, et al. 2005. The design of the Borealis stream processing engine. In Proc. of CIDR’05. 277–289.

[2]

Daniel J. Abadi, Don Carney, Ugur Çetintemel, Mitch Cherniack, Christian Convey, Sangdon Lee, Michael Stonebraker, Nesime Tatbul, and Stan Zdonik. 2003. Aurora: A new model and architecture for data stream management. VLDB J. 12, 2 (2003), 120–139.

Digital Library

[3]

Ahmed S. Abdelhamid, Ahmed R. Mahmood, Anas Daghistani, and Walid G. Aref. 2020. Prompt: Dynamic data-partitioning for distributed micro-batch stream processing systems. In Proc. of ACM SIGMOD’20. ACM, New York, NY, 2455–2469.

[4]

Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael Fernández-Moctezuma, Reuven Lax, Sam McVeety, et al. 2015. The dataflow model: A practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing. Proc. VLDB Endow. 8, 12 (2015), 1792–1803.

Digital Library

[5]

Walid A. Y. Aljoby, Xin Wang, Tom Z. J. Fu, and Richard T. B. Ma. 2019. On SDN-enabled online and dynamic bandwidth allocation for stream analytics. IEEE J. Sel. Areas Commun. 37, 8 (2019), 1688–1702.

Digital Library

[6]

Lisa Amini, Navendu Jain, Anshul Sehgal, Jeremy Silber, and Olivier Verscheure. 2006. Adaptive control of extreme-scale stream processing systems. In Proc. of IEEE ICDCS’06.

[7]

Leonardo Aniello, Roberto Baldoni, and Leonardo Querzoni. 2013. Adaptive online scheduling in storm. In Proc. of ACM DEBS’13. 207–218.

[8]

Atakan Aral, Melike Erol-Kantarci, and Ivona Brandic. 2020. Staleness control for edge data analytics. Proc. ACM Meas. Anal. Comput. Syst. 4, 2 (2020), Article 38, 24 pages.

Digital Library

[9]

Marcos D. de Assunção, Alexandre da Silva Veith, and Rajkumar Buyya. 2018. Distributed data stream processing and edge computing: A survey on resource elasticity and future directions. J. Netw. Comput. Appl. 103 (2018), 1–17.

Digital Library

[10]

Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. 2002. Models and issues in data stream systems. In Proc. of ACM PODS’02. 1–16.

[11]

Brian Babcock, Mayur Datar, and Rajeev Motwani. 2004. Load shedding for aggregation queries over data streams. In Proc. of ICDE’04. IEEE, Los Alamitos, CA, 350–361.

[12]

Magdalena Balazinska, Hari Balakrishnan, Samuel Madden, and Michael Stonebraker. 2008. Fault-tolerance in the Borealis distributed stream processing system. ACM Trans. Database Syst. 33, 1 (2008), Article 3, 44 pages.

Digital Library

[13]

Magdalena Balazinska, Hari Balakrishnan, and Mike Stonebraker. 2004. Contract-based load management in federated distributed systems. In Proc. of USENIX NSDI’04.

[14]

Cagri Balkesen, Nesime Tatbul, and M. Tamer Özsu. 2013. Adaptive input admission and management for parallel stream processing. In Proc. of ACM DEBS’13. 15–26.

[15]

Edmon Begoli, Tyler Akidau, Slava Chernyak, Fabian Hueske, Kathryn Knight, Kenneth Knowles, Daniel Mills, and Dan Sotolongo. 2021. Watermarks in stream processing systems: Semantics and comparative analysis of Apache Flink and Google Cloud dataflow. Proc. VLDB Endow. 14, 12 (2021), 3135–3147.

Digital Library

[16]

Mehdi M. Belkhiria, Marin Bertier, and Cédric Tedeschi. 2020. Group mutual exclusion to scale distributed stream processing pipelines. In Proc. of IEEE/ACM UCC’20. 247–256.

[17]

Paolo Bellavista, Antonio Corradi, Spyros Kotoulas, and Andrea Reale. 2014a. Adaptive fault-tolerance for dynamic resource provisioning in distributed stream processing systems. In Proc. of EDBT’14. 85–96.

[18]

Paolo Bellavista, Antonio Corradi, Andrea Reale, and Nicola Ticca. 2014b. Priority-based resource scheduling in distributed stream processing systems for big data applications. In Proc. of IEEE/ACM UCC’14. 363–370.

[19]

Gunter Bolch, Stefan Greiner, Hermann de Meer, and Kishor S. Trivedi. 2006. Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications (2nd ed.). Wiley.

[20]

Michael Borkowski, Christoph Hochreiner, and Stefan Schulte. 2019. Minimizing cost by reducing scaling operations in distributed stream processing. Proc. VLDB Endow. 12, 7 (2019), 724–737.

Digital Library

[21]

Thilina Buddhika, Ryan Stern, Kira Lindburg, Kathleen Ericson, and Shrideep Pallickara. 2017. Online scheduling and interference alleviation for low-latency, high-throughput processing of data streams. IEEE Trans. Parallel Distrib. Syst. 28, 12 (2017), 3553–3569.

Digital Library

[22]

Michael Cammert, Jurgen Kramer, Bernhard Seeger, and Sonny Vaupel. 2008. A cost-based approach to adaptive resource management in data stream systems. IEEE Trans. Knowl. Data Eng. 20, 2 (2008), 230–245.

Digital Library

[23]

Matthieu Caneill, Ahmed El-Rheddane, Vincent Leroy, and Noël De Palma. 2016. Locality-aware routing in stateful streaming applications. In Proc. of ACM/IFIP/USENIX MIDDLEWARE’16. ACM, New York, NY, Article 4, 13 pages.

[24]

Paris Carbone, Stephan Ewen, Gyula Fóra, Seif Haridi, Stefan Richter, and Kostas Tzoumas. 2017. State management in Apache Flink®: Consistent stateful distributed stream processing. Proc. VLDB Endow. 10, 12 (2017), 1718–1729.

Digital Library

[25]

Valeria Cardellini, Francesco Lo Presti, Matteo Nardelli, and Gabriele Russo Russo. 2018b. Decentralized self-adaptation for elastic data stream processing. Future Gener. Comput. Syst. 87 (2018), 171–185.

Digital Library

[26]

Valeria Cardellini, Francesco Lo Presti, Matteo Nardelli, and Gabriele Russo Russo. 2018a. Optimal operator deployment and replication for elastic distributed data stream processing. Concurr. Comp. Pract. Exp. 30, 9 (2018).

[27]

Valeria Cardellini, Matteo Nardelli, and Dario Luzi. 2016. Elastic stateful stream processing in storm. In Proc. of HPCS’16. IEEE, Los Alamitos, CA, 583–590.

[28]

Barbara Carminati, Elena Ferrari, Jianneng Cao, and Kian Lee Tan. 2010. A framework to enforce access control over data streams. ACM Trans. Inf. Syst. Secur. 13, 3 (2010), Article 28, 31 pages.

Digital Library

[29]

Paul Castro, Vatche Ishakian, Vinod Muthusamy, and Aleksander Slominski. 2019. The rise of serverless computing. Commun. ACM 62, 12 (2019), 44–54.

Digital Library

[30]

Javier Cerviño, Evangelia Kalyvianaki, Joaquín Salvachúa, and Peter R. Pietzuch. 2012. Adaptive provisioning of stream processing systems in the cloud. In Proc. of IEEE ICDE’12. 295–301.

[31]

Sirish Chandrasekaran, Owen Cooper, Amol Deshpande, Michael J. Franklin, Joseph M. Hellerstein, Wei Hong, Sailesh Krishnamurthy, Samuel R. Madden, and Fred Reiss. 2003. TelegraphCQ: Continuous dataflow processing. In Proc. of ACM SIGMOD’03. 668.

[32]

Mengyuan Chao and Radu Stoleru. 2020. R-MStorm: A resilient mobile stream processing system for dynamic edge networks. In Proc. of IEEE ICFC’20. 64–72.

[33]

Mengyuan Chao, Chen Yang, Yukun Zeng, and Radu Stoleru. 2018. F-MStorm: Feedback-based online distributed mobile stream processing. In Proc. of IEEE/ACM SEC’18. 273–285.

[34]

Shilpa Chaturvedi and Yogesh Simmhan. 2019. Toward resilient stream processing on clouds using moving target defense. In Proc. of IEEE ISORC’19. 134–142.

[35]

Shilpa Chaturvedi, Sahil Tyagi, and Yogesh Simmhan. 2021. Cost-effective sharing of streaming dataflows for IoT applications. IEEE Trans. Cloud Comput. 9, 4 (2021), 1391–1407.

[36]

Andreas Chatzistergiou and Stratis D. Viglas. 2014. Fast heuristics for near-optimal task allocation in data stream processing over clusters. In Proc. of ACM CIKM’14. 1579–1588.

[37]

Xin Chen, Ymir Vigfusson, Douglas M. Blough, Fang Zheng, Kun-Lung Wu, and Liting Hu. 2017. GOVERNOR: Smoother stream processing through smarter backpressure. In Proc. of IEEE ICAC’17. 145–154.

[38]

Dazhao Cheng, Xiaobo Zhou, Yu Wang, and Changjun Jiang. 2018. Adaptive scheduling parallel jobs with dynamic batching in spark streaming. IEEE Trans. Parallel Distrib. Syst. 29, 12 (2018), 2672–2685.

[39]

Gianpaolo Cugola and Alessandro Margara. 2012. Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44, 3 (2012), Article 15, 62 pages.

Digital Library

[40]

Gianpaolo Cugola and Alessandro Margara. 2013. Deployment strategies for distributed complex event processing. Computing 95, 2 (2013), 129–156.

Digital Library

[41]

Tathagata Das, Yuan Zhong, Ion Stoica, and Scott Shenker. 2014. Adaptive stream processing using dynamic batch sizing. In Proc. of ACM SoCC’14. Article 16, 13 pages.

[42]

Miyuru Dayarathna and Srinath Perera. 2018. Recent advancements in event processing. ACM Comput. Surv. 51, 2 (2018), Article 33, 36 pages.

[43]

Tiziano De Matteis and Gabriele Mencagli. 2017b. Elastic scaling for distributed latency-sensitive data stream operators. In Proc. of PDP’17. IEEE, Los Alamitos, CA, 61–68.

[44]

Tiziano De Matteis and Gabriele Mencagli. 2017a. Proactive elasticity and energy awareness in data stream processing. J. Syst. Softw. 127 (2017), 302–319.

Digital Library

[45]

Felipe R. de Souza, Alexandre da Silva Veith, Marcos D. de Assunção, and Eddy Caron. 2020. Scalable joint optimization of placement and parallelism of data stream processing applications on cloud-edge infrastructure. In Service-Oriented Computing. Lecture Notes in Computer Science, Vol. 12571. Springer, 149–164.

[46]

Guangxiang Du and Indranil Gupta. 2016. New techniques to curtail the tail latency in stream processing systems. In Proc. of DCC@PODC’16. ACM, New York, NY, Article 7, 6 pages.

[47]

Christopher Eibel, Christian Gulden, Wolfgang Schröder-Preikschat, and Tobias Distler. 2018. Strome: Energy-aware data-stream processing. In Distributed Applications and Interoperable Systems. Lecture Notes in Computer Science, Vol. 10853. Springer, 40–57.

[48]

Leila Eskandari, Zhiyi Huang, and David M. Eyers. 2016. P-scheduler: Adaptive hierarchical scheduling in Apache Storm. In Proc. of ACSW’16.ACM, New York, NY, Article 26, 10 pages.

[49]

Junhua Fang, Pingfu Chao, Rong Zhang, and Xiaofang Zhou. 2019. Integrating workload balancing and fault tolerance in distributed stream processing system. World Wide Web 22, 6 (2019), 2471–2496.

[50]

Junhua Fang, Rong Zhang, Tom Z. J. Fu, Zhenjie Zhang, Aoying Zhou, and Xiaofang Zhou. 2018. Distributed stream rebalance for stateful operator under workload variance. IEEE Trans. Parallel Distrib. Syst. 29, 10 (2018), 2223–2240.

[51]

Omar Farhat, Khuzaima Daudjee, and Leonardo Querzoni. 2021. Klink: Progress-aware scheduling for streaming data systems. In Proc. of ACM SIGMOD’21. 485–498.

[52]

Raul Castro Fernandez, Matteo Migliavacca, Evangelia Kalyvianaki, and Peter R. Pietzuch. 2013. Integrating scale out and fault tolerance in stream processing using operator state management. In Proc. of ACM SIGMOD’13. 725–736.

[53]

Avrilia Floratou, Ashvin Agrawal, Bill Graham, Sriram Rao, and Karthik Ramasamy. 2017. Dhalion: Self-regulating stream processing in Heron. Proc. VLDB Endow. 10, 12 (2017), 1825–1836.

Digital Library

[54]

Marios Fragkoulis, Paris Carbone, Vasiliki Kalavri, and Asterios Katsifodimos. 2020. A survey on the evolution of stream processing systems. CoRR abs/2008.00842 (2020).

[55]

Tom Z. J. Fu, Jianbing Ding, Richard T. B. Ma, Marianne Winslett, Yin Yang, and Zhenjie Zhang. 2017. DRS: Auto-scaling for real-time stream analytics. IEEE/ACM Trans. Netw. 25, 6 (2017), 3338–3352.

Digital Library

[56]

Xinwei Fu, Talha Ghaffar, James C. Davis, and Dongyoon Lee. 2019. EdgeWise: A better stream processing engine for the edge. In Proc. of USENIX ATC’19. 929–946.

[57]

Bugra Gedik, Scott Schneider, Martin Hirzel, and Kun-Lung Wu. 2014. Elastic scaling for data stream processing. IEEE Trans. Parallel Distrib. Syst. 25, 6 (2014), 1447–1463.

Digital Library

[58]

Lukasz Golab and M. Tamer Özsu. 2003. Issues in data stream management. ACM SIGMOD Rec. 32, 2 (2003), 5–14.

Digital Library

[59]

Xiaohui Gu, Philip S. Yu, and Klara Nahrstedt. 2005. Optimal component composition for scalable stream processing. In Proc. of IEEE ICDCS’05. 773–782.

[60]

Vincenzo Gulisano, Ricardo Jiménez-Peris, Marta Patiño-Martinez, Claudio Soriente, and Patrick Valduriez. 2012. StreamCloud: An elastic and scalable data streaming system. IEEE Trans. Parallel Distrib. Syst. 23, 12 (2012), 2351–2365.

Digital Library

[61]

Vincenzo Gulisano, Marina Papatriantafilou, and Alessandro Vittorio Papadopoulos. 2019. Elasticity. In Encyclopedia of Big Data Technologies. Springer.

[62]

Qingsong Guo and Yongluan Zhou. 2017a. CBP: A new parallelization paradigm for massively distributed stream processing. In Database Systems for Advanced Applications. Lecture Notes in Computer Science, Vol. 10178. Springer, 304–320.

[63]

Qingsong Guo and Yongluan Zhou. 2017b. Stateful load balancing for parallel stream processing. In Euro-Par 2017: Parallel Processing Workshops. Lecture Notes in Computer Science, Vol. 10659. Springer, 80–93.

[64]

Zheng Han, Rui Chu, Haibo Mi, and Huaimin Wang. 2014. Elastic allocator: An adaptive task scheduler for streaming query in the cloud. In Proc. of IEEE SOSE’14. 284–289.

[65]

Aurélien Havet, Rafael Pires, Pascal Felber, Marcelo Pasin, Romain Rouvoy, and Valerio Schiavoni. 2017. SecureStreams: A reactive middleware framework for secure data stream processing. In Proc. of ACM DEBS’17. 124–133.

[66]

Benjamin Heintz, Abhishek Chandra, and Ramesh K. Sitaraman. 2020. Optimizing timeliness and cost in geo-distributed streaming analytics. IEEE Trans. Cloud Comput. 8, 1 (2020), 232–245.

[67]

Thomas Heinze, Zbigniew Jerzak, Gregor Hackenbroich, and Christof Fetzer. 2014a. Latency-aware elastic scaling for distributed data stream processing systems. In Proc. of ACM DEBS’14. 13–22.

[68]

Thomas Heinze, Valerio Pappalardo, Zbigniew Jerzak, and Christof Fetzer. 2014b. Auto-scaling techniques for elastic data stream processing. In Proc. of IEEE ICDEW’14. 296–302.

[69]

Thomas Heinze, Lars Roediger, Andreas Meister, Yuanzhen Ji, Zbigniew Jerzak, and Christof Fetzer. 2015a. Online parameter optimization for elastic data stream processing. In Proc. of ACM SoCC’15. 276–287.

[70]

Thomas Heinze, Mariam Zia, Robert Krahn, Zbigniew Jerzak, and Christof Fetzer. 2015b. An adaptive replication scheme for elastic data stream processing systems. In Proc. of ACM DEBS’15. 150–161.

[71]

Herodotos Herodotou, Yuxing Chen, and Jiaheng Lu. 2020. A survey on automatic parameter tuning for big data processing systems. ACM Comput. Surv. 53, 2 (2020), Article 43, 37 pages.

[72]

Nicolas Hidalgo, Daniel Wladdimiro, and Erika Rosas. 2017. Self-adaptive processing graph with operator fission for elastic stream processing. J. Syst. Softw. 127 (2017), 205–216.

Digital Library

[73]

Martin Hirzel, Robert Soulé, Scott Schneider, Bugra Gedik, and Robert Grimm. 2013. A catalog of stream processing optimizations. ACM Comput. Surv. 46, 4 (2013), Article 46, 34 pages.

[74]

Christoph Hochreiner, Michael Vögler, Stefan Schulte, and Schahram Dustdar. 2016. Elastic stream processing for the Internet of Things. In Proc. of IEEE CLOUD’16. 100–107.

[75]

Moritz Hoffmann, Andrea Lattuada, Frank McSherry, Vasiliki Kalavri, John Liagouris, and Timothy Roscoe. 2019. Megaphone: Latency-conscious state migration for distributed streaming dataflows. Proc. VLDB Endow. 12, 9 (2019), 1002–1015.

Digital Library

[76]

Mohammad R. Hoseiny Farahabady, Ali Jannesari, Javid Taheri, Wei Bao, Albert Y. Zomaya, and Zahir Tari. 2020. Q-Flink: A QoS-aware controller for Apache Flink. In Proc. of IEEE/ACM CCGRID’20. 629–638.

[77]

Mohammad R. Hoseiny Farahabady, Hamid R. Dehghani Samani, Yidan Wang, Albert Y. Zomaya, and Zahir Tari. 2016. A QoS-aware controller for Apache storm. In Proc. of IEEE NCA’16. 334–342.

[78]

Mohammad R. Hoseiny Farahabady, Albert Y. Zomaya, and Zahir Tari. 2017. QoS- and contention- aware resource provisioning in a stream processing engine. In Proc. of IEEE CLUSTER’17. 137–146.

[79]

Qun Huang and Patrick P. C. Lee. 2016. Toward high-performance distributed stream processing via approximate fault tolerance. Proc. VLDB Endow. 10, 3 (2016), 73–84.

Digital Library

[80]

Xi Huang, Ziyu Shao, and Yang Yang. 2020. POTUS: Predictive online tuple scheduling for data stream processing systems. IEEE Trans. Cloud Comput.To appear.

[81]

Jeong-Hyon Hwang, Ugur Çetintemel, and Stan Zdonik. 2008. Fast and highly-available stream processing over wide area networks. In Proc. of IEEE ICDE’08. 804–813.

[82]

Shigeru Imai, Stacy Patterson, and Carlos A. Varela. 2018. Uncertainty-aware elastic virtual machine scheduling for stream processing systems. In Proc. of IEEE/ACM CCGRID’18. 62–71.

[83]

Changjiang Jia, Yan Cai, Yuen-Tak Yu, and T. H. Tse. 2016. 5W+1H pattern: A perspective of systematic mapping studies and a case study on cloud software testing. J. Syst. Softw. 116 (2016), 206–219.

Digital Library

[84]

Aymen Jlassi and Cédric Tedeschi. 2020. Merge, split, and cluster: Dynamic deployment of stream processing applications. In Proc. of IEEE/ACM CCGRID’20. 71–80.

[85]

Albert Jonathan, Abhishek Chandra, and Jon B. Weissman. 2020. WASP: Wide-area adaptive stream processing. In Proc. of ACM/IFIP MIDDLEWARE’20. ACM, New York, NY, 221–235.

[86]

Basri Kahveci and Bugra Gedik. 2020. Joker: Elastic stream processing with organic adaptation. J. Parallel Distrib. Comput. 137 (2020), 205–223.

Digital Library

[87]

Vasiliki Kalavri, John Liagouris, Moritz Hoffmann, Desislava C. Dimitrova, Matthew Forshaw, and Timothy Roscoe. 2018. Three steps is all you need: Fast, accurate, automatic scaling decisions for distributed streaming dataflows. In Proc. of USENIX OSDI’18. 783–798.

[88]

Faria Kalim, Le Xu, Sharanya Bathey, Richa Meherwal, and Indranil Gupta. 2018. Henge: Intent-driven multi-tenant stream processing. In Proc. of ACM SoCC’18. 249–262.

[89]

Evangelia Kalyvianaki, Themistoklis Charalambous, Marco Fiscato, and Peter Pietzuch. 2012. Overload management in data stream processing systems with latency guarantees. In Proc. of FCW’12.

[90]

Evangelia Kalyvianaki, Marco Fiscato, Theodoros Salonidis, and Peter R. Pietzuch. 2016. THEMIS: Fairness in federated stream processing under overload. In Proc. of ACM SIGMOD’16. 541–553.

[91]

Evangelia Kalyvianaki, Wolfram Wiesemann, Quang H. Vu, Daniel Kuhn, and Peter R. Pietzuch. 2011. SQPR: Stream query planning with reuse. In Proc. of IEEE ICDE’11. 840–851.

[92]

Nikos R. Katsipoulakis, Alexandros Labrinidis, and Panos K. Chrysanthis. 2017. A holistic view of stream partitioning costs. Proc. VLDB Endow. 10, 11 (2017), 1286–1297.

Digital Library

[93]

Nikos R. Katsipoulakis, Alexandros Labrinidis, and Panos K. Chrysanthis. 2018. Concept-driven load shedding: Reducing size and error of voluminous and variable data streams. In Proc. of IEEE Big Data’18. 418–427.

[94]

Nikos R. Katsipoulakis, Alexandros Labrinidis, and Panos K. Chrysanthis. 2020. SPEAr: Expediting stream processing with accuracy guarantees. In Proc. of IEEE ICDE’20. 1105–1116.

[95]

Wilhelm Kleiminger, Evangelia Kalyvianaki, and Peter R. Pietzuch. 2011. Balancing load in stream processing with the cloud. In Proc. of IEEE ICDE’11. 16–21.

[96]

Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. 2018. Pocket: Elastic ephemeral storage for serverless analytics. In Proc. of USENIX OSDI’18. 427–444.

[97]

Alexandros Koliousis, Matthias Weidlich, Raul Castro Fernandez, Alexander L. Wolf, Paolo Costa, and Peter R. Pietzuch. 2016. SABER: Window-based hybrid stream processing for heterogeneous architectures. In Proc. of ACM SIGMOD’16. 555–569.

[98]

Roland Kotto Kombi, Nicolas Lumineau, and Philippe Lamarre. 2017. A preventive auto-parallelization approach for elastic stream processing. In Proc. of IEEE ICDCS’17. 1532–1542.

[99]

Alok G. Kumbhare, Yogesh Simmhan, and Viktor K. Prasanna. 2014. PLAStiCC: Predictive look-ahead scheduling for continuous dataflows on clouds. In Proc. of IEEE/ACM CCGrid’14. 344–353.

[100]

Alok G. Kumbhare, Yogesh L. Simmhan, Marc Frincu, and Viktor K. Prasanna. 2015. Reactive resource provisioning heuristics for dynamic dataflows on cloud infrastructure. IEEE Trans. Cloud Comput. 3, 2 (2015), 105–118.

[101]

Geetika T. Lakshmanan, Ying Li, and Robert E. Strom. 2008. Placement strategies for internet-scale data stream systems. IEEE Internet Comput. 12, 6 (2008), 50–60.

Digital Library

[102]

Geetika T. Lakshmanan and Robert E. Strom. 2008. Biologically-inspired distributed middleware management for stream processing systems. In Middleware 2008.Lecture Notes in Computer Science, Vol. 5346. Springer, 223–242.

[103]

Do Le Quoc, Martin Beck, Pramod Bhatotia, Ruichuan Chen, Christof Fetzer, and Thorsten Strufe. 2017a. PrivApprox: Privacy-preserving stream analytics. In Proc. of USENIX ATC’17. 659–672.

[104]

Do Le Quoc, Ruichuan Chen, Pramod Bhatotia, Christof Fetzer, Volker Hilt, and Thorsten Strufe. 2017b. StreamApprox: Approximate computing for stream analytics. In Proc. of ACM/IFIP/USENIX MIDDLEWARE’17. ACM, New York, NY, 185–197.

[105]

Chuan Lei and Elke A. Rundensteiner. 2014. Robust distributed query processing for streaming data. ACM Trans. Database Syst. 39, 2 (2014), Article 17, 45 pages.

Digital Library

[106]

Jack Li, Calton Pu, Yuan Chen, Daniel Gmach, and Dejan S. Milojicic. 2016. Enabling elastic stream processing in shared clusters. In Proc. of IEEE CLOUD’16. 108–115.

[107]

Kejian Li, Gang Liu, and Minhua Lu. 2019. A holistic stream partitioning algorithm for distributed stream processing systems. In Proc. of PDCAT’19. IEEE, Los Alamitos, CA, 202–207.

[108]

Teng Li, Zhiyuan Xu, Jian Tang, and Yanzhi Wang. 2018. Model-free control for distributed stream data processing using deep reinforcement learning. Proc. VLDB Endow. 11, 6 (2018), 705–718.

Digital Library

[109]

Xiaofei Liao, Yu Huang, Long Zheng, and Hai Jin. 2019. Efficient time-evolving stream processing at scale. IEEE Trans. Parallel Distrib. Syst. 30, 10 (2019), 2165–2178.

[110]

Xunyun Liu and Rajkumar Buyya. 2017. D-storm: Dynamic resource-efficient scheduling of stream processing applications. In Proc. of ICPADS’17. 485–492.

[111]

Xunyun Liu and Rajkumar Buyya. 2020. Resource management and scheduling in distributed stream processing systems: A taxonomy, review, and future directions. ACM Comput. Surv. 53, 3 (2020), Article 50, 41 pages.

[112]

Xunyun Liu, Amir Vahid Dastjerdi, Rodrigo N. Calheiros, Chenhao Qu, and Rajkumar Buyya. 2018. A stepwise auto-profiling method for performance optimization of streaming applications. ACM Trans. Auton. Adapt. Syst. 12, 4 (2018), Article 24, 33 pages.

Digital Library

[113]

Björn Lohrmann, Peter Janacik, and Odej Kao. 2015. Elastic stream processing with latency guarantees. In Proc. of IEEE ICDCS’15. 399–410.

[114]

Björn Lohrmann, Daniel Warneke, and Odej Kao. 2014. Nephele streaming: Stream processing under QoS constraints at scale. Clust. Comput. 17, 1 (2014), 61–78.

Digital Library

[115]

Federico Lombardi, Leonardo Aniello, Silvia Bonomi, and Leonardo Querzoni. 2018. Elastic symbiotic scaling of operators and resources in stream processing systems. IEEE Trans. Parallel Distrib. Syst. 29, 3 (2018), 572–585.

[116]

Manisha Luthra, Boris Koldehofe, Pascal Weisenburger, Guido Salvaneschi, and Raheel Arif. 2018. TCEP: Adapting to dynamic user environments by enabling transitions between operator placement mechanisms. In Proc. of ACM DEBS’18. 136–147.

[117]

Kasper Madsen, Yongluan Zhou, and Jianneng Cao. 2017. Integrative dynamic reconfiguration in a parallel stream processing engine. In Proc. of IEEE ICDE’17. 227–230.

[118]

Kasper Madsen, Yongluan Zhou, and Li Su. 2016. Enorm: Efficient window-based computation in large-scale distributed stream processing systems. In Proc. of ACM DEBS’16. 37–48.

[119]

Luo Mai, Kai Zeng, Rahul Potharaju, Le Xu, Steve Suh, Shivaram Venkataraman, Paolo Costa, et al. 2018. Chi: A scalable and programmable control plane for distributed stream processing systems. Proc. VLDB Endow. 11, 10 (2018), 1303–1316.

Digital Library

[120]

Vania Marangozova-Martin, Noël De Palma, and Ahmed El-Rheddane. 2019. Multi-level elasticity for data stream processing. IEEE Trans. Parallel Distrib. Syst. 30, 10 (2019), 2326–2337.

[121]

Yuan Mei, Luwei Cheng, Vanish Talwar, Michael Y. Levin, Gabriela Jacques-Silva, Nikhil Simha, Anirban Banerjee, et al. 2020. Turbine: Facebook’s service management platform for stream processing. In Proc. of IEEE ICDE’20. 1591–1602.

[122]

Gabriele Mencagli. 2016. A game-theoretic approach for elastic distributed data stream processing. ACM Trans. Auton. Adapt. Syst. 11, 2 (2016), Article 13, 34 pages.

Digital Library

[123]

Gabriele Mencagli, Massimo Torquati, and Marco Danelutto. 2018. Elastic-PPQ: A two-level autonomic system for spatial preference query processing over dynamic data streams. Future Gener. Comput. Syst. 79 (2018), 862–877.

[124]

Gabriele Mencagli, Massimo Torquati, Marco Danelutto, and Tiziano De Matteis. 2017. Parallel continuous preference queries over out-of-order and bursty data streams. IEEE Trans. Parallel Distrib. Syst. 28, 9 (2017), 2608–2624.

Digital Library

[125]

Bonaventura Del Monte, Steffen Zeuch, Tilmann Rabl, and Volker Markl. 2020. Rhino: Efficient management of very large distributed state for stream processing engines. In Proc. of ACM SIGMOD’20. ACM, New York, NY, 2471–2486.

[126]

Weimin Mu, Zongze Jin, Junwei Wang, Weilin Zhu, and Weiping Wang. 2019. BGElasor: Elastic-scaling framework for distributed streaming processing with deep neural network. In Network and Parallel Computing. Lecture Notes in Computer Science, Vol. 11783. Springer, 120–131.

[127]

Hannaneh Najdataei, Yiannis Nikolakopoulos, Marina Papatriantafilou, Philippas Tsigas, and Vincenzo Gulisano. 2019. STRETCH: Scalable and elastic deterministic streaming analysis with virtual shared-nothing parallelism. In Proc. of ACM DEBS’19. 7–18.

[128]

Matteo Nardelli, Valeria Cardellini, Vincenzo Grassi, and Francesco Lo Presti. 2019. Efficient operator placement for distributed data stream processing applications. IEEE Trans. Parallel Distrib. Syst. 30, 8 (2019), 1753–1767.

[129]

Stefan Nastic, Thomas Rausch, Ognjen Scekic, Schahram Dustdar, Marjan Gusev, Bojana Koteska, Magdalena Kostoska, Boro Jakimovski, Sasko Ristov, and Radu Prodan. 2017. A serverless real-time data analytics platform for edge computing. IEEE Internet Comput. 21, 4 (2017), 64–71.

Digital Library

[130]

Xiang Ni, Scott Schneider, Raju Pavuluri, Jonathan Kaus, and Kun-Lung Wu. 2019. Automating multi-level performance elastic components for IBM streams. In Proc. of ACM/IFIP Middleware’19. ACM, New York, NY, 163–175.

[131]

Dan O’Keeffe, Theodoros Salonidis, and Peter R. Pietzuch. 2018. Frontier: Resilient edge processing for the Internet of Things. Proc. VLDB Endow. 11, 10 (2018), 1178–1191.

Digital Library

[132]

Beate Ottenwälder, Boris Koldehofe, Kurt Rothermel, Kirak Hong, David J. Lillethun, and Umakishore Ramachandran. 2014. MCEP: A mobility-aware complex event processing system. ACM Trans. Internet Technol. 14, 1 (2014), Article 6, 24 pages.

Digital Library

[133]

Dimitris Palyvos-Giannas, Gabriele Mencagli, Marina Papatriantafilou, and Vincenzo Gulisano. 2021. Lachesis: A middleware for customizing OS scheduling of stream processing queries. In Proc. of ACM Middleware’21. 365–378.

[134]

Olga Papaemmanouil, Ugur Çetintemel, and John Jannotti. 2009. Supporting generic cost models for wide-area stream processing. In Proc. of IEEE ICDE’09. 1084–1095.

[135]

Heejin Park, Shuang Zhai, Long Lu, and Felix X. Lin. 2019. Streambox-TZ: Secure stream analytics at the edge with trustzone. In Proc. of USENIX ATC’19. 537–554.

[136]

Thao N. Pham, Panos K. Chrysanthis, and Alexandros Labrinidis. 2016. Avoiding class warfare: Managing continuous queries with differentiated classes of service. VLDB J. 25, 2 (2016), 197–221.

Digital Library

[137]

Thao N. Pham, Nikos R. Katsipoulakis, Panos K. Chrysanthis, and Alexandros Labrinidis. 2017. Uninterruptible migration of continuous queries without operator state migration. ACM SIGMOD Rec. 46, 3 (2017), 17–22.

Digital Library

[138]

Peter R. Pietzuch, Jonathan Ledlie, Jeffrey Shneidman, Mema Roussopoulos, Matt Welsh, and Margo I. Seltzer. 2006. Network-aware operator placement for stream-processing systems. In Proc. of IEEE ICDE’06. 49–60.

[139]

Cui Qin, Holger Eichelberger, and Klaus Schmid. 2019. Enactment of adaptation in data stream processing with latency implications—A systematic literature review. Inf. Softw. Technol. 111 (2019), 1–21.

Digital Library

[140]

Parisa Rahimzadeh, Jinsung Lee, Youngbin Im, Siun-Chuon Mau, Eric C. Lee, Bradford O. Smith, Fatemah Al-Duoli, Carlee Joe-Wong, and Sangtae Ha. 2020. SPARCLE: Stream processing applications over dispersed computing networks. In Proc. of IEEE ICDCS’20. 1067–1078.

[141]

Sajith Ravindra, Miyuru Dayarathna, and Sanath Jayasena. 2017. Latency aware elastic switching-based stream processing over compressed data streams. In Proc. of ACM/SPEC ICPE’17. 91–102.

[142]

Thomas Repantis, Xiaohui Gu, and Vana Kalogeraki. 2009. QoS-aware shared component composition for distributed stream processing systems. IEEE Trans. Parallel Distrib. Syst. 20, 7 (2009), 968–982.

Digital Library

[143]

Nicolo Rivetti, Emmanuelle Anceaume, Yann Busnel, Leonardo Querzoni, and Bruno Sericola. 2016. Online scheduling for shuffle grouping in distributed stream processing systems. In Proc. of ACM/IFIP/USENIX Middleware’16.

[144]

Stamatia Rizou, Frank Dürr, and Kurt Rothermel. 2010. Solving the multi-operator placement problem in large-scale operator networks. In Proc. of IEEE ICCCN’10. 1–6.

[145]

Henriette Röger and Ruben Mayer. 2019. A comprehensive survey on parallelization and elasticity in stream processing. ACM Comput. Surv. 52, 2 (2019), Article 36, 37 pages.

[146]

Olubisi Runsewe and Nancy Samaan. 2017. Cloud resource scaling for big data streaming applications using a layered multi-dimensional hidden Markov model. In Proc. of IEEE/ACM CCGRID’17. 848–857.

[147]

Gabriele Russo Russo, Valeria Cardellini, Giuliano Casale, and Francesco Lo Presti. 2021. MEAD: Model-based vertical auto-scaling for data stream processing. In Proc. of IEEE/ACM CCGRID’21. 314–323.

[148]

Gabriele Russo Russo, Valeria Cardellini, and Francesco Lo Presti. 2019. Reinforcement learning based policies for elastic stream processing on heterogeneous resources. In Proc. of ACM DEBS’19. 31–42.

[149]

Gabriele Russo Russo, Valeria Cardellini, Francesco Lo Presti, and Matteo Nardelli. 2021. Towards a security-aware deployment of data streaming applications in fog computing. In Fog/Edge Computing For Security, Privacy, and Applications. Springer, 355–385.

[150]

Hooman P. Sajjad, Ken Danniswara, Ahmad Al-Shishtawy, and Vladimir Vlassov. 2016. SpanEdge: Towards unifying stream processing over central and near-the-edge data centers. In Proc. of IEEE/ACM SEC’16. 168–178.

[151]

Farah Aït Salaht, Frédéric Desprez, and Adrien Lebre. 2020. An overview of service placement problem in fog and edge computing. ACM Comput. Surv. 53, 3 (2020), Article 65, 35 pages.

[152]

Benjamin Satzger, Waldemar Hummer, Philipp Leitner, and Schahram Dustdar. 2011. ESC: Towards an elastic stream computing platform for the cloud. In Proc. of IEEE CLOUD’11. 348–355.

[153]

Enrique Saurez, Kirak Hong, Dave Lillethun, Umakishore Ramachandran, and Beate Ottenwälder. 2016. Incremental deployment and migration of geo-distributed situation awareness applications in the fog. In Proc. of ACM DEBS’16. 258–269.

[154]

Scott Schneider, Henrique Andrade, Bugra Gedik, Alain Biem, and Kun-Lung Wu. 2009. Elastic scaling of data parallel operators in stream processing. In Proc. of IEEE IPDPS’09. 1–12.

[155]

Scott Schneider, Joel L. Wolf, Kirsten Hildrum, Rohit Khandekar, and Kun-Lung Wu. 2016. Dynamic load balancing for ordered data-parallel regions in distributed streaming systems. In Proc. of ACM/IFIP/USENIX Middleware’16. ACM, New York, NY, Article 21, 14 pages.

[156]

Scott Schneider and Kun-Lung Wu. 2017. Low-synchronization, mostly lock-free, elastic scheduling for streaming runtimes. In Proc. of ACM SIGPLAN PLDI’17. 648–661.

[157]

M. A. Shah, J. M. Hellerstein, Sirish Chandrasekaran, and M. J. Franklin. 2003. Flux: An adaptive partitioning operator for continuous query systems. In Proc. of ICDE’03. IEEE, Los Alamitos, CA, 25–36.

[158]

Mohamed A. Sharaf, Panos K. Chrysanthis, Alexandros Labrinidis, and Kirk Pruhs. 2008. Algorithms and metrics for processing multiple heterogeneous continuous queries. ACM Trans. Database Syst. 33, 1 (2008), Article 5, 44 pages.

Digital Library

[159]

Anshu Shukla and Yogesh Simmhan. 2018. Toward reliable and rapid elasticity for streaming dataflows on clouds. In Proc. of IEEE ICDCS’18. 1096–1106.

[160]

Alexandre da Silva Veith, Felipe R. de Souza, Marcos D. de Assunção, Laurent Lefèvre, and Julio C. Santos dos Anjos. 2019. Multi-objective reinforcement learning for reconfiguring data stream analytics on edge computing. In Proc. of ICPP’19. ACM, New York, NY, Article 106, 10 pages.

[161]

Rayman Preet Singh, Bharath Kumarasubramanian, Prateek Maheshwari, and Samarth Shetty. 2020. Auto-sizing for stream processing applications at LinkedIn. In Proc. of USENIX HotCloud’20.

[162]

Ahmad Slo, Sukanya Bhowmik, and Kurt Rothermel. 2019. eSPICE: Probabilistic load shedding from input event streams in complex event processing. In Proc. of ACM/IFIP Middleware’19. ACM, New York, NY, 215–227.

[163]

Ahmad Slo, Sukanya Bhowmik, and Kurt Rothermel. 2020. State-aware load shedding from input event streams in complex event processing. IEEE Trans. Big Data.To appear.

[164]

Michael Stonebraker, Uǧur Çetintemel, and Stan Zdonik. 2005. The 8 requirements of real-time stream processing. ACM SIGMOD Rec. 34, 4 (2005), 42–47.

Digital Library

[165]

Dawei Sun, Shang Gao, Xunyun Liu, Xindong You, and Rajkumar Buyya. 2020. Dynamic redirection of real-time data streams for elastic stream computing. Future Gener. Comput. Syst. 112 (2020), 193–208.

[166]

Dawei Sun, Guangyan Zhang, Songlin Yang, Weimin Zheng, Samee Ullah Khan, and Keqin Li. 2015. Re-Stream: Real-time and energy-efficient resource scheduling in big data stream computing environments. Inf. Sci. 319 (2015), 92–112.

Digital Library

[167]

Nicoleta Tantalaki, Stavros Souravlas, and Manos Roumeliotis. 2020. A review on big data real-time stream processing and its scheduling techniques. Int. J. Parallel Emergent Distributed Syst. 35, 5 (2020), 571–601.

[168]

Nesime Tatbul, Uğur Çetintemel, Stan Zdonik, Mitch Cherniack, and Michael Stonebraker. 2003. Load shedding in a data stream manager. In Proc. of VLDB’03. 309–320.

[169]

Nesime Tatbul, Uǧur Çetintemel, and Stanley B. Zdonik. 2007. Staying FIT: Efficient load shedding techniques for distributed stream processing. In Proc. of VLDB’07. ACM, New York, NY, 159–170.

[170]

Abhishek Tiwari, Brian Ramprasad, Seyed H. Mortazavi, Moshe Gabel, and Eyal de Lara. 2019. Reconfigurable streaming for the mobile edge. In Proc. of HotMobile’19. ACM, New York, NY, 153–158.

[171]

Quoc-Cuong To, Juan Soto, and Volker Markl. 2018. A survey of state management in big data processing systems. VLDB J. 27, 6 (2018), 847–872.

Digital Library

[172]

Rafael Tolosana-Calasanz, Javier Diaz Montes, Omer F. Rana, and Manish Parashar. 2017. Feedback-control and queueing theory-based resource management for streaming applications. IEEE Trans. Parallel Distrib. Syst. 28, 4 (2017), 1061–1075.

Digital Library

[173]

Geoffrey Phi C. Tran, John Paul Walters, and Stephen P. Crago. 2018. Reducing tail latencies while improving resiliency to timing errors for stream processing workloads. In Proc. of IEEE/ACM UCC’18. 194–203.

[174]

Peter A. Tucker, David Maier, Tim Sheard, and Leonidas Fegaras. 2003. Exploiting punctuation semantics in continuous data streams. IEEE Trans. Knowl. Data Eng. 15, 3 (2003), 555–568.

Digital Library

[175]

Radu Tudoran, Olivier Nano, Ivo Santos, Alexandru Costan, Hakan Soncu, Luc Bouge, and Gabriel Antoniu. 2014. JetStream: Enabling high performance event streaming across cloud data-centers. In Proc. of ACM DEBS’14. 23–34.

[176]

Jan Sipke van der Veen, Bram van der Waaij, Elena Lazovik, Wilco Wijbrandi, and Robert J. Meijer. 2015. Dynamically scaling Apache Storm for the analysis of streaming data. In Proc. of IEEE BigDataService’15. 154–161.

[177]

Shivaram Venkataraman, Aurojit Panda, Kay Ousterhout, Michael Armbrust, Ali Ghodsi, Michael J. Franklin, Benjamin Recht, and Ion Stoica. 2017. Drizzle: Fast and adaptable stream processing at scale. In Proc. of ACM SOSP’17. 374–389.

[178]

Ke Wang, Avrilia Floratou, Ashvin Agrawal, and Daniel Musgrave. 2020. Spur: Mitigating slow instances in large-scale streaming pipelines. In Proc. of ACM SIGMOD’20. 2271–2285.

[179]

Li Wang, Tom Z. J. Fu, Richard T. B. Ma, Marianne Winslett, and Zhenjie Zhang. 2019a. Elasticutor: Rapid elasticity for realtime stateful stream processing. In Proc. of ACM SIGMOD’19. 573–588.

[180]

Yidan Wang, Zahir Tari, Mohammad R. Hoseiny Farahabady, and Albert Y. Zomaya. 2017. Model-based scheduling for stream processing systems. In Proc. of IEEE HPCC/SmartCity/DSS’17. 215–222.

[181]

Yidan Wang, Zahir Tari, Xiaoran Huang, and Albert Y. Zomaya. 2019b. A network-aware and partition-based resource management scheme for data stream processing. In Proc. of ICPP’19. ACM, New York, NY, Article 20, 10 pages.

[182]

Xiaohui Wei, Lina Li, Xiang Li, Xingwang Wang, Shang Gao, and Hongliang Li. 2019. Pec: Proactive elastic collaborative resource scheduling in data stream processing. IEEE Trans. Parallel Distrib. Syst. 30, 7 (2019), 1628–1642.

Digital Library

[183]

Song Wu, Die Hu, Shadi Ibrahim, Hai Jin, Jiang Xiao, Fei Chen, and Haikun Liu. 2019. When FPGA-accelerator meets stream data processing in the edge. In Proc. of IEEE ICDCS’19. 1818–1829.

[184]

Song Wu, Mi Liu, Shadi Ibrahim, Hai Jin, Lin Gu, Fei Chen, and Zhiyi Liu. 2018. TurboStream: Towards low-latency data stream processing. In Proc. of IEEE ICDCS’18. 983–993.

[185]

Ying Xing, Stanley B. Zdonik, and Jeong-Hyon Hwang. 2005. Dynamic load distribution in the Borealis stream processor. In Proc. of IEEE ICDE’05. 791–802.

[186]

Jielong Xu, Zhenhua Chen, Jian Tang, and Sen Su. 2014. T-storm: Traffic-aware online scheduling in storm. In Proc. of IEEE ICDCS’14. 535–544.

[187]

Jinlai Xu, Balaji Palanisamy, Qingyang Wang, Heiko Ludwig, and Sandeep Gopisetty. 2022. Amnis: Optimized stream processing for edge computing. J. Parallel Distrib. Comput. 160 (2022), 49–64.

Digital Library

[188]

Le Xu, Boyang Peng, and Indranil Gupta. 2016. Stela: Enabling stream processing systems to scale-in and scale-out on-demand. In Proc. of IEEE IC2E’16. 22–31.

[189]

Le Xu, Shivaram Venkataraman, Indranil Gupta, Luo Mai, and Rahul Potharaju. 2021. Move fast and meet deadlines: Fine-grained real-time stream processing with Cameo. In Proc. of USENIX NSDI’21. 389–405.

[190]

Nikos Zacheilas, Vana Kalogeraki, Nikolaos Zygouras, Nikolaos Panagiotou, and Dimitrios Gunopulos. 2015. Elastic complex event processing exploiting prediction. In Proc. of IEEE Big Data’15. 213–222.

[191]

Nikos Zacheilas, Nikolas Zygouras, Nikolaos Panagiotou, Vana Kalogeraki, and Dimitrios Gunopulos. 2016. Dynamic load balancing techniques for distributed complex event processing systems. In Distributed Applications and Interoperable Systems. Lecture Notes in Computer Science, Vol. 9687. Springer, 174–188.

[192]

Matei Zaharia, Tathagata Das, Haoyuan Li, Timothy Hunter, Scott Shenker, and Ion Stoica. 2013. Discretized streams: Fault-tolerant streaming computation at scale. In Proc. of ACM SOSP’13. 423–438.

[193]

Ali Reza Zamani, Daniel Balouek-Thomert, Juan J. Villalobos, Ivan Rodero, and Manish Parashar. 2020. An edge-aware autonomic runtime for data streaming and in-transit processing. Future Gener. Comput. Syst. 110 (2020), 107–118.

[194]

Steffen Zeuch, Ankit Chaudhary, Bonaventura Del Monte, Haralampos Gavriilidis, Dimitrios Giouroukis, Philipp Grulich, Sebastian Bress, Jonas Traub, and Voker Markl. 2020. The NebulaStream platform for data and application management in the Internet of Things. In Proc. of CIDR’20.

[195]

Ben Zhang, Xin Jin, Sylvia Ratnasamy, John Wawrzynek, and Edward A. Lee. 2018. AWStream: Adaptive wide-area streaming analytics. In Proc. of ACM SIGCOMM’18. 236–252.

[196]

Quan Zhang, Yang Song, Ramani Routray, and Weisong Shi. 2016. Adaptive block and batch sizing for batched stream processing system. In Proc. of IEEE ICAC’16. 35–44.

[197]

Shuhao Zhang, Feng Zhang, Yingjun Wu, Bingsheng He, and Paul Johns. 2019. Hardware-conscious stream processing: A survey. ACM SIGMOD Rec. 48, 4 (2019), 18–29.

Digital Library

[198]

Yongluan Zhou, Beng Chin Ooi, Kian-Lee Tan, and Ji Wu. 2006. Efficient dynamic operator placement in a locally distributed continuous query system. In On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE. Lecture Notes in Computer Science, Vol. 4275. Springer, 54–71.

[199]

Yongluan Zhou, Ji Wu, and Ahmed Khan Leghari. 2013. Multi-query scheduling for time-critical data stream applications. In Proc. of SSDBM’13. ACM, New York, NY, Article 15, 12 pages.

Cited By

Tan JTang ZCai WTan WXiao XZhang JGao YLi K(2025)A Cost-Aware Operator Migration Approach for Distributed Stream Processing SystemIEEE Transactions on Cloud Computing10.1109/TCC.2025.353851213:1(441-454)Online publication date: Jan-2025
https://doi.org/10.1109/TCC.2025.3538512
Ecker RKaragiannis VSober MSchulte S(2025)Latency-Aware Placement of Stream Processing Operators in Modern-Day Stream Processing FrameworksJournal of Parallel and Distributed Computing10.1016/j.jpdc.2025.105041(105041)Online publication date: Jan-2025
https://doi.org/10.1016/j.jpdc.2025.105041
Gao JCai C(2025)Incorporating stochasticity in demands for optimizing resource allocation in versatile edge systems devoid of layer constraintsAin Shams Engineering Journal10.1016/j.asej.2025.10330016:3(103300)Online publication date: Mar-2025
https://doi.org/10.1016/j.asej.2025.103300
Show More Cited By

Index Terms

Runtime Adaptation of Data Stream Processing Systems: The State of the Art

Recommendations

Generic windowing support for extensible stream processing systems

Stream processing applications process high volume, continuous feeds from live data sources, employ data-in-motion analytics to analyze these feeds, and produce near real-time insights with low latency. One of the fundamental characteristics of such ...
Towards Elastic and Sustainable Data Stream Processing on Edge Infrastructure
ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

Much of the data produced today is processed as it is generated by data stream processing systems. Although the cloud is often the target infrastructure for deploying data stream processing applications, resources located at the edges of the Internet ...
A New Application Benchmark for Data Stream Processing Architectures in an Enterprise Context: Doctoral Symposium
DEBS '17: Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems

Against the backdrop of ever-growing data volumes and trends like the Internet of Things (IoT) or Industry 4.0, Data Stream Processing Systems (DSPSs) or data stream processing architectures in general receive a greater interest. Continuously analyzing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 54, Issue 11s

January 2022

785 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3551650

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2022

Online AM: 10 February 2022

Accepted: 01 January 2022

Revised: 01 December 2021

Received: 01 April 2021

Published in CSUR Volume 54, Issue 11s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
2,943
Total Downloads

Downloads (Last 12 months)708
Downloads (Last 6 weeks)87

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tan JTang ZCai WTan WXiao XZhang JGao YLi K(2025)A Cost-Aware Operator Migration Approach for Distributed Stream Processing SystemIEEE Transactions on Cloud Computing10.1109/TCC.2025.353851213:1(441-454)Online publication date: Jan-2025
https://doi.org/10.1109/TCC.2025.3538512
Ecker RKaragiannis VSober MSchulte S(2025)Latency-Aware Placement of Stream Processing Operators in Modern-Day Stream Processing FrameworksJournal of Parallel and Distributed Computing10.1016/j.jpdc.2025.105041(105041)Online publication date: Jan-2025
https://doi.org/10.1016/j.jpdc.2025.105041
Gao JCai C(2025)Incorporating stochasticity in demands for optimizing resource allocation in versatile edge systems devoid of layer constraintsAin Shams Engineering Journal10.1016/j.asej.2025.10330016:3(103300)Online publication date: Mar-2025
https://doi.org/10.1016/j.asej.2025.103300
Ezhilchelvan PMitrani I(2024)Checkpointing models for tasks of different typesACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/36586679:3(1-23)Online publication date: 21-May-2024
https://dl.acm.org/doi/10.1145/3658667
Kozar ADel Monte BZeuch SMarkl V(2024)Fault Tolerance Placement in the Internet of ThingsProceedings of the ACM on Management of Data10.1145/36549412:3(1-29)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654941
Kumar SAgarwal S(2024) Rule based complex event processing for IoT applications: Review, classification and challenges Expert Systems10.1111/exsy.13597Online publication date: 30-Mar-2024
https://doi.org/10.1111/exsy.13597
Havers BPapatriantafilou MGulisano V(2024)Nona: A Framework for Elastic Stream Provenance2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00071(703-714)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00071
Arsalane KPierre GIbrahim S(2024)Toward Stream Processing Elasticity in Realistic Geo-Distributed Environments2024 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E61754.2024.00020(118-125)Online publication date: 24-Sep-2024
https://doi.org/10.1109/IC2E61754.2024.00020
Benamer A(2024)Semi-Optimal MicroService Placement in the Fog with Quality of Service (QoS) Awareness2024 10th International Conference on Control, Decision and Information Technologies (CoDIT)10.1109/CoDIT62066.2024.10708415(455-460)Online publication date: 1-Jul-2024
https://doi.org/10.1109/CoDIT62066.2024.10708415
Liu RYue PShangguan BGong JXiang LLu B(2024)RTGDC: a real-time ingestion and processing approach in geospatial data cube for digital twin of earthInternational Journal of Digital Earth10.1080/17538947.2024.236538617:1Online publication date: 19-Jun-2024
https://doi.org/10.1080/17538947.2024.2365386
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents