research-article

Distream: scaling live video analytics with workload-adaptive distributed edge intelligence

Authors:

Mi ZhangAuthors Info & Claims

SenSys '20: Proceedings of the 18th Conference on Embedded Networked Sensor Systems

Pages 409 - 421

https://doi.org/10.1145/3384419.3430721

Published: 16 November 2020 Publication History

Abstract

Video cameras have been deployed at scale today. Driven by the breakthrough in deep learning (DL), organizations that have deployed these cameras start to use DL-based techniques for live video analytics. Although existing systems aim to optimize live video analytics from a variety of perspectives, they are agnostic to the workload dynamics in real-world deployments. In this work, we present Distream, a distributed live video analytics system based on the smart camera-edge cluster architecture, that is able to adapt to the workload dynamics to achieve low-latency, high-throughput, and scalable live video analytics. The key behind the design of Distream is to adaptively balance the workloads across smart cameras and partition the workloads between cameras and the edge cluster. In doing so, Distream is able to fully utilize the compute resources at both ends to achieve optimized system performance. We evaluated Distream with 500 hours of distributed video streams from two real-world video datasets with a testbed that consists of 24 cameras and a 4-GPU edge cluster. Our results show that Distream consistently outperforms the status quo in terms of throughput, latency, and latency service level objective (SLO) miss rate.

References

[1]

2016. Nvidia Titan X. https://www.nvidia.com/en-us/geforce/products/10series/titan-x-pascal/

[2]

2017. 24-Port Gigabit Stackable Smart Managed Switch with 4 10GbE SFP+ ports. http://us.dlink.com/products/business-solutions/dgs-1510-28x/

[3]

2017. Nvidia Jetson TX1. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems-dev-kits-modules/

[4]

2017. OpenCV Background Subtraction. https://docs.opencv.org/3.4/db/d5c/tutorial_py_bg_subtraction.html

[5]

2018. Networking Solutions for IP Surveillance. https://www.netgear.com/images/pdf/IP-Video-Surveillance_Networking-Solution-Guide.pdf

[6]

2018. Nvidia Jetson TX2. https://devblogs.nvidia.com/jetson-tx2-delivers-twice-intelligence-edge/

[7]

2019. JacksonHole. https://www.seejh.com/live

[8]

Faruk Akgul. 2013. ZeroMQ. Packt Publishing Ltd.

[9]

Christopher Canel, Thomas Kim, Giulio Zhou, Conglong Li, Hyeontaek Lim, David G Andersen, Michael Kaminsky, and Subramanya R Dulloor. 2019. Scaling video analytics on constrained edge nodes. arXiv preprint arXiv:1905.13536 (2019).

[10]

Byung-Gon Chun, Sunghwan Ihm, Petros Maniatis, Mayur Naik, and Ashwin Patti. 2011. Clonecloud: elastic execution between mobile device and cloud. In Proceedings of the sixth conference on Computer systems. 301--314.

Digital Library

[11]

Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J Franklin, Joseph E Gonzalez, and Ion Stoica. 2017. Clipper: A low-latency online prediction serving system. In 14th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 17). 613--627.

[12]

Eduardo Cuervo, Aruna Balasubramanian, Dae-ki Cho, Alec Wolman, Stefan Saroiu, Ranveer Chandra, and Paramvir Bahl. 2010. MAUI: making smartphones last longer with code offload. In Proceedings of the 8th international conference on Mobile systems, applications, and services. 49--62.

Digital Library

[13]

Biyi Fang, Xiao Zeng, Faen Zhang, Hui Xu, and Mi Zhang. 2020. FlexDNN: Input-Adaptive On-Device Deep Learning for Efficient Mobile Vision. In Proceedings of the 5th ACM/IEEE Symposium on Edge Computing (SEC).

[14]

Biyi Fang, Xiao Zeng, and Mi Zhang. 2018. NestDNN: Resource-aware multi-tenant on-device deep learning for continuous mobile vision. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking (MobiCom). 115--127.

Digital Library

[15]

Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Computer Vision (ICCV), 2017 IEEE International Conference on. IEEE, 2980--2988.

[16]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[17]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).

[18]

Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodik, Paramvir Bahl, Matthai Philipose, Phillip B Gibbons, and Onur Mutlu. 2018. Focus: Querying large video datasets with low latency and low cost. arXiv preprint arXiv:1801.03493 (2018).

[19]

Chien-Chun Hung, Ganesh Ananthanarayanan, Peter Bodik, Leana Golubchik, Minlan Yu, Paramvir Bahl, and Matthai Philipose. 2018. VideoEdge: Processing Camera Streams using Hierarchical Clusters. In 2018 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 115--131.

[20]

Loc N Huynh, Youngki Lee, and Rajesh Krishna Balan. 2017. Deepmon: Mobile gpu-based deep learning framework for continuous vision applications. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. 82--95.

Digital Library

[21]

Samvit Jain, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, and Joseph Gonzalez. 2019. Scaling video analytics systems to large camera deployments. In Proceedings of the 20th International Workshop on Mobile Computing Systems and Applications. 9--14.

Digital Library

[22]

Junchen Jiang, Ganesh Ananthanarayanan, Peter Bodik, Siddhartha Sen, and Ion Stoica. 2018. Chameleon: scalable adaptation of video analytics. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 253--266.

Digital Library

[23]

Junchen Jiang, Yuhao Zhou, Ganesh Ananthanarayanan, Yuanchao Shu, and Andrew A Chien. 2019. Networked Cameras Are the New Big Data Clusters. In Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges. 1--7.

Digital Library

[24]

Shuang Jiang, Zhiyao Ma, Xiao Zeng, Chenren Xu, Mi Zhang, Chen Zhang, and Yunxin Liu. 2020. SCYLLA: QoE-aware Continuous Mobile Vision with FPGA-based Dynamic Deep Neural Network Reconfiguration. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1369--1378.

[25]

Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. NoScope: optimizing neural network queries over video at scale. Proceedings of the VLDB Endowment 10, 11 (2017), 1586--1597.

Digital Library

[26]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436.

[27]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 21--37.

[28]

Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270 (2018).

[29]

Lin Ma, Dana Van Aken, Ahmed Hefny, Gustavo Mezerhane, Andrew Pavlo, and Geoffrey J Gordon. 2018. Query-based workload forecasting for self-driving database management systems. In Proceedings of the 2018 International Conference on Management of Data. ACM, 631--645.

Digital Library

[30]

Xiaolei Ma, Houyue Zhong, Yi Li, Junyan Ma, Zhiyong Cui, and Yinhai Wang. 2020. Forecasting transportation network speed using deep capsule networks with nested lstm models. IEEE Transactions on Intelligent Transportation Systems (2020).

[31]

Shadi A Noghabi, Landon Cox, Sharad Agarwal, and Ganesh Ananthanarayanan. 2020. THE EMERGING LANDSCAPE OF EDGE COMPUTING. GetMobile: Mobile Computing and Communications 23, 4 (2020), 11--20.

Digital Library

[32]

Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018).

[33]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91--99.

[34]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4510--4520.

[35]

Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, and Ravi Sundaram. 2019. Nexus: a GPU cluster engine for accelerating DNN-based video analysis. In Proceedings of the 27th ACM Symposium on Operating Systems Principles. 322--337.

Digital Library

[36]

Shivaram Venkataraman, Aurojit Panda, Kay Ousterhout, Michael Armbrust, Ali Ghodsi, Michael J Franklin, Benjamin Recht, and Ion Stoica. 2017. Drizzle: Fast and adaptable stream processing at scale. In Proceedings of the 26th Symposium on Operating Systems Principles. ACM, 374--389.

Digital Library

[37]

Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.

Digital Library

[38]

Ben Zhang, Xin Jin, Sylvia Ratnasamy, John Wawrzynek, and Edward A Lee. 2018. AWStream: adaptive wide-area streaming analytics. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. ACM, 236--252.

Digital Library

[39]

Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J Freedman. 2017. Live Video Analytics at Scale with Approximation and Delay-Tolerance. In NSDI, Vol. 9. 1.

Digital Library

[40]

Mi Zhang, Faen Zhang, Nicholas D Lane, Yuanchao Shu, Xiao Zeng, Biyi Fang, Shen Yan, and Hui Xu. 2020. Deep Learning in the Era of Edge Computing: Challenges and Opportunities. Fog Computing: Theory and Practice (2020), 67--78.

[41]

Zoran Zivkovic and Ferdinand Van Der Heijden. 2006. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern recognition letters 27, 7 (2006), 773--780.

Cited By

Golpayegani FChen NAfraz NGyamfi EMalekjafarian ASchäfer DKrupitzer C(2024)Adaptation in Edge Computing: A Review on Design Principles and Research ChallengesACM Transactions on Autonomous and Adaptive Systems10.1145/366420019:3(1-43)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3664200
Shi SLing NJiang ZHuang XHe YZhao XYang BBian CXia JYan ZYeung RXing GGanesan DShi W(2024)Soar: Design and Deployment of A Smart Roadside Infrastructure System for Autonomous DrivingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649352(139-154)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3636534.3649352
Gao GDong YWang RZhou X(2024)EdgeVision: Towards Collaborative Video Analytics on Distributed Edges for Performance MaximizationIEEE Transactions on Multimedia10.1109/TMM.2024.338567826(9083-9094)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3385678
Show More Cited By

Index Terms

Distream: scaling live video analytics with workload-adaptive distributed edge intelligence
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
    2. Distributed artificial intelligence

Recommendations

Latency-Aware Placement of Data Stream Analytics on Edge Computing
Service-Oriented Computing
Abstract
The interest in processing data events under stringent time constraints as they arrive has led to the emergence of architecture and engines for data stream processing. Edge computing, initially designed to minimize the latency of content delivered ...
Addressing Application Latency Requirements through Edge Scheduling
Abstract
Latency-sensitive and data-intensive applications, such as IoT or mobile services, are leveraged by Edge computing, which extends the cloud ecosystem with distributed computational resources in proximity to data providers and consumers. This ...
Edge artificial intelligence for big data: a systematic review
Abstract
Edge computing, artificial intelligence (AI), and machine learning (ML) concepts have become increasingly prevalent in Internet of Things (IoT) applications. As the number of IoT devices continues to grow, relying solely on cloud computing for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SenSys '20: Proceedings of the 18th Conference on Embedded Networked Sensor Systems

November 2020

852 pages

ISBN:9781450375900

DOI:10.1145/3384419

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SenSys '20

Sponsor:

SenSys '20: The 18th ACM Conference on Embedded Networked Sensor Systems

November 16 - 19, 2020

Virtual Event, Japan

Acceptance Rates

Overall Acceptance Rate 174 of 867 submissions, 20%

Upcoming Conference

SenSys '24

Sponsor:
sigbed
sigbed
sigbed
sigbed
sigbed

The 22nd ACM Conference on Embedded Networked Sensor Systems

November 4 - 7, 2024

Hangzhou , China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

55
Total Citations
View Citations
1,242
Total Downloads

Downloads (Last 12 months)326
Downloads (Last 6 weeks)35

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Golpayegani FChen NAfraz NGyamfi EMalekjafarian ASchäfer DKrupitzer C(2024)Adaptation in Edge Computing: A Review on Design Principles and Research ChallengesACM Transactions on Autonomous and Adaptive Systems10.1145/366420019:3(1-43)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3664200
Shi SLing NJiang ZHuang XHe YZhao XYang BBian CXia JYan ZYeung RXing GGanesan DShi W(2024)Soar: Design and Deployment of A Smart Roadside Infrastructure System for Autonomous DrivingProceedings of the 30th Annual International Conference on Mobile Computing and Networking10.1145/3636534.3649352(139-154)Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1145/3636534.3649352
Gao GDong YWang RZhou X(2024)EdgeVision: Towards Collaborative Video Analytics on Distributed Edges for Performance MaximizationIEEE Transactions on Multimedia10.1109/TMM.2024.338567826(9083-9094)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3385678
Yan YZhang SJin YCheng FQian ZLu S(2024)Spatial and Temporal Detection With Attention for Real-Time Video Analytics at EdgesIEEE Transactions on Mobile Computing10.1109/TMC.2024.336101623:10(9254-9270)Online publication date: Oct-2024
https://doi.org/10.1109/TMC.2024.3361016
Zhang LXu JLu ZSong L(2024)CrossVision: Real-time On-Camera Video Analysis via Common RoI Load BalancingIEEE Transactions on Mobile Computing10.1109/TMC.2023.3301391(1-13)Online publication date: 2024
https://doi.org/10.1109/TMC.2023.3301391
Chen YZhang SJin YQian ZXiao MLi WLiang YLu S(2024)Crowdsourcing Upon Learning: Energy-Aware Dispatch With Guarantee for Video AnalyticsIEEE Transactions on Mobile Computing10.1109/TMC.2023.326980723:4(3138-3155)Online publication date: Apr-2024
https://doi.org/10.1109/TMC.2023.3269807
Rong CWang JWang JZhou YZhang J(2024)Live Migration of Video Analytics Applications in Edge ComputingIEEE Transactions on Mobile Computing10.1109/TMC.2023.324653923:3(2078-2092)Online publication date: Mar-2024
https://doi.org/10.1109/TMC.2023.3246539
Zhang WJing YZhang YLin TYan J(2024)Retina-U: A Two-Level Real-Time Analytics Framework for UHD Live Video StreamingIEEE Transactions on Broadcasting10.1109/TBC.2023.334564670:2(429-440)Online publication date: Jun-2024
https://doi.org/10.1109/TBC.2023.3345646
Wang ZJi YZhou JJin MZhu M(2024)Soteria: A Privacy-Aware and Budget-Limited Multi-Armed Bandit-Based Crowdsourcing Approach for Edge Video Analytics2024 2nd International Conference On Mobile Internet, Cloud Computing and Information Security (MICCIS)10.1109/MICCIS63508.2024.00034(155-162)Online publication date: 19-Apr-2024
https://doi.org/10.1109/MICCIS63508.2024.00034
Wang CYang PHou JLiu ZZhang N(2024)Dependence-Aware Multitask Scheduling for Edge Video Analytics With Accuracy GuaranteeIEEE Internet of Things Journal10.1109/JIOT.2024.339729611:16(26970-26983)Online publication date: 15-Aug-2024
https://doi.org/10.1109/JIOT.2024.3397296
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents