research-article

Public Access

RISE: An Automated Framework for Real-Time Intelligent Video Surveillance on FPGA

Authors:

Bita Darvish Rouhani,

Azalia Mirhoseini,

Farinaz KoushanfarAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 16, Issue 5s

Article No.: 158, Pages 1 - 18

https://doi.org/10.1145/3126549

Published: 27 September 2017 Publication History

Abstract

This paper proposes RISE, an automated Reconfigurable framework for real-time background subtraction applied to Intelligent video SurveillancE. RISE is devised with a new streaming-based methodology that adaptively learns/updates a corresponding dictionary matrix from background pixels as new video frames are captured over time. This dictionary is used to highlight the foreground information in each video frame. A key characteristic of RISE is that it adaptively adjusts its dictionary for diverse lighting conditions and varying camera distances by continuously updating the corresponding dictionary. We evaluate RISE on natural-scene vehicle images of different backgrounds and ambient illuminations. To facilitate automation, we provide an accompanying API that can be used to deploy RISE on FPGA-based system-on-chip platforms. We prototype RISE for end-to-end deployment of three widely-adopted image processing tasks used in intelligent transportation systems: License Plate Recognition (LPR), image denoising/reconstruction, and principal component analysis. Our evaluations demonstrate up to 87-fold higher throughput per energy unit compared to the prior-art software solution executed on ARM Cortex-A15 embedded platform.

References

[1]

V. Cevher, A. Sankaranarayanan, M. F. Duarte, D. Reddy, R. G. Baraniuk, and R. Chellappa. 2008. Compressive sensing for background subtraction. 155--168.

[2]

D. Schreiber and M. Rauter. 2009. GPU-based non-parametric background subtraction for a practical surveillance system. 870--877.

[3]

F. Porikli. 2006. Achieving real-time object detection and tracking under extreme conditions. Journal of Real-Time Image Processing 1, 1, 33--40.

[4]

J. Oliveira, A. Printes, R. Freire, E. Melcher, and I.S. Silva. 2006. FPGA architecture for static background subtraction in real time. 26--31.

Digital Library

[5]

C. Sanchez-Ferreira J. Mori, and C. Llanos. 2012. Background subtraction algorithm for moving object detection in FPGA. 1--6.

[6]

C. Lu, J. Shi, and J. Jia. 2013. Online robust dictionary learning. 415--422.

Digital Library

[7]

E. J. Candès and M. B. Wakin. 2008. An introduction to compressive sampling. IEEE signal processing magazine, 25, 2, 21--30.

[8]

J. Tropp, A. C. Gilbert et al. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53, 12, 4655--4666.

Digital Library

[9]

A. Mirhoseini, E. Dyer, E. Songhori, R. Baraniuk, and F. Koushanfar. 2015. Rankmap: A platform-aware framework for distributed learning from dense datasets. arXiv preprint arXiv:1503.08169.

[10]

A. Mirhoseini, B. D. Rouhani, E. M. Songhori, and F. Koushanfar. 2016. Perform-ml: Performance optimized machine learning by platform and content aware customization. 20.

Digital Library

[11]

J. Sauvola and M. Pietikäinen. 2000. Adaptive document image binarization. Pattern recognition, 33, 2, 225--236.

[12]

M. Karkooti, J. R. Cavallaro, and C. Dick. 2005. FPGA implementation of matrix inversion using qrd-rls algorithm.

[13]

Å. Björck. 1967. Solving linear least squares problems by gram-schmidt orthogonalization. BIT Numerical Mathematics, 7, 1, 1--21.

[14]

W. Hoffmann. 1989. Iterative algorithms for gram-schmidt orthogonalization. Computing, 41, 4, 335--348.

Digital Library

[15]

XILLYBUS. 2017. http://xillybus.com/.

[16]

XPower. 2012. {Online}. Available: http://www.xilinx.com/support/documentation/user_guides/ug440.pdf.

[17]

Rise source codes. https://github.com/Bitadr/RISE.

[18]

Y. Wen, Y. Lu, J. Yan, Z. Zhou, K. M. Von Deneen, and P. Shi. 2011. An algorithm for license plate recognition applied to intelligent transportation system. IEEE Transactions on Intelligent Transportation Systems, 12, 3, 830--845.

Digital Library

[19]

C. N. E. Anagnostopoulos, I. E. Anagnostopoulos, V. Loumos, and E. Kayafas. 2006. A license plate-recognition algorithm for intelligent transportation system applications. Intelligent Transportation Systems, IEEE Transactions on, 7, 3, 377--392.

Digital Library

[20]

C. Arth, H. Bischof, and C. Leistner. 2006. Tricam-an embedded platform for remote traffic surveillance. 125--125.

Digital Library

[21]

C.-N. E. Anagnostopoulos, I. E. Anagnostopoulos, I. D. Psoroulas, V. Loumos, and E. Kayafas. 2008. License plate recognition from still images and video sequences: A survey. IEEE Transactions on intelligent transportation systems, 9, 3, 377--391.

Digital Library

[22]

M. Elad and M. Aharon. 2006. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing, 15, 12, 3736--3745.

Digital Library

[23]

S. D. A. LightField. 2014. {Online}. Available: http://lightfield.stanford.edu/.

[24]

H. R. S. D. Salina. 2014. {Online}. Available: http://www.ehu.es/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.

[25]

LPR database. 2017. http://www.medialab.ntua.gr/research/LPRdatabase/Still_images/difficult_cases/.

[26]

Video surveillance frame rate. 2016. http://ipvm.com/reports/.

[27]

B. Rouhani, E. Songhori, A. Mirhoseini, and F. Koushanfar. 2015. Ssketch: An automated framework for streaming sketch-based analysis of big data on FPGA. In 23rd International Symposium on Field-Programmable Custom Computing Machines conference (FCCM).

Digital Library

[28]

M. Andrecut. 2008. Fast GPU implementation of sparse signal recovery from random projections. arXiv preprint arXiv:0809.1833.

[29]

J. D. Blanchard and J. Tanner. 2013. GPU accelerated greedy algorithms for compressed sensing. Mathematical Programming Computation, 5, 3, 267--304.

[30]

Y. Fang, L. Chen, J. Wu, and B. Huang. 2011. GPU implementation of orthogonal matching pursuit for compressive sensing. 1044--1047.

Digital Library

[31]

P. Maechler, P. Greisen, N. Felber, and A. Burg. 2010. Matching pursuit: Evaluation and implementatio for LTE channel estimation. 589--592.

[32]

A. Septimus and R. Steinberg. 2010. Compressive sampling hardware reconstruction. 3316--3319.

[33]

L. Bai, P. Maechler, M. Muehlberghuber, and H. Kaeslin. 2012. High-speed compressed sensing reconstruction on FPGA using OMP and AMP. 53--56.

[34]

J. L. Stanislaus and T. Mohsenin. 2013. Low-complexity FPGA implementation of compressive sensing reconstruction. 671--675.

Digital Library

[35]

A. M. Kulkarni, H. Homayoun, and T. Mohsenin. 2014. A parallel and reconfigurable architecture for efficient OMP compressive sensing reconstruction. 299--304.

Digital Library

[36]

T. Hosaka, T. Kobayashi, and N. Otsu. 2011. Object detection using background subtraction and foreground motion estimation. IPSJ Transactions on Computer Vision and Applications, 3, 9--20.

[37]

P. Viola, M. J. Jones, and D. Snow. 2005. Detecting pedestrians using patterns of motion and appearance. International Journal of Computer Vision, 63, 2, 153--161.

Digital Library

[38]

K. Mikolajczyk and C. Schmid. 2005. A performance evaluation of local descriptors. IEEE transactions on pattern analysis and machine intelligence, 27, 10, 1615--1630.

Digital Library

Cited By

Mokayed HClark TAlkhaled LMarashli MChai H(2022)On Restricted Computational Systems, Real-time Multi-tracking and Object Recognition Tasks are Possible2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)10.1109/IEEM55944.2022.9989755(1523-1528)Online publication date: 7-Dec-2022
https://doi.org/10.1109/IEEM55944.2022.9989755
Sassu ASaenz-Cogollo JAgelli M(2021)Deep-Framework: A Distributed, Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video StreamsSensors10.3390/s2112404521:12(4045)Online publication date: 11-Jun-2021
https://doi.org/10.3390/s21124045
Hanavi Hidayat F(2020)Intelligent Video Analytic for Suspicious Object Detection : A Systematic Review2020 International Conference on ICT for Smart Society (ICISS)10.1109/ICISS50791.2020.9307600(1-8)Online publication date: 19-Nov-2020
https://doi.org/10.1109/ICISS50791.2020.9307600
Show More Cited By

Index Terms

RISE: An Automated Framework for Real-Time Intelligent Video Surveillance on FPGA
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems
  2. Real-time systems
2. Computing methodologies
  1. Machine learning

Recommendations

Background subtraction based on phase feature and distance transform

A novel background subtraction method that can work under complex environments is presented in this paper. The proposed method consists of two stages: coarse foreground detection through the phase based background model we present, and foreground ...
Detection of foreground in dynamic scene via two-step background subtraction

Various computer vision applications such as video surveillance and gait analysis have to perform human detection. This is usually done via background modeling and subtraction. It is a challenging problem when the image sequence captures the human ...
A Vehicle Detection Algorithm Based on Three-Frame Differencing and Background Subtraction
ISCID '12: Proceedings of the 2012 Fifth International Symposium on Computational Intelligence and Design - Volume 01

A vehicle detection algorithm based on three-frame differencing and background subtraction is presented in this paper. Firstly, improved GMM is for background subtraction, then the moving object region is gained using background subtraction, and then ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 16, Issue 5s

Special Issue ESWEEK 2017, CASES 2017, CODES + ISSS 2017 and EMSOFT 2017

October 2017

1448 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/3145508

Editor:
Sandeep K. Shukla
Indian Institute of Technology, India

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 27 September 2017

Accepted: 01 June 2017

Revised: 01 June 2017

Received: 01 April 2017

Published in TECS Volume 16, Issue 5s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)13

Reflects downloads up to 20 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mokayed HClark TAlkhaled LMarashli MChai H(2022)On Restricted Computational Systems, Real-time Multi-tracking and Object Recognition Tasks are Possible2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM)10.1109/IEEM55944.2022.9989755(1523-1528)Online publication date: 7-Dec-2022
https://doi.org/10.1109/IEEM55944.2022.9989755
Sassu ASaenz-Cogollo JAgelli M(2021)Deep-Framework: A Distributed, Scalable, and Edge-Oriented Framework for Real-Time Analysis of Video StreamsSensors10.3390/s2112404521:12(4045)Online publication date: 11-Jun-2021
https://doi.org/10.3390/s21124045
Hanavi Hidayat F(2020)Intelligent Video Analytic for Suspicious Object Detection : A Systematic Review2020 International Conference on ICT for Smart Society (ICISS)10.1109/ICISS50791.2020.9307600(1-8)Online publication date: 19-Nov-2020
https://doi.org/10.1109/ICISS50791.2020.9307600
Shidik GNoersasongko ENugraha AAndono PJumanto JKusuma E(2019)A Systematic Review of Intelligence Video Surveillance: Trends, Techniques, Frameworks, and DatasetsIEEE Access10.1109/ACCESS.2019.29553877(170457-170473)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2955387
Darvish Rouhani BGhasemzadeh MKoushanfar FAnderson JBazargan K(2018)CausaLearnProceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays10.1145/3174243.3174259(1-10)Online publication date: 15-Feb-2018
https://dl.acm.org/doi/10.1145/3174243.3174259
Cai XHu WMa TMa R(2018)A hybrid scheduling algorithm for reconfigurable processor architecture2018 13th IEEE Conference on Industrial Electronics and Applications (ICIEA)10.1109/ICIEA.2018.8397812(745-749)Online publication date: May-2018
https://doi.org/10.1109/ICIEA.2018.8397812

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents