research-article

A New Approach for Pedestrian Density Estimation Using Moving Sensors and Computer Vision

Authors:

Eric K. Tokuda,

Yitzchak Lockerman,

Gabriel B. A. Ferreira,

Ethan Sorrelgreen,

Roberto M. Cesar-Jr.,

Claudio T. SilvaAuthors Info & Claims

ACM Transactions on Spatial Algorithms and Systems (TSAS), Volume 6, Issue 4

Article No.: 26, Pages 1 - 20

https://doi.org/10.1145/3397575

Published: 03 July 2020 Publication History

Abstract

An understanding of person dynamics is indispensable for numerous urban applications, including the design of transportation networks and planning for business development. Pedestrian counting often requires utilizing manual or technical means to count individuals in each location of interest. However, such methods do not scale to the size of a city and a new approach to fill this gap is here proposed. In this project, we used a large dense dataset of images of New York City along with computer vision techniques to construct a spatio-temporal map of relative person density. Due to the limitations of state-of-the-art computer vision methods, such automatic detection of person is inherently subject to errors. We model these errors as a probabilistic process, for which we provide theoretical analysis and thorough numerical simulations. We demonstrate that, within our assumptions, our methodology can supply a reasonable estimate of person densities and provide theoretical bounds for the resulting error.

References

[1]

Ian F. Akyildiz, Tommaso Melodia, and Kaushik R. Chowdhury. 2007. A survey on wireless multimedia sensor networks. Comput. Netw. 51, 4 (2007), 921--960.

Digital Library

[2]

Ian F. Akyildiz, Weilian Su, Yogesh Sankarasubramaniam, and Erdal Cayirci. 2002. A survey on sensor networks. IEEE Commun. Mag. 40, 8 (2002), 102--114.

Digital Library

[3]

Sean M. Arietta, Alexei A. Efros, Ravi Ramamoorthi, and Maneesh Agrawala. 2014. City forensics: Using visual elements to predict non-visual city attributes. IEEE Trans. Vis. Comput. Graph. 20, 12 (2014), 2624--2633.

[4]

Stefano Basagni, Alessio Carosi, and Chiara Petrioli. 2007. Controlled vs. uncontrolled mobility in wireless sensor networks: Some performance insights. In Proceedings of the Vehicular Technology Conference. IEEE, 269--273.

[5]

S. Behnel, R. Bradshaw, C. Citro, L. Dalcin, D. S. Seljebotn, and K. Smith. 2011. Cython: The best of both worlds. Comput. Sci. Eng. 13, 2 (2011), 31--39.

Digital Library

[6]

Jon Louis Bentley. 1975. Multidimensional binary search trees used for associative searching. Commun. ACM 18, 9 (1975), 509--517.

Digital Library

[7]

Christopher J. C. Burges. 1998. A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Disc. 2, 2 (1998), 121--167.

Digital Library

[8]

Tracy Camp, Jeff Boleng, and Vanessa Davies. 2002. A survey of mobility models for ad hoc network research. Wirel. Commun. Mobile Comput. 2, 5 (2002), 483--502.

[9]

Sunny Consolvo, David W. McDonald, Tammy Toscos, Mike Y. Chen, Jon Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand, Ryan Libby et al. 2008. Activity sensing in the wild: A field trial of UbiFit garden. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’08). ACM, 1797--1806.

Digital Library

[10]

Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE, 3213--3223.

[11]

Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. 2016. R-FCN: Object detection via region-based fully convolutional networks. arXiv preprint arXiv:1605.06409 (2016).

[12]

Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1. IEEE, 886--893.

Digital Library

[13]

Vanessa Ann Davies et al. 2000. Evaluating Mobility Models within an Ad Hoc Network. Master’s thesis. Citeseer. Colorado School of Mines, CO, USA. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.18.4191

[14]

Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zisserman. 2010. The PASCAL visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 2 (2010), 303--338.

Digital Library

[15]

Silvia Gandy, Benjamin Recht, and Isao Yamada. 2011. Tensor completion and low-n-rank tensor recovery via convex optimization. Inv. Prob. 27, 2 (2011), 025010.

[16]

Chenqiang Gao, Pei Li, Yajun Zhang, Jiang Liu, and Lan Wang. 2016. People counting based on head detection combining Adaboost and CNN in crowded surveillance environment. Neurocomputing 208 (2016), 108--116.

Digital Library

[17]

Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. 2013. Vision meets robotics: The KITTI dataset. Int. J. Rob. Res. (Sept. 2013).

Digital Library

[18]

Google Inc. Retrieved from https://maps.google.com.

[19]

Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. 1968. A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cyber. 4, 2 (1968), 100--107.

[20]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE.

[21]

Dirk Helbing. 1998. A fluid dynamic model for the movement of pedestrians. arXiv preprint cond-mat/9805213 (1998).

[22]

Chi-Fu Huang and Yu-Chee Tseng. 2005. The coverage problem in a wireless sensor network. Mobile Netw. Applic. 10, 4 (2005), 519--528.

Digital Library

[23]

Tomoharu Iwata, Hitoshi Shimizu, Futoshi Naya, and Naonori Ueda. 2017. Estimating people flow from spatiotemporal population data via collective graphical mixture models. ACM Trans. Spatial Algor. Syst. 3, 1 (2017), 2.

[24]

David B. Johnson and David A. Maltz. 1996. Dynamic source routing in ad hoc wireless networks. Mobile Comput. 353, 1 (1996), 153--181.

[25]

Sophia Karagiorgou, Dieter Pfoser, and Dimitrios Skoutas. 2017. A layered approach for more robust generation of road network maps from vehicle tracking data. ACM Trans. Spatial Algor. Syst. 3, 1 (2017), 3.

[26]

Leonard Kleinrock. 1976. Queueing Systems, Volume 2: Computer Applications. Vol. 66. John Wiley 8 Sons.

[27]

Donald Ervin Knuth. 1997. The Art of Computer Programming. Vol. 3. Addison-Wesley.

[28]

Alex Krizhevsky et al. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. Citeseer. University of Toronto, ON, Canada.

[29]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 1097--1105.

Digital Library

[30]

Nicholas D. Lane, Emiliano Miluzzo, Hong Lu, Daniel Peebles, Tanzeem Choudhury, and Andrew T. Campbell. 2010. A survey of mobile phone sensing. IEEE Commun. Mag. 48, 9 (2010), 140--150.

Digital Library

[31]

Averill M. Law and W. David Kelton. 2007. Simulation Modeling and Analysis. Vol. 3. McGraw-Hill New York,.

[32]

Uichin Lee, Eugenio Magistretti, Mario Gerla, Paolo Bellavista, and Antonio Corradi. 2009. Dissemination and harvesting of urban data using vehicular sensing platforms. IEEE Trans. Vehic. Technol. 58, 2 (2009), 882--901.

[33]

Victor Lesser, Charles L. Ortiz Jr, and Milind Tambe. 2012. Distributed Sensor Networks: A Multiagent Perspective. Vol. 9. Springer Science 8 Business Media.

[34]

Weizi Li, David Wolinski, and Ming C. Lin. 2017. City-scale traffic animation using statistical learning and metamodel-based optimization. ACM Trans. Graph. 36, 6 (2017), 200.

Digital Library

[35]

Ben Liang and Zygmunt J. Haas. 1999. Predictive distance-based mobility management for PCS networks. In Proceedings of the 18th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM’99), Vol. 3. IEEE, New York, 1377--1384.

[36]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV’14). Springer, 740--755.

[37]

Will Maddern, Geoffrey Pascoe, Chris Linegar, and Paul Newman. 2017. 1 year, 1000 km: The Oxford RobotCar dataset. Int. J. Rob. Res. 36, 1 (2017), 3--15.

Digital Library

[38]

Makoto Matsumoto and Takuji Nishimura. 1998. Mersenne Twister: A 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Trans. Model. Comput. Simul. 8, 1 (1998), 3--30.

Digital Library

[39]

Fabio Miranda, Harish Doraiswamy, Marcos Lage, Luc Wilson, Mondrian Hsieh, and Claudio T. Silva. 2018. Shadow accrual maps: Efficient accumulation of city-scale shadows over time. IEEE Trans. Vis. Comput. Graph 25, 3 (2018). https://ieeexplore.ieee.org/abstract/document/8283638.

[40]

Muaz A. Niazi and Amir Hussain. 2011. A novel agent-based simulation framework for sensing in complex adaptive environments. IEEE Sens. J. 11, 2 (2011), 404--412.

[41]

NYC open data. Retrieved from https://opendata.cityofnewyork.us/.

[42]

Sangmin Oh, Anthony Hoogs, Amitha Perera, Naresh Cuntoor, Chia-Chih Chen, Jong Taek Lee, Saurajit Mukherjee, J. K. Aggarwal, Hyungtae Lee, Larry Davis, et al. 2011. A large-scale benchmark dataset for event recognition in surveillance video. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE, 3153--3160.

Digital Library

[43]

OpenStreetMap. 2017. Planet dump. Retrieved from https://planet.osm.org.

[44]

Mohd Fauzi Othman and Khairunnisa Shazali. 2012. Wireless sensor network applications: A study in environment monitoring system. Proc. Eng. 41 (2012), 1204--1210.

[45]

Stephen K. Park and Keith W. Miller. 1988. Random number generators: Good ones are hard to find. Commun. ACM 31, 10 (1988), 1192--1201.

Digital Library

[46]

Photosynth. Retrieved from https://blogs.msdn.microsoft.com/photosynth/ 2017/02/06/microsoft-photosynth-has-been-shut-down/.

[47]

Rajib Kumar Rana, Chun Tung Chou, Salil S. Kanhere, Nirupama Bulusu, and Wen Hu. 2010. Ear-phone: An end-to-end participatory urban noise mapping system. In Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks. ACM, 105--116.

Digital Library

[48]

Jonathan Reades, Francesco Calabrese, Andres Sevtsuk, and Carlo Ratti. 2007. Cellular census: Explorations in urban data collection. IEEE Perv. Comput. 6, 3 (2007), 30--38.

Digital Library

[49]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 91--99.

Digital Library

[50]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 3 (2015), 211--252.

Digital Library

[51]

Muhammad Saqib, Sultan Daud Khan, Nabin Sharma, and Michael Blumenstein. 2018. Person head detection in multiple scales using deep convolutional neural networks. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’18). IEEE, 1--7.

[52]

Xiang Sheng, Jian Tang, and Weiyi Zhang. 2012. Energy-efficient collaborative sensing with mobile phones. In IEEE International Conference on Computer Communications (INFOCOM'09). IEEE, Florida, USA, 1916--1924.

[53]

Wenhuan Shi, Shuhan Shen, and Yuncai Liu. 2009. Automatic generation of road network map from massive GPS, vehicle trajectories. In Proceedings of the 12th International IEEE Conference on Intelligent Transportation Systems (ITSC’09). IEEE, 1--6.

[54]

Claudio T. Silva, Juliana Freire, Fabio Miranda, Marcos Lage, Harish Doraiswamy, Maryam Hosseini, Eric K. Tokuda, Gabriel Ferreira, and Roberto M. Cesar Jr. 2019. Integrated Analytics and Visualization for Multi-modality Transportation Data. Technical Report. C2SMART Connected Cities for Smart Transportation.

[55]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, et al. 2015. Going deeper with convolutions. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’15).

[56]

Jing Tian, Jorg Hahner, Christian Becker, Illya Stepanov, and Kurt Rothermel. 2002. Graph-based mobility model for mobile ad hoc network simulation. In Proceedings of the 35th Annual Simulation Symposium. IEEE, 337--344.

[57]

Ben L. Titzer, Daniel K. Lee, and Jens Palsberg. 2005. Avrora: Scalable sensor network simulation with precise timing. In Proceedings of the International Symposium on Information Processing in Sensor Networks. IEEE, 477--482.

[58]

Eric K. Tokuda, Roberto M. Cesar, and Claudio T. Silva. 2019. Quantifying the presence of graffiti in urban environments. In Proceedings of the International Conference on Big Data and Smart Computing (BigComp’19). IEEE, 1--4.

[59]

United States Environment Protection Agency Air Data USEEPAAD. Retrieved from https://www3.epa.gov/airdata/ad_data_daily.html.

[60]

Carlos A. Vanegas, Daniel G. Aliaga, and Bedrich Benes. 2012. Automatic extraction of Manhattan-world building masses from 3D laser range scans. IEEE Trans. Vis. Comput. Graph 18, 10 (2012), 1627--1637.

Digital Library

[61]

Roberto Vezzani and Rita Cucchiara. 2010. Video surveillance online repository (ViSOR): An integrated framework. Multimedia Tools Applic. 50, 2 (2010), 359--380.

Digital Library

[62]

Meritxell Vinyals, Juan A. Rodriguez-Aguilar, and Jesus Cerquides. 2011. A survey on sensor networks from a multiagent perspective. Comput. J. 54, 3 (2011), 455--470.

Digital Library

[63]

Guiling Wang, Guohong Cao, and Tom LaPorta. 2003. A bidding protocol for deploying mobile sensors. In Proceedings of the International Conference on Network Protocols. IEEE, 315--324.

[64]

William H. Whyte. 2012. City: Rediscovering the Center. University of Pennsylvania Press, Philadelphia, PA.

[65]

Danny B. Yang, Leonidas J. Guibas, et al. 2003. Counting people in crowds with a real-time network of simple image sensors. In Proceedings of the International Conference on Computer Vision Workshops. IEEE, 122.

[66]

Eyasu Zemene, Yonatan Tariku Tesfaye, Haroon Idrees, Andrea Prati, Marcello Pelillo, and Mubarak Shah. 2018. Large-scale image geo-localization using dominant sets. IEEE Trans. Pattern Anal.Mach. Intell. 41, 1 (2018), 148--161.

Digital Library

[67]

Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. 2014. Urban computing: Concepts, methodologies, and applications. ACM Trans. Intell. Syst. Technol. 5, 3 (2014), 38.

Digital Library

[68]

Xiaojin Jerry Zhu. 2005. Semi-supervised Learning Literature Survey. Technical Report. University of Wisconsin-Madison Department of Computer Sciences.

Cited By

Oda TYoshimura Y(2024)Quantifying the vibrancy of streets: Large-scale pedestrian density estimation with dashcam dataTransportation Research Part C: Emerging Technologies10.1016/j.trc.2024.104840167(104840)Online publication date: Oct-2024
https://doi.org/10.1016/j.trc.2024.104840
Karagulian FLiberto CCorazza MValenti GDumitru ANigro M(2023)Pedestrian Flows Characterization and Estimation with Computer Vision TechniquesUrban Science10.3390/urbansci70200657:2(65)Online publication date: 14-Jun-2023
https://doi.org/10.3390/urbansci7020065
Schuhback SWischhof LOtt J(2023)Cellular Sidelink Enabled Decentralized Pedestrian SensingIEEE Access10.1109/ACCESS.2023.324294611(13349-13369)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3242946
Show More Cited By

Index Terms

A New Approach for Pedestrian Density Estimation Using Moving Sensors and Computer Vision
1. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

A Conceptual Structure for Computer Vision
CRV '11: Proceedings of the 2011 Canadian Conference on Computer and Robot Vision

The research presented in this paper represents several novel conceptual contributions to the computer vision literature. In this position paper, our goal is to define the scope of computer vision analysis and discuss a new categorisation of the ...
High Level Computer Vision Using OpenCV
SIBGRAPI-T '11: Proceedings of the 2011 24th SIBGRAPI Conference on Graphics, Patterns, and Images Tutorials

This paper presents some more advanced topics in image processing and computer vision, such as Principal Components Analysis, Matching Techniques, Machine Learning Techniques, Tracking and Optical Flow and Parallel Computer Vision using CUDA. These ...
Detecting advertising on building façades with computer vision
Abstract
Outdoor advertising influences the visual environment of any modern city. Advertising and information signs on building façades are one of the types of outdoor advertising. As a rule, there are laws and design codes in cities that define ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Spatial Algorithms and Systems

ACM Transactions on Spatial Algorithms and Systems Volume 6, Issue 4

December 2020

185 pages

ISSN:2374-0353

EISSN:2374-0361

DOI:10.1145/3404105

Editor:
Walid G. Aref
Purdue University, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2020

Online AM: 07 May 2020

Accepted: 01 April 2020

Revised: 01 September 2019

Received: 01 September 2018

Published in TSAS Volume 6, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

CNPq and CAPES
NSF
FAPESP
DARPA D3M program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
359
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)2

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Oda TYoshimura Y(2024)Quantifying the vibrancy of streets: Large-scale pedestrian density estimation with dashcam dataTransportation Research Part C: Emerging Technologies10.1016/j.trc.2024.104840167(104840)Online publication date: Oct-2024
https://doi.org/10.1016/j.trc.2024.104840
Karagulian FLiberto CCorazza MValenti GDumitru ANigro M(2023)Pedestrian Flows Characterization and Estimation with Computer Vision TechniquesUrban Science10.3390/urbansci70200657:2(65)Online publication date: 14-Jun-2023
https://doi.org/10.3390/urbansci7020065
Schuhback SWischhof LOtt J(2023)Cellular Sidelink Enabled Decentralized Pedestrian SensingIEEE Access10.1109/ACCESS.2023.324294611(13349-13369)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3242946
Middya ARoy S(2021)Spatial Interpolation Techniques on Participatory Sensing DataACM Transactions on Spatial Algorithms and Systems10.1145/34576097:3(1-32)Online publication date: 8-Jun-2021
https://dl.acm.org/doi/10.1145/3457609
Li SRen WWang FAraujo ITokuda EJunior RCesar-Jr. RWang ZCao X(2021)A Comprehensive Benchmark Analysis of Single Image Deraining: Current Challenges and Future PerspectivesInternational Journal of Computer Vision10.1007/s11263-020-01416-w129:4(1301-1322)Online publication date: 1-Apr-2021
https://dl.acm.org/doi/10.1007/s11263-020-01416-w

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents