Abstract
In this chapter, we consider aspects of the crowd that can be modeled holistically, by analyzing global properties. We first discuss the dynamic texture model for representing holistic motion flow, which treats the video as a sample from a linear dynamical system. By defining appropriate distances and kernels between dynamic textures, crowd motion can be recognized with standard classification algorithms. Besides motion flow, crowd size, i.e., the number of objects within a crowd can also be modeled holistically. From a suitable set of low-level features, crowd counts can be estimated with a regression function that directly maps features into the number of objects within the crowd. In both cases, the surveillance task is solvable by analyzing global scene properties, and there is no need to detect or track individual objects. In result, the solutions tend to be robust even when the crowd is large, there are substantial occlusions, complex object interactions, or the objects are small.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Here we focus on the case where the initial state x 0 is fixed. More generally, the initial state could be distributed as a Gaussian, \(x_{1} \sim \mathcal{N}(\mu,S)\)
- 2.
One of these conditions is that the parameter n must be set to the true state-space dimension! Another condition is that the state noise and observation noise are realized from the same white noise process.
References
Ali, S., Shah, M.: A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2007)
Bach, F., Lanckriet, G., Jordan, M.: Multiple kernel learning, conic duality, and the SMO algorithm. In: International Conference on Machine Learning, ACM Press (2004)
Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., Werman, M.: Texture mixing and texture movie synthesis using statistical learning. IEEE Trans. Vis. Comput. Graph. 7(2), 120–135 (2001)
Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12, 43–77 (1994)
Bauer, D.: Comparing the CCA subspace method to pseudo maximum likelihood methods in the case of no exogenous inputs. J. Time Ser. Anal. 26, 631–668 (2005)
Bissacco, A., Chiuso, A., Ma, Y., Soatto, S.: Recognition of human gaits. In: IEEE Conference on Computer Vision and Pattern Recognition 20, IEEE (2001)
Brostow, G.J., Cipolla, R.: Unsupervised Bayesian detection of independent motion in crowds. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol 1, pp. 594–601 (2006)
Cetingul, E., Chaudhry, R., Vidal, R.: A system theoretic approach to synthesis and classification of lip articulation. In: International Workshop on Dynamical Vision, Springer LNCS (2007)
Chan, A.B.: Beyond dynamic textures: a family of stochastic dynamical models for video with applications to computer vision. PhD thesis, UCSD (2008)
Chan, A.B., Dong, D.: Generalized gaussian process models. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2011)
Chan, A.B., Vasconcelos, N.: Probabilistic kernels for the classification of auto-regressive visual processes. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 1, pp. 846–851 (2005)
Chan, A.B., Vasconcelos, N.: Classifying video with kernel dynamic textures. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2007)
Chan, A.B., Vasconcelos, N.: Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 909–926 (2008)
Chan, A.B., Vasconcelos, N.: Bayesian Poisson regression for crowd counting. In: IEEE International Conference on Computer Vision, IEEE (2009a)
Chan, A.B., Vasconcelos, N.: Layered dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell.: Spec. Issue Probab. Graph. Models Comput. Vis. 31(10), 1862–1879 (2009b)
Chan, A.B., Vasconcelos, N.: Variational layered dynamic textures. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2009c)
Chan, A., Vasconcelos, N.: Counting people with low-level features and Bayesian regression. IEEE Trans. Image Process. 21(4), 2160–2177 (2012)
Chan, A.B., Liang, Z.S.J., Vasconcelos, N.: Privacy preserving crowd monitoring: counting people without people models or tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2008)
Chan, A., Morrow, M., Vasconcelos, N.: Analysis of crowded scenes using holistic properties. In: 11th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS’09) (online) (2009)
Chan, A.B., Coviello, E., Lanckriet, G.R.G.: Clustering dynamic textures with the hierarchical EM algorithm. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2010a)
Chan, A.B., Mahadevan, V., Vasconcelos, N.: Generalized Stauffer-Grimson background subtraction for dynamic scenes. Mach. Vis. Appl. 22(5) 751–766 (2011)
Chaudry, R., Ravichandran, A., Hager, G., Vidal, R.: Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: IEEE International Conference on Computer Vision and Pattern Recognition, IEEE (2009)
Cho, S.Y., Chow, T.W.S., Leung, C.T.: A neural-based crowd estimation by hybrid global learning algorithm. IEEE Trans. Syst. Man Cybern. 29, 535–541 (1999)
Cock, K.D., Moor, B.D.: Subspace angles between linear stochastic models. In: IEEE Conference on Decision and Control, Proceedings, IEEE, pp. 1561–1566 (2000)
Cong, Y., Gong, H., Zhu, S.C., Tang, Y.: Flow mosaicking: real-time pedestrian counting without scene-specific learning. In: IEEE CVPR, IEEE (2009)
Cooper, L., Liu, J., Huang, K.: Spatial segmentation of temporal texture using mixture linear models. In: Dynamical Vision Workshop in the IEEE International Conference of Computer Vision, Springer LNCS (2005)
Costantini, R., Sbaiz, L., Süsstrunk, S.: Higher order SVD analysis for dynamic texture synthesis. IEEE Trans. Image Process. 17(1), 42–52 (2008)
Cover, T., Thomas, J.: Elements of Information Theory. Wiley, New York (1991)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 2, pp. 886–893 (2005)
Davies, A.C., Yin, J.H., Velastin, S.A.: Crowd monitoring using image processing. Electron. Commun. Eng. J. 7, 37–47 (1995)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977)
Dong, L., Parameswaran, V., Ramesh, V., Zoghlami, I.: Fast crowd segmentation using shape indexing. In: IEEE International Conference on Computer Vision, IEEE (2007)
Doretto, G., Soatto, S.: Dynamic shape and appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2006–2019 (2006)
Doretto, G., Chiuso, A., Wu, Y.N., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51(2), 91–109 (2003a)
Doretto, G., Cremers, D., Favaro, P., Soatto, S.: Dynamic texture segmentation. In: IEEE International Conference on Computer Vision, IEEE, vol. 2, pp. 1236–1242 (2003b)
Doretto, G., Jones, E., Soatto, S.: Spatially homogeneous dynamic textures. In: ECCV, Springer-Verlag LNCS 3021–3024 (2004)
Felzenszwalb, P., McAllester, D., Ramanan, D.: A discriminatively trained, multiscale, deformable part model. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2008)
Fitzgibbon, A.W.: Stochastic rigidity: image registration for nowhere-static scenes. In: IEEE International Conference on Computer Vision, IEEE, vol. 1, pp. 662–670 (2001)
Gelb, A.: Applied Optimal Estimation. MIT, Cambridge (1974)
Ghanem, B., Ahuja, N.: Phase based modelling of dynamic textures. In: IEEE Internationl Conference on Computer Vision, IEEE (2007)
Ghoreyshi, A., Vidal, R.: Segmenting dynamic textures with Ising descriptors, ARX models and level sets. In: Dynamical Vision Workshop in the European Conference on Computer Vision, Springer LNCS (2006)
Horn, B.K.P.: Robot Vision. McGraw-Hill, New York (1986)
Horn, B., Schunk, B.: Determining optical flow. Artif. Intell. 17, 185–204 (1981)
Hu, M., Ali, S., Shah, M.: Detecting global motion patterns in complex videos. In: IEEE International Conference on Pattern Recognition, IEEE (2008a)
Hu, M., Ali, S., Shah, M.: Learning motion patterns in crowded scenes using motion flow field. In: IEEE International Conference on Pattern Recognition, IEEE (2008b)
Isard, M., Blake, A.: Condensation – conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)
Kay, S.M.: Fundamentals of Statistical Signal Processing: Estimation Theory. Prentice-Hall, Upper Saddle River (1993)
Kong, D., Gray, D., Tao, H.: Counting pedestrians in crowds using viewpoint invariant training. In: British Machine Vision Conference, BMVA (2005)
Lanckriet, G., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)
Larimore, W.E.: Canonical variate analysis in identification, filtering, and adaptive control. In: IEEE Conference on Decision and Control, IEEE, vol. 2, pp. 596–604 (1990)
Leibe, B., Seemann, E., Schiele, B.: Pedestrian detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 1, pp. 875–885 (2005)
Leibe, B., Schindler, K., Van Gool, L.: Coupled detection and trajectory estimation for multi-object tracking. In: IEEE International Conference on Computer Vision, IEEE (2007)
Lempitsky, V., Zisserman, A.: Learning to count objects in images. In: Advances in Neural Information Processing Systems, NIPS (2010)
Lin, S.F., Chen, J.Y., Chao, H.X.: Estimation of number of people in crowded scenes using perspective transformation. IEEE Trans. Syst. Man Cybern. 31(6), 645–654 (2001)
Liu, C.B., Lin, R.S., Ahuja, N., Yang, M.H.: Dynamic texture synthesis as nonlinear manifold learning and traversing. In: British Machine Vision Conference, vol. 2, pp. 859–868. BMVA (2006)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceeding on DARPA Image Understanding Workshop, pp. 121–130. Morgan Kaufmann Publishers, (1981)
Mahadevan, V., Vasconcelos, N.: Spatiotemporal saliency in highly dynamic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 171–177 (2010)
Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2010)
Marana, A.N., Costa, L.F., Lotufo, R.A., Velastin, S.A.: On the efficacy of texture analysis for crowd monitoring. In: IEEE Proceedings of Computer Graphics, Image Processing, and Vision, IEEE, pp. 354–361 (1998)
Marana, A.N., Costa, L.F., Lotufo, R.A., Velastin, S.A.: Estimating crowd density with minkoski fractal dimension. In: IEEE Proceedings of International Conference Acoustics, Speech, Signal Processing, IEEE, vol. 6, pp. 3521–3524 (1999)
Martin, R.J.: A metric for ARMA processes. IEEE Trans. Signal Process. 48(4), 1164–1170 (2000)
Mehran, R., Oyama, A., Shah, M.: Abnormal crowd behavior detection using social force model. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2009)
Mehran, R., Moore, B., Shah, M.: A streakline representation of flow in crowded scenes. In: European Conference on Computer Vision, LNCS (2010)
Monnet, A., Mittal, A., Paragios, N., Ramesh, V.: Background modeling and subtraction of dynamic scenes. In: CVPR, IEEE (2003)
Overschee, P.V., Moor, B.D.: N4SID: subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 30, 75–93 (1994)
Paragios, N., Ramesh, V.: A MRF-based approach for real-time subway monitoring. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 1, pp. 1034–1040 (2001)
Polana, R., Nelson, R.C.: Recognition of motion from temporal texture. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, pp. 129–134 (1992)
Rabaud, V., Belongie, S.J.: Counting crowded moving objects. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2006)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT, Cambridge (2006)
Ravichandran, A., Vidal, R.: Video registration using dynamic textures. IEEE Trans. Pattern Anal. Mach. Intell. 33(1), pp. 158–171 (2011)
Ravichandran, A., Chaudhry, R., Vidal, R.: View-invariant dynamic texture recognition using a bag of dynamical systems. Video Registration using Dynamic Textures. In: IEEE International Conference on Computer Vision and Pattern Recognition, IEEE 33(1) 158–171 (2011)
Regazzoni, C.S., Tesei, A.: Distributed data fusion for real-time crowding estimation. Signal Process. 53, 47–63 (1996)
Saisan, P., Doretto, G., Wu, Y., Soatto, S.: Dynamic texture recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 2, pp. 58–63 (2001)
Saleemi, I., Hartung, L., Shah, M.: Scene understanding by statistical modeling of motion patterns. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE (2010)
Shumway, R.H., Stoffer, D.S.: An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal. 3(4), 253–264 (1982)
Siddiqi, S.M., Boots, B., Gordon, G.J.: A constraint generation approach to learning stable linear dynamical systems. In: Advances in Neural Information Processing Systems, NIPS (2007)
Szummer, M., Picard, R.: Temporal texture modeling. In: IEEE Conference on Image Processing, IEEE, vol. 3, pp. 823–826 (1996)
Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (1995)
Vidal, R.: Online clustering of moving hyperplanes. In: Neural Information and Processing Systems, NIPS (2006)
Vidal, R., Favaro, P.: Dynamicboost: boosting time series generated by dynamical systems. In: IEEE International Conference on Computer Vision, IEEE
Vidal, R., Ravichandran, A.: Optical flow estimation & segmentation of multiple moving dynamic textures. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 516–521 (2005)
Viola, P., Jones, M., Snow, D.: Detecting pedestrians using patterns of motion and appearance. Int. J. Comput. Vis. 63(2), 153–161 (2005)
Vishwanathan, S.V.N., Smola, A.J., Vidal, R.: Binet-cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes. Int. J. Comput. Vis. 73(1), 95–119 (2007)
Wang, J., Adelson, E.: Representing moving images with layers. IEEE Trans. Image Proc. 3(5), 625–638 (1994)
Washington State Department of Transportation. http://www.wsdot.wa.gov (2005)
Woolfe, F., Fitzgibbon, A.: Shift-invariant dynamic texture recognition. In: ECCV, Springer LNCS (2006)
Wu, B., Nevatia, R.: Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. In: IEEE International Conference on Computer Vision, IEEE, vol. 1, pp. 90–97 (2005)
Yang, Y., Liu, J., Shah, M.: Video scene understanding using multi-scale analysis. In: IEEE International Conference on Computer Vision, IEEE (2009)
Yuan, L., Wen, F., Liu, C., Shum, H.Y.: Synthesizing dynamic textures with closed-loop linear dynamic systems. In: European Conference on Computer Vision, pp. 603–616. Springer LNCS (2004)
Zhao, T., Nevatia, R.: Bayesian human segmentation in crowded situations. In: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, vol. 2, pp. 459–466 (2003)
Zhong, J., Sclaroff, S.: Segmenting foreground objects from a dynamic textured background via a robust Kalman filter. In: IEEE ICCV, IEEE (2003)
Acknowledgements
The authors wish to thank the Washington State DOT for the videos of highway traffic [85], Jeffrey Cuenco and Zhang-Sheng John Liang for annotating part of the pedestrian video data, Navneet Dalal and Pedro Felzenszwalb for the people detection algorithms [29, 37], and Piotr Dollar for running these algorithms. This work was supported by NSF CCF-0830535, IIS-0812235, IIS-0534985, NSF IGERT award DGE-0333451, and the Research Grants Council of the Hong Kong Special Administrative Region, China (CityU 110610).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Chan, A.B., Vasconcelos, N. (2013). Surveillance of Crowded Environments: Modeling the Crowd by Its Global Properties. In: Ali, S., Nishino, K., Manocha, D., Shah, M. (eds) Modeling, Simulation and Visual Analysis of Crowds. The International Series in Video Computing, vol 11. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8483-7_12
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8483-7_12
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8482-0
Online ISBN: 978-1-4614-8483-7
eBook Packages: Computer ScienceComputer Science (R0)