Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Simultaneous Motion Detection and Background Reconstruction with a Conditional Mixed-State Markov Random Field

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this work we present a new way of simultaneously solving the problems of motion detection and background image reconstruction. An accurate estimation of the background is only possible if we locate the moving objects. Meanwhile, a correct motion detection is achieved if we have a good available background model. The key of our joint approach is to define a single random process that can take two types of values, instead of defining two different processes, one symbolic (motion detection) and one numeric (background intensity estimation). It thus allows to exploit the (spatio-temporal) interaction between a decision (motion detection) and an estimation (intensity reconstruction) problem. Consequently, the meaning of solving both tasks jointly, is to obtain a single optimal estimate of such a process. The intrinsic interaction and simultaneity between both problems is shown to be better modeled within the so-called mixed-state statistical framework, which is extended here to account for symbolic states and conditional random fields. Experiments on real sequences and comparisons with existing motion detection methods support our proposal. Further implications for video sequence inpainting will be also discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Benboudjema, D., & Pieczynski, W. (2007). Unsupervised statistical segmentation of nonstationary images using triplet Markov fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1367–1378.

    Article  Google Scholar 

  • Benedek, C., Sziranyi, T., Kato, Z., & Zerubia, J. (2007). A multi-layer Mrf model for object-motion detection in unregistered airborne image-pairs. In IEEE international conference on image processing, 2007 (ICIP07), 16 2007–Oct. 19 2007 (Vol. 6, pp. 141–144).

    Google Scholar 

  • Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society. Series B, 36, 192–236.

    MathSciNet  MATH  Google Scholar 

  • Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society. Series B, 48(3), 259–302.

    MathSciNet  MATH  Google Scholar 

  • Black, M. J., & Rangarajan, A. (1996). On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision, 19(1), 57–91.

    Article  Google Scholar 

  • Blanchet, J., & Forbes, F. (2008). Triplet Markov fields for the classification of complex structure data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(6), 1055–1067.

    Article  Google Scholar 

  • Bouthemy, P., & Lalande, P. (1993). Recovery of moving object masks in an image sequence using local spatiotemporal contextual information. Optical Engineering, 32(6), 1205–1212.

    Article  Google Scholar 

  • Bouthemy, P., Hardouin, Ch., Piriou, G., & Yao, J.-F. (2006). Mixed-state auto-models and motion texture modeling. Journal of Mathematical Imaging and Vision, 25(3), 387–402.

    Article  MathSciNet  Google Scholar 

  • Bugeau, A., & Pérez, P. (2007). Detection and segmentation of moving objects in highly dynamic scenes. In CVPR ’07: proc. of the 2007 IEEE conf. on computer vision and pattern recognition, Minneapolis, MI.

    Google Scholar 

  • Caillol, H., Hillion, A., & Pieczynski, W. (1993). Fuzzy random fields and unsupervised image segmentation. IEEE Transactions on Geoscience and Remote Sensing, 31, 801–810.

    Article  Google Scholar 

  • Carincotte, C., Derrode, S., Sicot, G., & Boucher, J. M. (2004). Unsupervised image segmentation based on a new fuzzy hmc model. In ICASSP’04 (pp. 17–21).

    Google Scholar 

  • Carincotte, C., Derrode, S., & Bourennane, S. (2006). Unsupervised change detection on sar images using fuzzy hidden Markov chains. IEEE Transactions on Geoscience and Remote Sensing, 44(2), 432–441.

    Article  Google Scholar 

  • Cernuschi-Frias, B. (2007). Mixed states Markov random fields with symbolic labels and multidimensional real values. Technical Report 6255, INRIA, July 2007.

  • Chellappa, R. (1985). Two-dimensional discrete Gaussian Markov random field models for image processing. In Progress in pattern recognition 2 (Vol. 85, pp. 79–112).

    Google Scholar 

  • Chen, J., & Tang, C. (2007). Spatio-temporal Markov random field for video denoising. In Proc. IEEE conf. on comp. vision and pattern recognition (CVPR’07), June (pp. 1–8).

    Google Scholar 

  • Collet, C., & Murtagh, F. (2004). Segmentation based on a hierarchical Markov model. Pattern Recognition, 37(12), 2337–2347.

    Article  Google Scholar 

  • Criminisi, A., Cross, G., Blake, A., & Kolmogorov, V. (2006). Bilayer segmentation of live video. In CVPR ’06: proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 53–60). Washington: IEEE Computer Society.

    Google Scholar 

  • Crivelli, T., Cernuschi-Frias, B., Bouthemy, P., & Yao, J.-F. (2006). Mixed-state Markov random fields for motion texture modeling and segmentation. In Proc. IEEE int. conf. on image processing (ICIP’06), Atlanta, USA (pp. 1857–1860).

    Google Scholar 

  • Crivelli, T., Piriou, G., Bouthemy, P., Cernuschi-Frías, B., & Yao, J.-F. (2008). Simultaneous motion detection and background reconstruction with a mixed-state conditional Markov random field. In ECCV ’08: proceedings of the 10th European conference on computer vision, Marseille, France (pp. 113–126).

    Google Scholar 

  • Crivelli, T., Bouthemy, P., Cernuschi-Frías, B., & Yao, J.-F. (2009). Learning mixed-state Markov models for statistical motion texture tracking. In MLVMA’09: 2nd IEEE int. workshop on machine learning for vision-based motion analysis, Kyoto, Japan.

    Google Scholar 

  • Elfadel, I., & Picard, R. (1994). Gibbs random fields, cooccurrences, and texture modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1), 24–37.

    Article  Google Scholar 

  • Elgammal, A. M., Harwood, D., & Davis, L. S. (2000). Non-parametric model for background subtraction. In ECCV ’00: proc. of the 6th European conf. on comp. vision-part II, London, UK (pp. 751–767).

    Google Scholar 

  • Fablet, R., & Bouthemy, P. (2003). Motion recognition using non-parametric image motion models estimated from temporal and multiscale co-occurrence statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1619–1624.

    Article  Google Scholar 

  • Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.

    Article  MATH  Google Scholar 

  • Guyon, X. (1995). Random fields on a network: modeling, statistics and applications. New York: Springer.

    MATH  Google Scholar 

  • Hardouin, C., & Yao, J. (2008). Multi-parameter auto-models and their applications. Biometrika, 95(2), 335–349.

    Article  MathSciNet  MATH  Google Scholar 

  • Heitz, F., & Bouthemy, P. (1993). Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(12), 1217–1232.

    Article  Google Scholar 

  • Jojic, N., & Frey, B. J. (2001). Learning flexible sprites in video layers. In IEEE int. conference on computer vision and pattern recognition 2001 (pp. 199–206).

    Google Scholar 

  • Kasetkasem, T., & Varshney, P. K. (2002). An image change detection algorithm based on Markov random field models. IEEE Transactions on Geoscience and Remote Sensing, 40(8), 1815–1823.

    Article  Google Scholar 

  • Ko, T., Soatto, S., & Estrin, D. (2008). Background subtraction on distributions. In ECCV08 (pp. 276–289).

    Google Scholar 

  • Koller, D., Lerner, U., & Angelov, D. (1999). A general algorithm for approximate inference and its application to hybrid Bayes nets. In Proc. of the fifteenth conference on uncertainty in artificial intelligence, Stockholm, Sweden (pp. 324–333).

    Google Scholar 

  • Kumar, S., & Hebert, M. (2006). Discriminative random fields. International Journal of Computer Vision, 68(2), 179–201.

    Article  Google Scholar 

  • Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. 18th international conf. on machine learning, Williamstown, MA, USA (pp. 282–289).

    Google Scholar 

  • Li, Y., & Huttenlocher, D. P. (2008). Learning for optical flow using stochastic optimization. In ECCV ’08: proceedings of the 10th European conference on computer vision, Marseille, France (pp. 379–391).

    Google Scholar 

  • Lorette, A., Descombes, X., & Zerubia, J. (2000). Texture analysis through a Markovian modelling and fuzzy classification: Application to urban area extraction from satellite images. International Journal of Computer Vision, 36(3), 221–236.

    Article  Google Scholar 

  • Lu, W. L., Murphy, K. P., Little, J. J., Sheffer, A., & Fu, H. B. (2009). A hybrid conditional random field for estimating the underlying ground surface from airborne lidar data. IEEE Transactions on Geoscience and Remote Sensing, 47(8), 2913–2922.

    Article  Google Scholar 

  • Migdal, J., & Grimson, W. E. (2005). Background subtraction using Markov thresholds. In WACV-MOTION ’05: proceedings of the IEEE workshop on motion and video computing (pp. 58–65). Washington: IEEE Computer Society.

    Chapter  Google Scholar 

  • Mittal, A., & Paragios, N. (2004). Motion-based background subtraction using adaptive kernel density estimation. In CVPR ’04: proc. of the 2004 IEEE conf. on computer vision and pattern recognition (Vol. 2, pp. 302–309).

    Chapter  Google Scholar 

  • Monnet, A., Mittal, A., Paragios, N., & Visvanathan, R. (2003). Background modeling and subtraction of dynamic scenes. In Proc. of the ninth IEEE int. conf. on computer vision (Vol. 2, pp. 1305–1312).

    Chapter  Google Scholar 

  • Murphy, K. P. (1999). A variational approximation for Bayesian networks with discrete and continuous latent variables. In Uncertainty in artificial intelligence (Vol. 15, pp. 457–466). San Mateo: Morgan Kaufmann.

    Google Scholar 

  • Parag, T., Elgammal, A., & Mittal, A. (2006). A framework for feature selection for background subtraction. In CVPR ’06: proc. of the IEEE conf. on computer vision and pattern recognition, Washington, DC, USA (pp. 1916–1923).

    Google Scholar 

  • Pieczynski, W., & Tebbache, A. (2000). Pairwise Markov random fields and segmentation of textured images. In Machine graphics and vision (pp. 705–718).

    Google Scholar 

  • Salzenstein, F., & Collet, C. (2006). Fuzzy Markov random fields versus chains for multispectral image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11), 1753–1767.

    Article  Google Scholar 

  • Salzenstein, F., & Pieczynski, W. (1997). Parameter estimation in hidden fuzzy Markov random fields and image segmentation. Graphical Models and Image Processing, 59(4), 205–220.

    Article  Google Scholar 

  • Sheikh, Y., & Shah, M. (2005). Bayesian modeling of dynamic scenes for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(11), 1778–1792.

    Article  Google Scholar 

  • Stauffer, C., & Grimson, W. E. L. (2000). Learning patterns of activity using real-time tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 747–757.

    Article  Google Scholar 

  • Sun, J., Zhang, W., Tang, X., & Shum, H. (2006). Background cut. In Proc. European conf. comp. vision, ECCV 2006 (pp. 628–641).

    Chapter  Google Scholar 

  • Veit, Th., Cao, F., & Bouthemy, P. (2006). An a contrario decision framework for region-based motion detection. International Journal of Computer Vision, 68(2), 163–178.

    Article  Google Scholar 

  • Wang, T., Li, J., Diao, Q., Hu, W., Zhang, Y., & Dulong, C. (2006). Semantic event detection using conditional random fields. In CVPRW ’06: proceedings of the conference on computer vision and pattern recognition workshop, Washington, DC, USA.

    Google Scholar 

  • Wren, C. R., Azarbayejani, A., Darrell, T., & Pentland, A. P. (1997). Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 780–785.

    Article  Google Scholar 

  • Wright, J., Ganesh, A., Rao, S., & Ma, Y. (2009). Robust principal component analysis: exact recovery of corrupted low-rank matrices. In NIPS 2009. 0905.0233.

    Google Scholar 

  • Wu, J., & Chung, A. C. S. (2007). A segmentation model using compound Markov random fields based on a boundary model. IEEE Transactions on Image Processing, 16(1), 241–252.

    Article  MathSciNet  Google Scholar 

  • Zivkovic, Z., & van der Heijden, F. (2004). Recursive unsupervised learning of finite mixture models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(5), 651–656.

    Article  Google Scholar 

  • Zivkovic, Z., & van der Heijden, F. (2006). Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognition Letters, 27(7), 773–780.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Buenos Aires, Buenos Aires, Argentina

    Tomás Crivelli & Bruno Cernuschi-Frías

  2. INRIA Rennes, Campus de Beaulieu, 35042, Rennes, France

    Tomás Crivelli & Patrick Bouthemy

  3. CONICET, Buenos Aires, Argentina

    Bruno Cernuschi-Frías

  4. IRMAR University of Rennes 1, Rennes, France

    Jian-feng Yao

Authors
  1. Tomás Crivelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick Bouthemy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Cernuschi-Frías
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian-feng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomás Crivelli.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Crivelli, T., Bouthemy, P., Cernuschi-Frías, B. et al. Simultaneous Motion Detection and Background Reconstruction with a Conditional Mixed-State Markov Random Field. Int J Comput Vis 94, 295–316 (2011). https://doi.org/10.1007/s11263-011-0429-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-011-0429-z

Keywords

Search

Navigation