Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

A Survey and Comparison of Discrete and Continuous Multi-label Optimization Approaches for the Potts Model

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a survey and a comparison of a variety of algorithms that have been proposed over the years to minimize multi-label optimization problems based on the Potts model. Discrete approaches based on Markov Random Fields as well as continuous optimization approaches based on partial differential equations can be applied to the task. In contrast to the case of binary labeling, the multi-label problem is known to be NP hard and thus one can only expect near-optimal solutions. In this paper, we carry out a theoretical comparison and an experimental analysis of existing approaches with respect to accuracy, optimality and runtime, aimed at bringing out the advantages and short-comings of the respective algorithms. Systematic quantitative comparison is done on the Graz interactive image segmentation benchmark. This paper thereby generalizes a previous experimental comparison (Klodt et al. 2008) from the binary to the multi-label case.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. In Komodakis and Tziritas (2005) several algorithms are proposed for different choices of parameters \(h_i\). In this paper we use the \(\alpha \)-expansion equivalence of FastPD (called \(\text{ PD2 }_{\mu = 1}\) by Komodakis and Tziritas) since it corresponds to the Potts model.

  2. http://vision.csd.uwo.ca/code/

  3. http://www.csd.uoc.gr/~komod/FastPD/

References

  • Alahari, K., Kohli, P., & Torr, P. H. S. (2010). Dynamic hybrid algorithms for MAP inference in discrete MRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(10), 1846–1857.

    Article  Google Scholar 

  • Batra, D. (2011). Making the right moves: Guiding alpha-expansion using local primal-dual gaps. In P. Kohli (Ed.), International Conference on Computer Vision and Pattern Recognition, Colorado Springs.

  • Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of Royal Statistical Society Series B, 48(3), 259–302.

    MathSciNet  MATH  Google Scholar 

  • Boykov, Y., & Kolmogorov, V. (2004). An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(9), 1124–1137.

    Article  Google Scholar 

  • Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1222–1239.

    Article  Google Scholar 

  • Chambolle, A., Cremers, D., Pock, T. (2008). A convex approach for computing minimal partitions. Technical Report TR-2008-05. Bonn: University of Bonn.

  • Chan, T., & Esedoglu, S., & Nikolova, M., (2006). Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5), 1632–1648.

    Google Scholar 

  • Chan, T., & Vese, L. (2001). Active contours without edges. IEEE Transactions on Image Processing, 10(2), 266–277.

    Article  MATH  Google Scholar 

  • Cremers, D., Rousson, M., & Deriche, R. (2007). A review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape. International Journal of Computer Vision, 72(2), 195–215.

    Article  Google Scholar 

  • Cremers, D., Sochen, N., & Schnörr, C. (2004). Multiphase dynamic labeling for variational recognition-driven image segmentation. In T. Pajdla & V. Hlavac (Eds.), European Conference on Computer Vision volume of 3024 LNCS (pp. 74–86). New York: Springer.

  • Cremers, D., Sochen, N., & Schnörr, C. (2006). A multiphase dynamic labeling model for variational recognition-driven image segmentation. International Journal of Computer Vision, 66(1), 67–81.

    Google Scholar 

  • Felzenszwalb, P., & Veksler, O. (2010). Tiered scene labeling with dynamic programming. In International Conference on Computer Vision and Pattern Recognition, San Francisco.

  • Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721–741.

    Article  MATH  Google Scholar 

  • Goldschlager, L., Shaw, R., & Staples, J. (1982). The maximum flow problem is log space complete for P. Theoretical Computer Science, 21, 105–111.

    Article  MathSciNet  MATH  Google Scholar 

  • Greig, D. M., Porteous, B. T., & Seheult, A. H. (1989). Exact maximum a posteriori estimation for binary images. Journal of Royal Statistical Society Series B, 51(2), 271–279.

    Google Scholar 

  • Ishikawa, H. (2003). Exact optimization for Markov random fields with convex priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1333–1336.

    Article  Google Scholar 

  • Kleinberg, J., & Tardos, E. (2002). Approximation algorithms for classification problems with pairwise relationships: Metric labeling and Markov random fields. Journal of the ACM, 49(5), 672–713 .

    Google Scholar 

  • Klodt, M., Schoenemann, T., Kolev, K., Schikora, M., & Cremers, D. (2008). An experimental comparison of discrete and continuous shape optimization methods. In European Conference on Computer Vision, Marseille, France.

  • Kolev, K., Pock, T., & Cremers, D. (2010). Anisotropic minimal surfaces integrating photoconsistency and normal information for multiview stereo. In European Conference on Computer Vision, Crete.

  • Kolmogorov, V. (2006). Convergent tree-reweighted message passing for energy minimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 1568–1583.

    Article  Google Scholar 

  • Komodakis, N., & Tziritas, G. (2005). A new framework for approximate labeling via graph cuts. In IEEE International Conference on Computer Vision, New Orleans.

  • Komodakis, N., Tziritas, G., & Paragios, N. (2007). Fast, approximately optimal solutions for single and dynamic MRFs. In International Conference on Computer Vision and Pattern Recognition, Ezhou.

  • Lellmann, J., Becker, F., & Schnörr, C. (2009). Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In IEEE International Conference on Computer Vision (pp. 646–653).

  • Lellmann, J., Kappes, J. H., Yuan, J., Becker, F., & Schnörr, C. (2009). Convex multi-class image labeling by simplex-constrained total variation. Scale Space and Variational Methods in Computer Vision (SSVM), 5567, 150–162.

    Article  Google Scholar 

  • Lellmann, J., Lenzen, F., & Schnörr, C. (2011). Optimality bounds for a variational relaxation of the image partitioning problem. In International Conference on Energy Minimization Methods for Computer Vision and Pattern Recognition. New York: Springer.

  • Lempitsky, V., Rother, C., & Blake, A. (2007). Logcut: Efficient graph cut optimization for Markov random fields. In IEEE International Conference on Computer Vision.

  • Liu, X., Veksler, O., & Samarabandu, J. (2010). Order preserving moves for graph cut based optimization. IEEE Transaction on Pattern Analysis and Machine Intellignece, 32(7), 1317–1324.

    Google Scholar 

  • Michelot, C. (1986). A finite algorithm for finding the projection of a point onto the canonical simplex of \(R^n\). Journal of Optimization Theory and Applications, 50(1), 189–193.

    Google Scholar 

  • Mumford, D., & Shah, J. (1989). Optimal approximation by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42, 577–685.

    Article  MathSciNet  MATH  Google Scholar 

  • Nieuwenhuis, C., & Cremers, D. (2012). Spatially varying color distributions for interactive multi-label segmentation. In IEEE Transactions on Pattern Analysis and Machine Intelligence, Philadelphia.

  • Nieuwenhuis, C., & Töppe, E., & Cremers, D. (2011). Space-varying color distributions for interactive multiregion segmentation: Discrete versus continuous approaches. In International Conference on Energy Minimization Methods for Computer Vision and Pattern Recognition, New York.

  • Osokin, A., Vetrov, D., & Kolmogorov, V. (2011). Submodular decomposition framework for inference in associative markov networks with global constraints. In International Conference on Computer Vision and Pattern Recognition, St. Petersburg.

  • Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Mateo: Morgan Kauffmann.

    Google Scholar 

  • Pock, T., & Chambolle, A. (2011). Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In IEEE International Conference on Computer Vision, Barcelona.

  • Pock, T., Cremers, D., Bischof, H., & Chambolle, A. (2009). An algorithm for minimizing the piecewise smooth Mumford-Shah functional. In IEEE International Conference on Computer Vision, Kyoto.

  • Pock, T., Cremers, D., Bischof, H., & Chambolle, A. (2010). Global solutions of variational models with convex regularization. SIAM Journal of Imaging Sciences, 3(4), 1122–1145.

    Article  MathSciNet  MATH  Google Scholar 

  • Santner, J. (2010). Interactive multi-label segmentation. Ph.D. thesis, University of Graz, Graz.

  • Schlesinger, M. I. (1976). Sintaksicheskiy analiz dvumernykh zritelnikh signalov v usloviyakh pomekh (syntactic analysis of two-dimensional visual signals in noisy conditions). Kibernetika, 4, 113–130. (in Russian).

    Google Scholar 

  • Strekalovskiy, E., & Cremers, D. (2011). Generalized ordering constraints for multilabel optimization. In IEEE International Conference on Computer Vision, Barcelona.

  • Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., & Rother, C. (2006). A comparative study of energy minimization methods for Markov random fields. In European Conference on Computer Vision, volume 3952 of, Lecture Notes in Computer Science, Graz (pp. 16–29).

  • Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., & Fan, A., et al. (2001). Model-based curve evolution technique for image segmentation. In Computer Vision Pattern Recognition, Kauai, Hawaii (pp. 463–468).

  • Veksler, O. (2007). Graph cut based optimization for MRFs with truncated convex priors. In International Conference on Computer Vision and Pattern Recognition, Beijing.

  • Veksler, O. (2009). Multi-label moves for MRFs with truncated convex priors. In International Conference on Energy Minimization Methods for Computer Vision and Pattern Recognition, Bonn.

  • Wainwright, M., Jaakkola, T., & Willsky, A. (2005). Map estimation via agreement on (hyper)trees: Message-passing and linear-programming approaches. IEEE Transactions on Information Theory, 51, 3697–3717.

    Google Scholar 

  • Werner, T. (2007). A linear programming approach to maxsum problem: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(7), 1165–1179.

    Google Scholar 

  • Zach, C., Gallup, D., Frahm, J. M., & Niethammer, M. (2008). Fast global labeling for real-time stereo using multiple plane sweeps. In Vision modeling and visualization workshop (VMV), Konstanz.

  • Zach, C., Häne, C., & Pollefeys, M. (2012). What is optimized in tight convex relaxations for multi-label problems? In International Conference on Computer Vision and Pattern Recognition, Lund Sweden.

  • Zach, C., Niethammer, M., & Frahm, J. M. (2009). Continuous maximal flows and Wulff shapes: Application to MRFs. In International Conference on Computer Vision and Pattern Recognition, Bonn.

Download references

Acknowledgments

We thank Vladimir Kolmogorov and Evgeny Strekalovskiy for fruitful discussions on the relations among various relaxations.

Author information

Authors and Affiliations

  1. Departments of Computer Science and Mathematics, TU München, Germany

    Claudia Nieuwenhuis, Eno Töppe & Daniel Cremers

Authors
  1. Claudia Nieuwenhuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eno Töppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Nieuwenhuis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nieuwenhuis, C., Töppe, E. & Cremers, D. A Survey and Comparison of Discrete and Continuous Multi-label Optimization Approaches for the Potts Model. Int J Comput Vis 104, 223–240 (2013). https://doi.org/10.1007/s11263-013-0619-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0619-y

Keywords

Search

Navigation