Abstract
Image segmentation is the task of partitioning a natural image into multiple contiguous regions, also known as segments, whereby adjacent regions are separated by salient edges or contours, and each region consists of pixels with homogeneous color or texture. In computer vision, this is widely accepted as a crucial step for any high-level vision tasks such as object recognition and understanding image semantics.
The whole is more than the sum of its parts.
—Aristotle
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Note that there is even ambiguity in segmentation done by different humans. In later sections, we will see how we could make such human-based evaluation somewhat meaningful.
- 2.
We will see how to incorporate pairwise information such as edges into such a simplified framework later. In particular, as we will see, such information can be incorporated through a special initialization to the segmentation algorithm.
- 3.
Another popular approach for constructing texture vectors is to use multivariate responses of a fixed 2D texture filter bank. A previous study by (Varma and Zisserman 2003) has argued that the difference in segmentation results between the two approaches is small, and yet it is more expensive to compute 2D filter bank responses.
- 4.
- 5.
For a large region with a sufficiently smooth boundary, the number of boundary-crossing windows is significantly smaller than the number of those in the interior. For boundary-crossing windows, their average coding length is roughly proportional to the number of pixels inside the region if the Gaussian distribution is sufficiently isotropic.
- 6.
We use the publicly available code for this method available at http://www.cs.sfu.ca/~mori/research/superpixels/ with parameter N_sp = 200.
- 7.
We will discuss several discrepancy measures in Section 10.4.2, such as the probabilistic Rand index (PRI) and variation of information (VOI).
- 8.
The quantitative performance of several existing algorithms was also evaluated in a recent work ((Arbelaez et al. 2009)), which was published roughly at the same time as this work. The reported results therein generally agree with our findings.
References
Arbelaez, P. (2006). Boundary extraction in natural images using ultrametric contour maps. In Workshop on Perceptual Organization in Computer Vision.
Arbelaez, P., Maire, M., Fowlkes, C., & Malik, J. (2009). From contours to regions: An empirical evaluation. In IEEE Conference on Computer Vision and Pattern Recognition.
Comanicu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 24, 603–619.
Cour, T., Benezit, F., & Shi, J. (2005). Spectral segmentation with multiscale graph decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Deng, Y., & Manjunath, B. (2001). Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8), 800–810.
Donoser, M., Urschler, M., Hirzer, M., & Bischof, H. (2009). Saliency driven total variation segmentation. In Proceedings of the International Conference on Computer Vision (ICCV).
Duda, R., Hart, P., & Stork, D. (2000). Pattern Classification (2nd ed.). Wiley, New York.
Efros, A. A., & Leung, T. K. (1999). Texture synthesis by non-parametric sampling. In IEEE International Conference on Computer Vision (pp. 1033–1038). Corfu, Greece.
Elder, J., & Zucker, S. (1996). Computing contour closures. In Proceedings of the European Conference on Computer Vision (ECCV).
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004). Efficient graph-based image segmentation. International Journal of Computer Vision (IJCV), 59(2), 167–181.
Freixenet, J., Munoz, X., Raba, D., Marti, J., & Cuff, X. (2002). Yet another survey on image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV).
Gevers, T., & Smeulders, A. (1997). Combining region splitting and edge detection through guided Delaunay image subdivision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Haralick, R., & Shapiro, L. (1985). Image segmentation techniques. Computer Vision, Graphics, and Image Processing, 29(1), 100–132.
Jain, A. (1989). Fundamentals of Digital Image Processing. Upper Saddle River: Prentice Hall.
Kim, J., Fisher, J., Yezzi, A., Cetin, M., & Willsky, A. (2005). A nonparametric statistical method for image segmentation using information theory and curve evolution. PAMI, 14(10), 1486–1502.
Kim, T., Lee, K., & Lee, S. (2010). Learning full pairwise affinities for spectral segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Kurita, T. (1995). An efficient clustering algorithm for region merging. IEICE Transactions of Information and Systems, E78-D(12), 1546–1551.
Levina, E., & Bickel, P. J. (2006). Texture synthesis and non-parametric resampling of random fields. Annals of Statistics, 34(4), 1751–1773.
Liu, Y. K., & Zalik, B. (2005). Efficient chain code with Huffman coding. Pattern Recognition, 38(4), 553–557.
Ma, Y., Derksen, H., Hong, W., & Wright, J. (2007). Segmentation of multivariate mixed data via lossy coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(9), 1546–1562.
Malik, J., Belongie, S., Leung, T., & Shi, J. (2001). Contour and texture analysis for image segmentation. International Journal of Computer Vision, 43(1), 7–27.
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In IEEE International Conference on Computer Vision.
Meila, M. (2005). Comparing clusterings: An axiomatic view. In Proceedings of the International Conference on Machine Learning.
Mobahi, H., Rao, S., Yang, A., & Sastry, S. (2011). Segmentation of natural images by texture and boundary compression. International Journal of Computer Vision, 95(1), 86–98.
Mori, G., Ren, X., Efros, A., & Malik, J. (2004). Recovering human body configurations: Combining segmentation and recognition. In IEEE Conference on Computer Vision and Pattern Recognition.
Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association, 66(336), 846–850.
Rao, S., Mobahi, H., Yang, A., & Sastry, S. (2009). Natural image segmentation with adaptive texture and boundary encoding. In Asian Conference on Computer Vision, 1 (pp. 135–146).
Ren, X., Fowlkes, C., & Malik, J. (2005). Scale-invariant contour completion using condition random fields. In IEEE International Conference on Computer Vision.
Ren, X., Fowlkes, C., & Malik, J. (2008). Learning probabilistic models for contour completion in natural images. International Journal of Computer Vision, 77, 47–63.
Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.
Tremeau, A., & Borel, N. (1997). A region growing and merging algorithm to color segmentation. Pattern Recognition, 30(7), 1191–1204.
Tu, Z., & Zhu, S. (2002). Image segmentation by data-driven Markov Chain Monte Carlo. PAMI, 24(5), 657–673.
Varma, M., & Zisserman, A. (2003). Texture classification: Are filter banks necessary? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Wang, J., Jia, Y., Hua, X., Zhang, C., & Quan, L. (2008a). Normalized tree partitioning for image segmentation. In IEEE Conference on Computer Vision and Pattern Recognition.
Yang, A., Wright, J., Ma, Y., & Sastry, S. (2008). Unsupervised segmentation of natural images via lossy data compression. Computer Vision and Image Understanding, 110(2), 212–225.
Yu, S. (2005). Segmentation induced by scale invariance. In IEEE Conference on Computer Vision and Pattern Recognition.
Zhu, Q., Song, G., & Shi, J. (2007). Untangling cycles for contour grouping. In Proceedings of the International Conference on Computer Vision (ICCV).
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag New York
About this chapter
Cite this chapter
Vidal, R., Ma, Y., Sastry, S.S. (2016). Image Segmentation. In: Generalized Principal Component Analysis. Interdisciplinary Applied Mathematics, vol 40. Springer, New York, NY. https://doi.org/10.1007/978-0-387-87811-9_10
Download citation
DOI: https://doi.org/10.1007/978-0-387-87811-9_10
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-87810-2
Online ISBN: 978-0-387-87811-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)