Abstract
We present a unified framework for separating specular and diffuse reflection components in images and videos of textured scenes. This can be used for specularity removal and for independently processing, filtering, and recombining the two components. Beginning with a partial separation provided by an illumination-dependent color space, the challenge is to complete the separation using spatio-temporal information. This is accomplished by evolving a partial differential equation (PDE) that iteratively erodes the specular component at each pixel. A family of PDEs appropriate for differing image sources (still images vs. videos), differing prior information (e.g., highly vs. lightly textured scenes), or differing prior computations (e.g., optical flow) is introduced. In contrast to many other methods, explicit segmentation and/or manual intervention are not required. We present results on high-quality images and video acquired in the laboratory in addition to images taken from the Internet. Results on the latter demonstrate robustness to low dynamic range, JPEG artifacts, and lack of knowledge of illuminant color. Empirical comparison to physical removal of specularities using polarization is provided. Finally, an application termed dichromatic editing is presented in which the diffuse and the specular components are processed independently to produce a variety of visual effects.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Bajcsy, R., Lee, S., Leonardis, A.: Detection of diffuse and specular interface reflections and inter-reflections by color image segmentation. IJCV 17(3), 241–272 (1996)
Blake, A., Brelstaff, G.: Geometry from specularities. In: ICCV, pp. 394–403 (1988)
Brockett, R., Maragos, P.: Evolution equations for continuous scale morphology. IEEE trans. on Sig. Proc. 42, 3377–3386 (1994)
Crandall, M.G., Ishii, H., Lions, P.L.: User’s Guide to Viscosity Solutions of Second Order Partial Differential Equations, July 1992, vol. 26 (1992)
Healey, G., Binford, T.O.: Local shape from specularity. CVGIP 42(1), 62–86 (1988)
Klinker, G., Shafer, S., Kanade, T.: The measurement of highlights in color images. IJCV 2(1), 7–32 (1988)
Lee, H.S.: Method for computing the scene-illuminant chromaticity from specular highlights. JOSAA 3(10), 1694–1699 (1986)
Mallick, S., Zickler, T., Kriegman, D., Belhumeur, P.: Beyond Lambert: Reconstructing specular surfaces using color. In: CVPR, pp. II:619–626 (2005)
Maragos, P., Butt, M.A.: Curve evolution and differential morphology. Fundamenta Informaticae 41, 91–129 (2000)
Nayar, S., Fang, X., Boult, T.: Separation of reflection components using color and polarization. Int. Journal of Computer Vision 21(3), 163–186 (1997)
Osadchy, M., Jacobs, D., Ramamoorthi, R.: Using specularities for recognition. In: ICCV, pp. 1512–1519 (2003)
Park, J.B.: Efficient color representation for image segmentation under nonwhite illumination. In: SPIE, vol. 5267, pp. 163–174 (2003)
Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. PAMI 12(7), 629–639 (1990)
Ping, T., Lin, S., Quan, L., Shum, H.-Y.: Highlight removal by illumination-constrained inpainting. In: ICCV, Nice, France, pp. 164–169 (2003)
Shafer, S.: Using color to separate reflection components. COLOR research and applications 10(4), 210–218 (1985)
Tan, R., Ikeuchi, K.: Reflection components decomposition of textured surfaces using linear basis functions. In: CVPR, pp. I:125–131 (2005)
Tan, R., Ikeuchi, K.: Separating reflection components of textured surfaces using a single image. PAMI 27(2), 178–193 (2005)
Tan, R.T., Nishino, K., Ikeuchi, K.: Separating reflection components based on chromaticity and noise analysis. PAMI 26(10), 1373–1381 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mallick, S.P., Zickler, T., Belhumeur, P.N., Kriegman, D.J. (2006). Specularity Removal in Images and Videos: A PDE Approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744023_43
Download citation
DOI: https://doi.org/10.1007/11744023_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33832-1
Online ISBN: 978-3-540-33833-8
eBook Packages: Computer ScienceComputer Science (R0)