Abstract
Inferring scene geometry from a sequence of camera images is one of the central problems in computer vision. While the overwhelming majority of related research focuses on diffuse surface models, there are cases when this is not a viable assumption: in many industrial applications, one has to deal with metal or coated surfaces exhibiting a strong specular behavior. We propose a novel and generalized constrained gradient descent method to determine the shape of a purely specular object from the reflection of a calibrated scene and additional data required to find a unique solution. This data is exemplarily provided by optical flow measurements obtained by small scale motion of the specular object, with camera and scene remaining stationary. We present a non-approximative general forward model to predict the optical flow of specular surfaces, covering rigid body motion as well as elastic deformation, and allowing for a characterization of problematic points. We demonstrate the applicability of our method by numerical experiments on synthetic and real data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aubert, G., Barlaud, M., Faugeras, O., & Jehan-Besson, S. (2003). Image segmentation using active contours: Calculus of variations or shape gradients? SIAM Journal on Applied Mathematics, 63(6), 2128–2154.
Balzer, J., Werling, S., & Beyerer, J. (2006). Regularization of the deflectometry problem using shading data. In Proceedings of the SPIE optics east.
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77.
Bonfort, T. (2006). Reconstruction de surfaces réfléchissantes à partir d’images. Ph.D. thesis, Institut National Polytechnique de Grenoble.
Bonfort, T., & Sturm, P. (2003). Voxel carving for specular surfaces. In ICCV (Vol. 1, pp. 591–596).
Burger, M. (2003). A framework for the construction of level set methods for shape optimization and reconstruction. Interfaces and Free Boundaries, 5, 301–329.
Charpiat, G., Keriven, R., Pons, J.-P., & Faugeras, O. (2005). Designing spatially coherent minimizing flows for variational problems based on active contours. In ICCV (Vol. 2, pp. 1403–1408).
Chen, M., & Arvo, J. (2000). Theory and application of specular path perturbation. ACM Transactions on Graphics, 19(1), 246–278.
Delfour, M., & Zolesio, J.-P. (2001). Shapes and geometries. Philadelphia: SIAM.
do Carmo, M. (1976). Differential geometry of curves and surfaces. Prentice-Hall: Englewood Cliffs.
Goldlücke, M., & Magnor, M. (2004). Weighted minimal hypersurfaces and their applications in computer vision. In ECCV (Vol. 2, pp. 366–378).
Halstead, M., Barsky, B., Klein, S., & Mandell, R. (1996). Reconstructing curved surfaces from specular reflection patterns using spline surface fitting of normals. In SIGGRAPH ’96 (pp. 335–342). New York: ACM Press.
Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.
Hicks, R. A., & Perline, R. K. (2004) The method of vector fields for catadioptric sensor design with applications to panoramic imaging. In CVPR (Vol. 2, pp. 143–150), Jun, 2004.
Horn, B. (1970). Shape from Shading: A method for obtaining the shape of a smooth opaque object from one view. Ph.D. thesis, Department of Electrical Engineering, MIT.
Horn, B. (1986). Robot vision. Cambridge: MIT Press.
Jehan-Besson, S., Herbulot, A., Barlaud, M., & Aubert, G. (2006). Handbook of mathematical models in computer vision. New York: Springer (Chap. 19, pp. 309–323).
Jin, H. (2003). Estimation of 3d surface shape and smooth radiance from 2d images: A level set approach. Journal of Scientific Computing, 19(1–3), 267–292.
Kammel, S. (2004). Deflektometrische untersuchung spiegelnd reflektierender freiformflächen. Ph.D. thesis, University of Karlsruhe.
Kickingereder, R., & Donner, K. (2004). Stereo vision on specular surfaces. In Proceedings of IASTED conference on visualization, imaging, and image processing (pp. 335–339).
Lellmann, J. (2006). Mathematical modelling and analysis of the deflectometry problem: level set based reconstruction of specular free-form surfaces from optical flow. German title: Mathematische Modellierung und Analyse des Deflektometrieproblems: Rekonstruktion reflektierender Freiformflächen anhand des optischen Flusses auf der Grundlage von Level Sets. Diploma thesis, University of Karlsruhe.
Oren, M., & Nayar, S. K. (1995). A theory of specular surface geometry. In ICCV (pp. 740–747).
Osher, S., & Fedkiw, R. (2002). Level set methods and dynamic implicit surfaces. New York: Springer.
Prados, E., Camilli, F., & Faugeras, O. (2007). A viscosity solution method for shape-from-shading without image boundary data. Mathematical Modelling and Numerical Analysis, 40(2), 393–412.
Roth, S., & Black, M. (2006). Specular flow and the recovery of surface structure. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR) (Vol. 2, pp. 1869–1876).
Rozenfeld, S., Shimshoni, I., & Lindenbaum, M. (2007). Dense mirroring surface recovery from 1d homographies and sparse correspondences. In Proceedings of the international conference on CVPR, Minneapolis.
Savarese, S., Chen, M., & Perona, P. (2005). Local shape from mirror reflections. International Journal of Computer Vision, 64(1), 31–67.
Sethian, J. (2005). Level set methods and fast marching methods (2nd ed.). Cambridge: Cambridge University Press.
Solem, J., & Overgaard, N. (2005a). A gradient descent procedure for variational dynamic surface problems with constraints. In Proceedings of the workshop on variational, geometric, and level set methods in computer vision (pp. 332–343).
Solem, J., & Overgaard, N. (2005b). A geometric formulation of gradient descent for variational problems with moving surfaces. In Scale-space (pp. 419–430).
Solem, J., Aanaes, H., & Heyden, A. (2004). A variational analysis of shape from specularities using sparse data. In 3DPVT (pp. 26–33).
Werling, S., Balzer, J., & Beyerer, J. (2007). Initial value estimation for robust deflectometric reconstruction. In Proceedings of 8th international conference on optical 3-d measurement techniques, Zurich.
Zissermann, A., Giblin, P., & Blake, A. (1989). The information available to a moving observer from specularities. Image and Vision Computing, 7, 38–42.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lellmann, J., Balzer, J., Rieder, A. et al. Shape from Specular Reflection and Optical Flow. Int J Comput Vis 80, 226–241 (2008). https://doi.org/10.1007/s11263-007-0123-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0123-3