Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Joint Estimation of Shape and Reflectance using Multiple Images with Known Illumination Conditions

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We propose a generative model based method for recovering both the shape and the reflectance of the surface(s) of a scene from multiple images, assuming that illumination conditions and cameras calibration are known in advance. Based on a variational framework and via gradient descents, the algorithm minimizes simultaneously and consistently a global cost functional with respect to both shape and reflectance. The motivations for our approach are threefold. (1) Contrary to previous works which mainly consider specific individual scenarios, our method applies indiscriminately to a number of classical scenarios; in particular it works for classical stereovision, multiview photometric stereo and multiview shape from shading. It works with changing as well as static illumination. (2) Our approach naturally combines stereo, silhouette and shading cues in a single framework. (3) Moreover, unlike most previous methods dealing with only Lambertian surfaces, the proposed method considers general dichromatic surfaces. We verify the method using various synthetic and real data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Bhat, D. N., & Nayar, S. K. (1998). Stereo and specular reflection. International Journal of Computer Vision, 26(2), 91–106.

    Article  Google Scholar 

  • Birkbeck, N., Cobzas, D., Sturm, P., & Jägersand, M. (2006). Variational shape and reflectance estimation under changing light and viewpoints. In European conference on computer vision (Vol. 1, pp. 536–549).

  • Blinn, J. F. (1997). Models of light reflection for computer synthesized pictures. In SIGGRAPH ’77: Proceedings of the 4th annual conference on computer graphics and interactive techniques (pp. 192–198). New York: ACM. http://doi.acm.org/10.1145/563858.563893.

    Google Scholar 

  • Charpiat, G., Maurel, P., Pons, J. P., Keriven, R., & Faugeras, O. (2007). Generalized gradients: Priors on minimization flows. International Journal of Computer Vision, 73(3), 325–344.

    Article  Google Scholar 

  • Faugeras, O., & Keriven, R. (1998). Variational-principles, surface evolution, PDEs, level set methods, and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344.

    Article  MATH  MathSciNet  Google Scholar 

  • Gargallo, P. (2008). Contributions to the Bayesian approach to multi-view stereo. PhD thesis, Institut National Polytechique de Grenoble, France.

  • Gargallo, P., Prados, E., & Sturm, P. (2007). Minimizing the reprojection error in surface reconstruction from images. In IEEE international conference on computer vision.

  • Georghiades, A. S. (2003). Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. In IEEE international conference on computer vision (Vol. 02, pp. 816–823).

  • Goesele, M., Curless, B., & Seitz, S. M. (2006). Multi-view stereo revisited. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 2402–2409).

  • Goldman, D. B., Curless, B., Hertzmann, A., & Seitz, S. M. (2005). Shape and spatially-varying BRDFs from photometric stereo. In IEEE international conference on computer vision (pp. 341–348).

  • Hernández Esteban, C., Vogiatzis, G., & Cipolla, R. (2008). Multiview photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(3), 548–554.

    Article  Google Scholar 

  • Hertzmann, A., & Seitz, S. M. (2005). Example-based photometric stereo: Shape reconstruction with general, varying BRDFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1254–1264.

    Article  Google Scholar 

  • Jin, H., Yezzi, A., & Soatto, S. (2002). Variational multiframe stereo in the presence of specular reflections. In International symposium on 3D data processing visualization and transmission (pp. 626–630).

  • Jin, H., Yezzi, A. J., Tsai, Y. H., Cheng, L. T., & Soatto, S. (2003). Estimation of 3D surface shape and smooth radiance from 2D images: A level set approach. Journal of Scientific Computing, 19(1–3), 267–292.

    Article  MATH  MathSciNet  Google Scholar 

  • Jin, H., Cremers, D., Yezzi, A. J., & Soatto, S. (2004). Shedding light on stereoscopic segmentation. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 36–42).

  • Jin, H., Soatto, S., & Yezzi, A. J. (2005). Multi-view stereo reconstruction of dense shape and complex appearance. International Journal of Computer Vision, 63(3), 175–189.

    Article  Google Scholar 

  • Jin, H., Cremers, D., Wang, D., Prados, E., Yezzi, A., & Soatto, S. (2008). 3-d reconstruction of shaded objects from multiple images under unknown illumination. International Journal of Computer Vision, 76(3).

  • Kim, J., Kolmogorov, V., & R Zabih, R. (2003). Visual correspondence using energy minimization and mutual information. In IEEE international conference on computer vision (pp. 1033–1040).

  • Kolev, K., Klodt, M., Brox, T., & Cremers, D. (2007a). Propagated photoconsistency and convexity in variational multiview 3d reconstruction. In Workshop on photometric analysis for computer vision, Rio de Janeiro, Brazil.

  • Kolev, K., Klodt, M., Brox, T., Esedoglu, S., & Cremers, D. (2007b). Continuous global optimization in multiview 3d reconstruction. In LNCS : Vol. 4679. Energy minimization methods in computer vision and pattern recognition (EMMCVPR), Ezhou, China (pp. 441–452). Berlin: Springer.

    Chapter  Google Scholar 

  • Lee, H. C., Breneman, E. J., & Schulte, C. P. (1990). Modeling light reflection for computer color vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(4), 402–409.

    Article  Google Scholar 

  • Lu, J., & Little, J. (1995). Reflectance function estimation and shape recovery from image sequence of a rotating object. In IEEE international conference on computer vision (pp. 80–86).

  • Mallick, S. P., Zickler, T., Kriegman, D. J., & Belhumeur, P. N. (2005). Beyond Lambert: Reconstructing specular surfaces using color. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 619–626).

  • Ngan, A., Durand, F., & Matusik, W. (2005). Experimental analysis of BRDF models. In Eurographics symposium on rendering (pp. 117–126).

  • Osher, S., & Fedkiw, R. (2002). The level set method and dynamic implicit surfaces. Berlin: Springer.

    Google Scholar 

  • Osher, S., & Sethian, J. A. (1988). Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79, 12–49. citeseer.ist.psu.edu/osher88fronts.html.

    Article  MATH  MathSciNet  Google Scholar 

  • Paris, S., Sillion, F. X., & Quan, L. (2006). A surface reconstruction method using global graph cut optimization. International Journal of Computer Vision, 66(2), 141–161.

    Article  Google Scholar 

  • Pons, J. P., Keriven, R., & Faugeras, O. (2005). Modelling dynamic scenes by registering multi-view image sequences. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 822–827).

  • Pons, J. P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision, 72(2), 179–193.

    Article  Google Scholar 

  • Powell, M. W., Sarkar, S., & Goldgof, D. (2001). A simple strategy for calibrating the geometry of light sources. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(9), 1022–1027.

    Article  Google Scholar 

  • Schultz, H. (1994). Retrieving shape information from multiple images of a specular surface. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 195–201. http://dx.doi.org/10.1109/34.273732.

    Article  Google Scholar 

  • Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In IEEE conference on computer vision and pattern recognition (pp. 519–528).

  • Sethian, J. (1999). Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials sciences. Cambridge monograph on applied and computational mathematics. Cambridge: Cambridge University Press.

    Google Scholar 

  • Snow, D., Viola, P., & Zabih, R. (2000). Exact voxel occupancy with graph cuts. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 345–352).

  • Soatto, S., Yezzi, A. J., & Jin, H. (2003). Tales of shape and radiance in multi-view stereo. In IEEE international conference on computer vision (pp. 974–981).

  • Solem, J. E., & Overgaard, N. C. (2005). A geometric formulation of gradient descent for variational problems with moving surfaces. In Scale-space (pp. 419–430).

  • Stark, M., Arvo, J., & Smits, B. (2005). Barycentric parameterizations for isotropic BRDFs. IEEE Transactions on Visualization and Computer Graphics, 11(2), 126–138.

    Article  Google Scholar 

  • Tran, S., & Davis, L. (2006). 3d surface reconstruction using graph cuts with surface constraints. In European conference on computer vision (Vol. 2, pp. 219–231).

  • Vogiatzis, G., Favaro, P., & Cipolla, R. (2005). Using frontier points to recover shape, reflectance and illumination. In IEEE international conference on computer vision (Vol. 1, pp. 228–235).

  • Vogiatzis, G., Hernández Esteban, C., Torr, P. H. S., & Cipolla, R. (2007). Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2241–2246.

    Article  Google Scholar 

  • Yang, R., Pollefeys, M., & Welch, G. (2003). Dealing with textureless regions and specular highlights-a progressive space carving scheme using a novel photo-consistency measure. In IEEE international conference on computer vision (pp. 576–583).

  • Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43.

    Article  MathSciNet  Google Scholar 

  • Yoon, K. J., & Kweon, I. S. (2006). Correspondence search in the presence of specular highlights using specular-free two-band images. In Asian conference on computer vision (pp. 761–770).

  • Yu, T., Xu, N., & Ahuja, N. (2004). Recovering shape and reflectance model of non-lambertian objects from multiple views. In: IEEE conference on computer vision and pattern recognition (pp. 226–233).

  • Yu, T., Xu, N., & Ahuja, N. (2007). Shape and view independent reflectance map from multiple views. International Journal of Computer Vision, 73(2), 123–138.

    Article  Google Scholar 

  • Zach, C., Sormann, M., & Karner, K. (2006). High-performance multi-view reconstruction. In 3DPVT (pp. 113–120). IEEE Computer Society.

  • Zhou, W., & Kambhamettu, C. (2002). Estimation of illuminant direction and intensity of multiple light sources. In European conference on computer vision (pp. 206–220).

  • Zickler, T. (2006). Reciprocal image features for uncalibrated Helmholtz stereopsis. In IEEE conference on computer vision and pattern recognition (pp. 1801–1808).

  • Zickler, T., Belhumeur, P. N., & Kriegman, D. J. (2002). Helmholtz stereopsis: Exploiting reciprocity for surface reconstruction. International Journal of Computer Vision, 49(2–3), 215–227.

    Article  MATH  Google Scholar 

  • Zickler, T., Mallick, S. P., Kriegman, D. J., & Belhumeur, P. (2008). Color subspaces as photometric invariants. International Journal of Computer Vision, 79(1), 13–30.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Room C508, Computer Vision Laboratory, Department of Information and Communications, Gwangju Institute of Science and Technology (GIST), 261 Cheomdan-gwagiro (Oryong-dong), Buk-gu, Gwangju, 500-712, Republic of Korea

    Kuk-Jin Yoon

  2. Perception Team, INRIA Grenoble, Montbonnot, Rhône Alpes, France

    Emmanuel Prados & Peter Sturm

Authors
  1. Kuk-Jin Yoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emmanuel Prados
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Sturm
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kuk-Jin Yoon.

Additional information

This work was supported by the Korea Research Foundation Grant funded by the Korean Government (MOEHRD) (KRF-2006-352-D00087) and by the Flamenco project (grant ANR-06-MDCA-007).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yoon, KJ., Prados, E. & Sturm, P. Joint Estimation of Shape and Reflectance using Multiple Images with Known Illumination Conditions. Int J Comput Vis 86, 192–210 (2010). https://doi.org/10.1007/s11263-009-0222-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0222-4

Keywords

Search

Navigation