Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Single-Shot Multi-light-Direction Searching on Discretized Lighting Space

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Existing methods for light direction estimation solve the problem only by presupposing certain assumptions, e.g., that only one light illuminates the object, the object has a uniform texture or smoothly varying surface, or there is a known object in the scene. However, these methods cannot be used reliably in real-world settings. We propose a framework that can be used in natural settings to estimate multi-light directions from a single image. To tackle this novel and challenging problem, we divide the lighting space into ranges and formulate the estimation problem as a problem of searching the ranges where the light is cast from a multi-illuminant image. We propose a two-step approach. First, we generate single-illuminant images, one of which assumes that there is only light on one range. Next, we select some images from the candidates that could each be a component of a multi-illuminant image under physical constraints. The experimental results demonstrated that the proposed method outperformed the direct-estimation method when the lighting space is finely divided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, et al. Tensorflow: a system for large-scale machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), 2016; pp. 265–283.

  2. Alldrin N, Mallick S, Kriegman D. Resolving the generalized bas-relief ambiguity by entropy minimization. In: the IEEE Conf. on Computer Vision and Pattern Recognit. 2007; pp. 1–7. IEEE.

  3. Barron JT, Malik J. Shape, illumination, and reflectance from shading. IEEE Trans Pattern Anal Machine Intell. 2015;37(8):1670–87.

    Article  Google Scholar 

  4. Boom B, Escolano S, Ning X, McDonagh S, Sandilands P, Fisher R. Point light source estimation based on scenes recorded by a RGB-d camera. In: The British Machine Vision Cinference (2013)

  5. Boyadzhiev I, Bala K, Paris S, Durand F. User-guided white balance for mixed lighting conditions. ACM Trans Graph. 2012;31(6):200–1.

    Article  Google Scholar 

  6. Chakrabarti A, Sunkavalli K. Single-image rgb photometric stereo with spatially-varying albedo. In: 2016 Fourth Int. Conf. on 3D Vision (3DV), 2016; pp. 258–266. IEEE.

  7. Chen G, Han K, Shi B, Matsushita Y, Wong KYK: Self-calibrating deep photometric stereo networks. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognit., 2019; pp. 8739–8747.

  8. Chen Z, Chen A, Zhang G, Wang C, Ji Y, Kutulakos KN, Yu J. A neural rendering framework for free-viewpoint relighting. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognit., 2020; pp. 5599–5610.

  9. Ecker A, Jepson AD. Polynomial shape from shading. In: the IEEE Computer Society Conf. on Computer Vision and Pattern Recognit., 2010; pp. 145–152. IEEE.

  10. Fyffe G, Debevec P. Single-shot photometric stereo by spectral multiplexing. In: the IEEE Int. Conf. on Computational Photography. 2011; pp. 1–6. IEEE.

  11. Grosse R, Johnson MK, Adelson EH, Freeman WT. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In: the IEEE Int. Conf. on Computer Vision. 2009; pp. 2335–2342. IEEE.

  12. Hara K, Nishino K, Lkeuchi K. Light source position and reflectance estimation from a single view without the distant illumination assumption. IEEE Trans Pattern Anal Machine Intell. 2005;27(4):493–505.

    Article  Google Scholar 

  13. Hu Y, Wang B, Lin S. Fc 4: Fully convolutional color constancy with confidence-weighted pooling. In: the IEEE Conf. on Computer Vision and Pattern Recognit. 2017; pp. 4085–4094.

  14. Hui Z, Chakrabarti A, Sunkavalli K, Sankaranarayanan AC. Learning to separate multiple illuminants in a single image. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognit. 2019; pp. 3780–3789.

  15. Hui Z, Sunkavalli K, Hadap S, Sankaranarayanan A. Post-capture lighting manipulation using flash photography. CoRR arxiv:abs/1704.05564 (2017)

  16. Ju Y, Qi L, Zhou H, Dong J, Lu L. Demultiplexing colored images for multispectral photometric stereo via deep neural networks. IEEE Access. 2018;6:30804–18.

    Article  Google Scholar 

  17. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  18. Lee C, Rosenfeld A. Improved methods of estimating shape from shading using the light source coordinate system. Artificial Intell. 1985;26(2):125–43. https://doi.org/10.1016/0004-3702(85)90026-8.

    Article  MathSciNet  MATH  Google Scholar 

  19. Lopez-Moreno J, Garces E, Hadap S, Reinhard E, Gutierrez D. Multiple light source estimation in a single image. Comput Graph Forum. 2013;32:170–82 (Wiley Online Library).

    Article  Google Scholar 

  20. Lopez-Moreno J, Hadap S, Reinhard E, Gutierrez D. Compositing images through light source detection. Comput Graph. 2010;34(6):698–707.

    Article  Google Scholar 

  21. Murmann L, Gharbi M, Aittala M, Durand F. A dataset of multi-illumination images in the wild. In: Proc. of the IEEE Int. Conf. on Computer Vision. 2019; pp. 4080–4089.

  22. Nestmeyer T, Lalonde JF, Matthews I, Lehrmann A. Learning physics-guided face relighting under directional light. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognit. 2020; pp. 5124–5133.

  23. Papadhimitri T, Favaro P. A new perspective on uncalibrated photometric stereo. In: the IEEE Conf. on Computer Vision and Pattern Recognit; 2013. pp. 1474–1481. IEEE.

  24. Papadhimitri T, Favaro P. A closed-form, consistent and robust solution to uncalibrated photometric stereo via local diffuse reflectance maxima. Int J Comput Vis. 2014;107(2):139–54.

    Article  Google Scholar 

  25. Pentland A. Finding the illuminant direction. Josa. 1982;72(4):448–55.

    Article  Google Scholar 

  26. Pharr M, Jakob W, Humphreys G. Physically based rendering: from theory to implementation. Burlington: Morgan Kaufmann; 2016.

    Google Scholar 

  27. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. Med Image Comput Comput-Assist Intervent (MICCAI). 2015;9351:234–41 ((LNCS) Springer).

    Google Scholar 

  28. Saff EB, Kuijlaars ABJ. Distributing many points on a sphere. Math Intell. 1997;19(1):5–11. https://doi.org/10.1007/BF03024331.

    Article  MathSciNet  MATH  Google Scholar 

  29. Samaras D, Metaxas D. Incorporating illumination constraints in deformable models for shape from shading and light direction estimation. IEEE Trans Pattern Anal Mach Intell. 2003;25(2):247–64.

    Article  Google Scholar 

  30. Sengupta S, Kanazawa A, Castillo CD, Jacobs DW. Sfsnet: Learning shape, reflectance and illuminance of facesin the wild’. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recogni; 2018. pp. 6296–6305.

  31. Shi B, Matsushita Y, Wei Y, Xu C, Tan P. Self-calibrating photometric stereo; 2010. pp. 1118–1125. IEEE.

  32. Shi W, Loy CC, Tang X. Deep specialized network for illuminant estimation. In: the European Conf. on Computer Vision; 2016. pp. 371–387. Springer.

  33. Xiong Y, Chakrabarti A, Basri R, Gortler S, Jacobs D, Zickler T. From shading to local shape. IEEE Trans Pattern Anal Mach Intell. 2015;37:67–79.

    Article  Google Scholar 

  34. Zhang Y, Yang Y. Multiple illuminant direction detection with application to image synthesis. IEEE Trans Pattern Anal Mach Intell. 2001;23(8):915–20.

    Article  Google Scholar 

  35. Zheng Q, Chellappa R. Estimation of illuminant direction, albedo, and shape from shading. IEEE Trans Pattern Anal Mach Intell. 1991;13(7):680–702. https://doi.org/10.1109/34.85658.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Keio University, 3-14-1, Hiyoshi, Yokohama-Shi Kohoku-Ku, Kanagawa, 223-0061, Japan

    Tomoya Kaichi & Toshiki Kikuchi

  2. Tokyo Denki University, 5 Senju Asahi-cho, Adachi-ku, Tokyo, 120-8551, Japan

    Yuko Ozasa

Authors
  1. Tomoya Kaichi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toshiki Kikuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuko Ozasa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomoya Kaichi.

Ethics declarations

Funding

This work was supported by JSPS KAKENHI Grant Number 19K20299.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaichi, T., Kikuchi, T. & Ozasa, Y. Single-Shot Multi-light-Direction Searching on Discretized Lighting Space. SN COMPUT. SCI. 2, 129 (2021). https://doi.org/10.1007/s42979-021-00546-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00546-3

Keywords

Search

Navigation