Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Artistic Augmentation of Photographs with Droplets

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Artistic augmentation of photographs with water droplets aims at generating aesthetic yet realistic images, and thus differs from traditional augmented reality in two aspects. One difference lies in the adoption of a new image as the environment map in order to render reflected or refracted effects on the surface of inserted water droplets. The other difference is in modeling of water droplets including hanging droplets and resting droplets. These differences raise two research challenges: 1) how to adjust the brightness and colors of the new environment map to maintain visual consistency between the new environment map and the original input image; 2) how to model hanging and resting droplets aesthetically. This paper proposes a framework that addresses these two challenges and demonstrates the effectiveness of our framework by generating example augmented images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kronander J, Banterle F, Gardner A, Miandji E, Unger J. Photorealistic rendering of mixed reality scenes. Comput. Graph. Forum, 2015, 34(2): 643-665.

    Article  Google Scholar 

  2. Hartley R I, Zisserman A. Multiple View Geometry in Computer Vision (2nd edition). Cambridge University Press, 2004.

  3. Schmalstieg D, Höllerer T. Augmented Reality: Theory and Practice (1st edition). Addison-Wesley Professional, 2016.

  4. van Krevelen D W F, Poelman R. A survey of augmented reality technologies, applications and limitations. The International Journal of Virtual Reality, 2010, 9(2): 1-20.

    Google Scholar 

  5. Rabbi I, Ullah S. A survey of augmented reality challenges and tracking. ACTA GRAPHICA, 2013, 24(1/2): 29-46.

  6. Debevec P. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proc. the 25th Annual Conference on Computer Graphics and Interactive Techniques, July 1998, pp.189-198.

  7. Stumpfel J, Tchou C, Jones A, Hawkins T, Wenger A, Debevec P. Direct HDR capture of the sun and sky. In Proc. the 3rd International Conference on Computer Graphics, November 2004, pp.145-149.

  8. Banterle F, Ledda P, Debattista K, Chalmers A. Inverse tone mapping. In Proc. the 4th International Conference on Computer Graphics and Interactive Techniques, November 2006, pp.349-356.

  9. Banterle F, Debattista K, Artusi A, Pattanaik S, Myszkowski K, Ledda P, Chalmers A. High dynamic range imaging and low dynamic range expansion for generating HDR content. Computer Graphics Forum, 2009, 28(8): 2343-2367.

    Article  Google Scholar 

  10. Pellacini F. Envylight: An interface for editing natural illumination. ACM Transactions on Graphics, 2010, 29(4): Article No. 34.

  11. Debevec P, Graham P, Busch J, Bolas M. A single-shot light probe. In Proc. the 39th International Conference on Computer Graphics and Interactive Techniques, August 2012, Article No. 10.

  12. Reinhard E, Ashikhmin M, Gooch B, Shirley P. Color transfer between images. IEEE Computer Graphics and Applications, 2001, 21(5): 34-41.

    Article  Google Scholar 

  13. Reinhard E, Akyuz O A, Colbert M, Hughes C, O’Connor M. Real-time color blending of rendered and captured video. In Proc. Interservice/Industry Training, Simulation and Education Conference, Dec. 2004, Article No. 1502.

  14. Khan E A, Reinhard E, Fleming R W, Bülthoff H H. Image-based material editing. ACM Transactions on Graphics, 2006, 25(3): 654-663.

    Article  Google Scholar 

  15. Lopez-Moreno J, Hadap S, Reinhard E, Gutierrez D. Compositing images through light source detection. Computers & Graphics, 2010, 34(6): 698-707.

    Article  Google Scholar 

  16. Karsch K, Hedau V, Forsyth D, Hoiem D. Rendering synthetic objects into legacy photographs. ACM Transactions on Graphics, 2011, 30(6): Article No. 157.

  17. Karsch K, Sunkavalli K, Hadap S, Carr N, Jin H, Fonte R, Sittig M, Forsyth D. Automatic scene inference for 3D object compositing. ACM Transactions on Graphics, 2014, 33(3): Article No. 32.

    Article  MATH  Google Scholar 

  18. Grosse R, Johnson M K, Adelson E H, Freeman W T. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In Proc. the 12th IEEE International Conference on Computer, September 2009, pp.2335-2342.

  19. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis & Ma- chine Intelligence, 2012, 34(11): 2274-2282.

    Article  Google Scholar 

  20. Lalonde J F, Efros A A, Narasimhan S G. Webcam clip art: Appearance and illuminant transfer from time-lapse sequences. ACM Transactions on Graphics, 2009, 28(5): Article No. 131.

    Article  Google Scholar 

  21. Perez R, Seals R, Michalsky J. All-weather model for sky luminance distribution — Preliminary configuration and validation. Solar Energy, 1993, 50(3): 235-245.

    Article  Google Scholar 

  22. Lalonde J F, Efros A A, Narasimhan S G. Estimating natural illumination from a single outdoor image. In Proc. the 12th IEEE International Conference on Computer Vision, September 2009, pp.183-190.

  23. Lalonde J F, Efros A A, Narasimhan S G. Estimating the natural illumination conditions from a single outdoor image. International Journal of Computer Vision, 2012, 98(2): 123-145.

    Article  MathSciNet  Google Scholar 

  24. Lalonde J F. Understanding and recreating visual appearance under natural illumination [Ph.D. Thesis]. Carnegie Mellon University, 2011.

  25. Liu Y, Qin X, Xu S, Nakamae E, Peng Q. Light source estimation of outdoor scenes for mixed reality. The Visual Computer, 2009, 25(5/6/7): 637-646.

    Article  Google Scholar 

  26. Liu Y, Granier X. Online tracking of outdoor lighting variations for augmented reality with moving cameras. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(4): 573-580.

    Article  Google Scholar 

  27. Xing G, Zhou X, Peng Q, Liu Y, Qin X. Lighting simulation of augmented outdoor scene based on a legacy photograph. Computer Graphics Forum, 2013, 32(7): 101-110.

    Article  Google Scholar 

  28. Blinn J F. A generalization of algebraic surface drawing. ACM Trans. Graph., 1982, 1(3): 235-256.

    Article  Google Scholar 

  29. O’Brien J F, Hodgins J K. Dynamic simulation of splashing fluids. In Proc. the 1995 Computer Animation, April 1995, pp.198-205.

  30. Murta A, Miller J. Modelling and rendering liquids in motion. In Proc. the 7th International Conference in Central Europe on Computer Graphics, Visualization and Interactive Digital Media, February 1999, pp.194-201.

  31. Fournier P, Habibi A, Poulin P. Simulating the flow of liquid droplets. In Proc. the 1998 Graphics Interface Conference, June 1998, pp.133-142.

  32. Kaneda K, Kagawa T, Yamashita H. Animation of water droplets on a glass plate. In Proc. the 5th International Workshop on Computer Animation, June 1993, pp.177-189.

  33. Kaneda K, Ikeda S, Yamashita H. Animation of water droplets moving down a surface. The Journal of Visualization and Computer Animation, 1999, 10(1): 15-26.

    Article  Google Scholar 

  34. Yu Y J, Jung H Y, Cho H G. A new water droplet model using metaball in the gravitational field. Computers & Graphics, 1999, 23(2): 213-222.

    Article  Google Scholar 

  35. Tong R, Kaneda K, Yamashita H. A volume-preserving approach for modeling and animating water flows generated by metaballs. The Visual Computer, 2002, 18(8): 469-480.

    Article  Google Scholar 

  36. Wolff L B. Using polarization to separate reflection components. In Proc. the 1989 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 1989, pp.363-369.

  37. Lalonde J F, Narasimhan S G, Efros A A. What do the sun and the sky tell us about the camera? International Journal of Computer Vision, 2010, 88(1): 24-51.

    Article  Google Scholar 

  38. Hold-Geoffroy Y, Sunkavalli K, Hadap S, Gambaretto E, Lalonde J F. Deep outdoor illumination estimation. In Proc. the 2017 IEEE International Conference on Computer Vision and Pattern Recognition, July 2017, pp.2373-2382.

  39. Gardner M, Sunkavalli K, Yumer E, Shen X, Gambaretto E, Gagné C, Lalonde J F. Learning to predict indoor illumination from a single image. arXiv:1704.00090, 2017. https://arxiv.org/abs/1704.00090, June 2019.

  40. Georgoulis S, Rematas K, Ritschel T, Fritz M, Tuytelaars T, Gool L V. What is around the camera? In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.5180-5188.

  41. Hold-Geoffroy Y, Sunkavalli K, Eisenmann J, Fisher M, Gambaretto E, Hadap S, Lalonde J. A perceptual measure for deep single image camera calibration. arXiv:1712.01259, 2017. https://arxiv.org/abs/1712.01259, June 2019.

  42. Xing G, Liu Y, Peng Q, Qin X. On-line illumination estimation of outdoor scenes based on area selection for augmented reality. In Proc. the 12th International Conference on Computer-Aided Design and Computer Graphics, September 2011, pp.439-442.

  43. Xing G, Liu Y, Qin X, Peng Q. A practical approach for real-time illumination estimation of outdoor videos. Computers & Graphics, 2012, 36(7): 857-865.

    Article  Google Scholar 

  44. Boivin S, Gagalowicz A. Image-based rendering of diffuse, specular and glossy surfaces from a single image. In Proc. the 28th Annual Conference on Computer Graphics and Interactive Techniques, August 2001, pp.107-116.

  45. Bousseau A, Paris S, Durand F. User-assisted intrinsic images. ACM Trans. Graph., 2009, 28(5): Article No. 130.

    Article  Google Scholar 

  46. MacRobert T. Spherical Harmonics: An Elementary Treatise on Harmonic Functions with Applications (3rd rev. edition). Pergamon Press, 1967.

  47. Sillion F X, Arvo J R, Westin S H, Greenberg D P. A global illumination solution for general reflectance distributions. ACM SIGGRAPH Comput. Graph., 1991, 25(4): 187-196.

    Article  Google Scholar 

  48. Ramamoorthi R, Hanrahan P. On the relationship between radiance and irradiance: Determining the illumination from images of a convex Lambertian object. Journal of the Optical Society of America A, Optics, Image Science, and Vision, 2001, 18(10): 2448-2459.

    Article  MathSciNet  Google Scholar 

  49. Ramamoorthi R, Hanrahan P. An efficient representation for irradiance environment maps. In Proc. the 28th Annual Conference on Computer Graphics and Interactive Techniques, August 2001, pp.497-500.

  50. Ren H, Xu S, Wu S T. Effects of gravity on the shape of liquid droplets. Optics Communications, 2010, 283(17): 3255-3258.

    Article  Google Scholar 

  51. Dixit S, Pincus A, Guo B, Faris G. Droplet shape analysis and permeability studies in droplet lipid bilayers. Langmuir: The ACS Journal of Surfaces and Colloids, 2012, 28(19): 7442-7451.

    Article  Google Scholar 

  52. Sahin S, Bliznyuk O, Cordova R A, Schröen K. Microfluidic EDGE emulsification: The importance of interface interactions on droplet formation and pressure stability. Scientific Reports, 2016, 6: Article No. 26407.

  53. Li Z, Wang S, Yu J, Ma K L. Restoration of brick and stone relief from single rubbing images. IEEE Transactions on Visualization and Computer Graphics, 2012, 18(2): 177-187.

    Article  Google Scholar 

  54. Zhang T, Zhang L, Yu J. Computer generation of 3D inscriptions from 2D images of Chinese calligraphy. Chinese Journal of Computers, 2014, 37(11): 2380-2388. (in Chinese)

    MathSciNet  Google Scholar 

  55. Hamlyn D W. The Psychology of Perception: A Philosophical Examination of Gestalt Theory and Derivative Theories of Perception (1st edition). Routledge, 1957.

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of CAD & CG, Zhejiang University, Hangzhou, 310027, China

    Mo-Han Zhang & Jin-Hui Yu

  2. Department of Computer Science, Harbin Finance University, Harbin, 150030, China

    Jin-Hui Yu

  3. Department of Computer Science, The University of Texas at Dallas, Richardson, TX, 75080-3021, U.S.A.

    Kang Zhang

  4. Faculty of Information Technology, Macau University of Science and Technology, Macau, 999078, China

    Kang Zhang

  5. National Engineering Research Center for E-Learning, Central China Normal University, Wuhan, 430079, China

    Jun-Song Zhang

Authors
  1. Mo-Han Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin-Hui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun-Song Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mo-Han Zhang.

Electronic supplementary material

ESM 1

(PDF 753 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, MH., Yu, JH., Zhang, K. et al. Artistic Augmentation of Photographs with Droplets. J. Comput. Sci. Technol. 34, 1294–1306 (2019). https://doi.org/10.1007/s11390-019-1976-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-019-1976-y

Keywords

Search

Navigation