research-article

Image based proximate shadow retargeting

Authors:

Matthias Fauconneau,

Kieran Mclister,

Kenny MitchellAuthors Info & Claims

CGVC '18: Proceedings of the Conference on Computer Graphics & Visual Computing

Pages 43 - 50

https://doi.org/10.2312/cgvc.20181206

Published: 13 September 2018 Publication History

Abstract

We introduce Shadow Retargeting which maps real shadow appearance to virtual shadows given a corresponding deformation of scene geometry, such that appearance is seamlessly maintained. By performing virtual shadow reconstruction from un-occluded real shadow samples observed in the camera frame, we recover the deformed shadow appearance efficiently. Our method uses geometry priors for the shadow casting object and a planar receiver surface. Inspired by image retargeting approaches [VTP* 10] we describe a novel local search strategy, steered by importance based deformed shadow estimation. Results are presented on a range of objects, deformations and illumination conditions in real-time Augmented Reality (AR) on a mobile device. We demonstrate the practical application of the method in generating otherwise laborious in-betweening frames for 3D printed stop motion animation.

References

[1]

{Bli88} Blinn J.: Me and my (fake) shadow. IEEE Comput. Graph. Appl. 8, 1 (Jan. 1988), 82--86. 4

Digital Library

[2]

{BSCB00} Bertalmio M., Sapiro G., Caselles V., Ballester C.: Image inpainting. Proceedings of the 27th annual conference on Computer graphics and interactive techniques SIGGRAPH 00 2 (2000). 3

Digital Library

[3]

{BVSO03} Bertalmio M., Vese L., Sapiro G., Osher S.: Simultaneous Structure and Texture Image Inpainting. IEEE Transactions on Image Processing (2003). 3

Digital Library

[4]

{CFV12} Castro T. K. D., Figueiredo L. H. D., Velho L.: Realistic shadows for mobile augmented reality. Proceedings - 2012 14th Symposium on Virtual and Augmented Reality, SVR 2012 (2012). 3

Digital Library

[5]

{CKM17} Casas L., Kosek M., Mitchell K.: Props Alive : A Framework for Augmented Reality Stop Motion Animation. In 2017 IEEE 10th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (Los Angeles, California, USA, 2017). 3, 6

[6]

{CMNK13} Calian D., Mitchell K., Nowrouzezahrai D., Kautz J.: The Shading Probe: Fast Appearance Acquisition for Mobile AR. SIGGRAPH Asia 2013 Technical Briefs (2013). 3

Digital Library

[7]

{CMPC06} Comport A. I., Marchand E., Pressigout M., Chaumette F.: Real-time markerless tracking for augmented reality: The virtual visual servoing framework. IEEE Transactions on Visualization and Computer Graphics 12, 4 (2006), 615--628. 2, 5

Digital Library

[8]

{Deb98} Debevec P.: Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proc. ACM SIGGRAPH '98 Proceeding (1998), ACM. 6

Digital Library

[9]

{Eme15} Emerson S.: Visual Effects at LAIKA, A Crossroads of Art and Technology. ACM SIGGRAPH 2015 Talks (2015), 2799649. 3

Digital Library

[10]

{Fer05} Fernando R.: Percentage-closer soft shadows. In ACM SIGGRAPH 2005 Sketches on - SIGGRAPH '05 (2005), p. 35. 5

Digital Library

[11]

{GZMT10} Goferman S., Zelnik-Manor L., Tal A.: Context-aware saliency detection. IEEE Conference on Computer Vision and Pattern Recognition (2010). 2

[12]

{HB12} Herling J., Broll W.: PixMix: A real-time approach to high-quality Diminished Reality. In 11th IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2012). 3

Digital Library

[13]

{HDH03} Haller M., Drab S., Hartmann W.: A real-time shadow approach for an augmented reality application using shadow volumes. Proceedings of the ACM symposium on Virtual reality software and technology - VRST '03 (2003). 3

Digital Library

[14]

{JNA*05} Jacobs K., Nahmias J.-D., Angus C., Reche A., Loscos C., Steed A.: Automatic generation of consistent shadows for augmented reality. ACM International Conference Proceeding Series; Vol. 112 (2005), 113. 3

[15]

{JS07} Jarusirisawad S., Saito H.: Diminished reality via multiple hand-held cameras. 2007 1st ACM/IEEE International Conference on Distributed Smart Cameras, ICDSC (2007), 251--258. 3

[16]

{Kaj86} Kajiya J. T.: The rendering equation. SIGGRAPH Comput. Graph. 20 (Aug. 1986), 143--150. 4

Digital Library

[17]

{KS00} Kang S. B., Shum H.-Y.: A review of image-based rendering techniques. Institute of Electrical and Electronics Engineers, Inc. 2

[18]

{KSY13} Kawai N., Sato T., Yokoya N.: Diminished reality considering background structures. In 2013 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013 (2013). 3

[19]

{LLT*11} Leão C. W. M., Lima J. P., Teichrieb V., Albuquerque E. S., Keiner J.: Altered reality: Augmenting and diminishing reality in real time. In Proceedings - IEEE Virtual Reality (2011). 2

Digital Library

[20]

{Man01} Mann S.: Fundamental issues in mediated reality, WearComp, and camera-based augmented reality. Fundamentals of Wearable Computers and Augmented Reality, Lawrence Erlbaum Associates (2001). 2

[21]

{MNZ*15} Magnenat S., Ngo D. T., Zünd F., Ryffel M., Noris G., Rothlin G., Marra A., Nitti M., Fua P., Gross M., Sumner R. W.: Live Texturing of Augmented Reality Characters from Colored Drawings. IEEE Transactions on Visualization and Computer Graphics 21 (2015), 1201--1210. 2

Digital Library

[22]

{Mor17} Mori S.: Diminished Hand : A Diminished Reality-Based Work Area Visualization. 2017 IEEE Virtual Reality (2017). 3, 5

[23]

{NGM*11} Nowrouzezahrai D., Geiger S., Mitchell K., Sumner R., Jarosz W., Gross M.: Light Factorization for Mixed-Frequency Shadows in Augmented Reality. In 10th IEEE International Symposium on Mixed and Augmented Reality, ISMAR (2011). 2, 3

Digital Library

[24]

{SSI99} Sato I., Sato Y., Ikeuchi K.: Illumination distribution from shadows. Computer Vision and Pattern Recognition (CVPR) (1999). 3

[25]

{STR*05} Setlur V., Takagi S., Raskar R., Gleicher M., Gooch B.: Automatic image retargeting. ACM International Conference on Mobile and Ubiquitous Multimedia (2005). 2

Digital Library

[26]

{TP12} Takeuchi Y., Perlin K.: ClayVision: the (elastic) image of the city. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2012), 2411--2420. 2, 3, 5

Digital Library

[27]

{VTP*10} Vaquero D., Turk M., Pulli K., Tico M., Gelfand N.: A survey of image retargeting techniques. Proc. Applications of Digital Image Processing XXXIII (2010). 1, 2

[28]

{Vuf17} Vuforia: Vuforia Object Recognition, 2017. 2, 5

[29]

{WPL18} Weber H., Prevost D., Lalonde J.-F.: Learning to estimate indoor lighting from 3d objects. CVPR (2018). 3

[30]

{WS03} Wang Y., Samaras D.: Estimation of multiple directional light sources for synthesis of augmented reality images. 3

[31]

{YTS*11} Yang L., Tse Y.-C., Sander P. V., Lawrence J., Nehab D., Hoppe H., Wilkins C. L.: Image-based bidirectional scene reprojection. ACM Trans. Graph. 30, 6 (Dec. 2011), 1--10. 7

Digital Library

[32]

{ZW13} Zheng X., Wang L.: A Video-based Interface for Hand-Driven Stop Motion Animation Production. IEEE Computer Graphics and Applications (2013). 3

Digital Library

[33]

{ZZK*16} Zilly F., Ziegler M., Keinert J., Schoberl M., Foessel S.: Computational Imaging for Stop-Motion Animated Video Productions. SMPTE Motion Imaging Journal 125, 1 (2016), 42--47. 3

[34]

{ZZX15} Zhang L., Zhang Q., Xiao C.: Shadow remover: Image shadow removal based on illumination recovering optimization. IEEE Transactions on Image Processing 24, 11 (Nov 2015), 4623--4636. 2

Cited By

Casas LMitchell K(2023)Intermediated Reality with an AI 3D Printed CharacterACM SIGGRAPH 2023 Real-Time Live!10.1145/3588430.3597251(1-2)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588430.3597251
Casas L(2019)Intermediated RealitySIGGRAPH Asia 2019 Doctoral Consortium10.1145/3366344.3366629(1-4)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3366344.3366629
Casas LCiccone LÇimen GWiedemann PFauconneau MSumner RMitchell KMikami KPan ZAdcock MThalmann DYang XItamiya TWu E(2018)Multi-reality gamesProceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry10.1145/3284398.3284411(1-4)Online publication date: 2-Dec-2018
https://dl.acm.org/doi/10.1145/3284398.3284411

Index Terms

Image based proximate shadow retargeting
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Mixed / augmented reality
      2. Perception

Recommendations

Interactive global illumination based on coherent surface shadow maps
GI '08: Proceedings of Graphics Interface 2008

Interactive rendering of global illumination effects is a challenging problem. While precomputed radiance transfer (PRT) is able to render such effects in real time the geometry is generally assumed static. This work proposes to replace the precomputed ...
Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization
In this paper, we present a novel shadow removal system for single natural images as well as color aerial images using an illumination recovering optimization method. We first adaptively decompose the input image into overlapped patches according to the ...
Shadow silhouette maps

The most popular techniques for interactive rendering of hard shadows are shadow maps and shadow volumes. Shadow maps work well in regions that are completely in light or in shadow but result in objectionable artifacts near shadow boundaries. In ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

CGVC '18: Proceedings of the Conference on Computer Graphics & Visual Computing

September 2018

175 pages

Conference Chair:
Robert S. Laramee
Swansea University
,
Program Chairs:
Gary KL Tam
Swansea University
,
Franck Vidal
Bangor University

Publisher

Eurographics Association

Goslar, Germany

Publication History

Published: 13 September 2018

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Casas LMitchell K(2023)Intermediated Reality with an AI 3D Printed CharacterACM SIGGRAPH 2023 Real-Time Live!10.1145/3588430.3597251(1-2)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588430.3597251
Casas L(2019)Intermediated RealitySIGGRAPH Asia 2019 Doctoral Consortium10.1145/3366344.3366629(1-4)Online publication date: 17-Nov-2019
https://dl.acm.org/doi/10.1145/3366344.3366629
Casas LCiccone LÇimen GWiedemann PFauconneau MSumner RMitchell KMikami KPan ZAdcock MThalmann DYang XItamiya TWu E(2018)Multi-reality gamesProceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry10.1145/3284398.3284411(1-4)Online publication date: 2-Dec-2018
https://dl.acm.org/doi/10.1145/3284398.3284411

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents