article

Path-space Motion Estimation and Decomposition for Robust Animation Filtering

Authors:

Henning Zimmer,

Fabrice Rousselle,

Wojciech Jarosz,

Olga Sorkine-Hornung,

Alexander Sorkine-HornungAuthors Info & Claims

Computer Graphics Forum, Volume 34, Issue 4

Pages 131 - 142

Published: 01 July 2015 Publication History

Abstract

Renderings of animation sequences with physics-based Monte Carlo light transport simulations are exceedingly costly to generate frame-by-frame, yet much of this computation is highly redundant due to the strong coherence in space, time and among samples. A promising approach pursued in prior work entails subsampling the sequence in space, time, and number of samples, followed by image-based spatio-temporal upsampling and denoising. These methods can provide significant performance gains, though major issues remain: firstly, in a multiple scattering simulation, the final pixel color is the composite of many different light transport phenomena, and this conflicting information causes artifacts in image-based methods. Secondly, motion vectors are needed to establish correspondence between the pixels in different frames, but it is unclear how to obtain them for most kinds of light paths e.g. an object seen through a curved glass panel.

References

[1]

<label>{BBPW04}</label> Brox T., Bruhn A., Papenberg N., Weickert J.: High accuracy optical flow estimation based on a theory for warping</otherTitle>. In <otherTitle>ECCV 2004, pp. pp.25-36. 2, 7

[2]

<label>{BCM08}</label> Buades A., Coll B., Morel J.: Nonlocal image and movie denoising. International Journal of Computer Vision Volume 76, Issue 2 2008, pp.123-139. 8

Digital Library

[3]

<label>{BE01}</label> Brostow G.J., Essa I.: Image-based motion blur for stop motion animation</otherTitle>. In <otherTitle>SIGGRAPH 2001, pp. pp.561-566. 11

Digital Library

[4]

<label>{BMS*12}</label> Bowles H., Mitchell K., Sumner R.W., Moore J., Gross M.H.: Iterative image warping. Comp. Graph. Forum Volume 31, Issue 2 2012, pp.237-246. 2

[5]

<label>{BSCB00}</label> Bertalmio M., Sapiro G., Caselles V., Ballester C.: Image inpainting</otherTitle>. In <otherTitle>SIGGRAPH 2000, pp. pp.417-424. 11

[6]

<label>{CW93}</label> Chen S.E., Williams L.: View interpolation for image synthesis</otherTitle>. In <otherTitle>SIGGRAPH 1993, pp. pp.279-288. 3, 11

Digital Library

[7]

<label>{DER*10}</label> Didyk P., Eisemann E., Ritschel T., Myszkowski K., Seidel H.-P.: Perceptually-motivated real-time temporal upsampling of 3D content for high-refresh-rate displays. Comp. Graph. Forum Volume 29, Issue 2 2010, pp.713-722. 3

[8]

<label>{DKR*14}</label> Da¿bała L., Kellnhofer P., Ritschel T., Didyk P., Templin K., Myszkowski K., Rokita P., Seidel H.-P.: Manipulating refractive and reflective binocular disparity. Comp. Graph. Forum Proc. Eurographics 2014 Volume 33, Issue 2 2014, pp.53-62. 2

Digital Library

[9]

<label>{God14}</label> Goddard L.: Silencing the noise on Elysium. In ACM SIGGRAPH 2014 Talks 2014, ACM. 2

Digital Library

[10]

<label>{Hec90}</label> Heckbert P.S.: Adaptive radiosity textures for bidirectional ray tracing</otherTitle>. In <otherTitle>SIGGRAPH 1990, pp. pp.145-154. 3

Digital Library

[11]

<label>{HEMS10}</label> Herzog R., Eisemann E., Myszkowski K., Seidel H.-P.: Spatio-temporal upsampling on the GPU</otherTitle>. In <otherTitle>SI3D 2010, pp. pp.91-98. 2, 3, 4, 8, 9

Digital Library

[12]

<label>{HMD*14}</label> Hill S., McAuley S., Dupuy J., Gotanda Y., Heitz E., Hoffman N., Lagarde S., Langlands A., Megibben I., Rayani F., de Rousiers C.: Physically based shading in theory and practice. In ACM SIGGRAPH 2014 Courses New York, NY, USA, 2014, SIGGRAPH '14, ACM. 2, 5

Digital Library

[13]

<label>{HST13}</label> He K., Sun J., Tang X.: Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. Volume 35, Issue 6 2013, pp.1397-1409. 3, 9

Digital Library

[14]

<label>{Ige99}</label> Igehy H.: Tracing ray differentials</otherTitle>. In <otherTitle>SIGGRAPH 1999, pp. pp.179-186. 2

Digital Library

[15]

<label>{Jak10}</label> Jakob W.: Mitsuba renderer, 2010. "http://www.mitsuba-renderer.org". 4

[16]

<label>{Jak13}</label> Jakob W.: Light Transport on Path-Space Manifolds. PhD thesis, Cornell University, Aug. 2013. 5

[17]

<label>{JM12}</label> Jakob W., Marschner S.: Manifold exploration: A markov chain monte carlo technique for rendering scenes with difficult specular transport. ACM Trans. Graph. Volume 31, Issue 4 2012, pp.58:1-58:13. 2, 5, 6

Digital Library

[18]

<label>{Kai12}</label> Kainz F.: Interpreting OpenEXR deep pixels, 2012. "http://openexr.com/InterpretingDeepPixels.pdf". 11

[19]

<label>{KKG*14}</label> Krivánek J., Keller A., Georgiev I., Kaplanyan A., Fajardo M., Meyer M., Nahmias J.-D., Karlík O., Canada J.: Recent advances in light transport simulation: Theory and practice. In ACM SIGGRAPH Courses 2014, SIGGRAPH '14, ACM. 1

Digital Library

[20]

<label>{LRR*14}</label> Lochmann G., Reinert B., Ritschel T., Müller S., Seidel H.: Real-time reflective and refractive novel-view synthesis. In VMV 20142014, pp. pp.9-16. 2, 3

[21]

<label>{MJL*13}</label> Moon B., Jun J.Y., Lee J., Kim K., Hachisuka T., Yoon S.: Robust image denoising using a virtual flash image for monte carlo ray tracing. Comp. Graph. Forum Volume 32, Issue 1 2013, pp.139-151. 3

[22]

<label>{MMB97}</label> Mark W.R., McMillan L., Bishop G.: Post-rendering 3D warping</otherTitle>. In <otherTitle>SI3D 1997, pp. pp.7-16, 180. 2

Digital Library

[23]

<label>{MYRD14}</label> Mehta S.U., Yao J., Ramamoorthi R., Durand F.: Factored axis-aligned filtering for rendering multiple distribution effects. ACM Trans. Graph. Volume 33, Issue 4 July 2014. 2

Digital Library

[24]

<label>{NVI12}</label> NVIDIA: Iray whitepaper, 2012. "http://www.nvidia-arc.com/fileadmin/user_upload/iray_2013/documents/iray_whitepaper.121206.pdf". 2

[25]

<label>{PGB03}</label> Pérez P., Gangnet M., Blake A.: Poisson image editing. ACM Trans. Graph. Volume 22, Issue 3 July 2003, pp.313-318. 11

Digital Library

[26]

<label>{RMZ13}</label> Rousselle F., Manzi M., Zwicker M.: Robust denoising using feature and color information. Computer Graphics Forum Volume 32, Issue 7 2013, pp.121-130. 3, 7, 8, 9

[27]

<label>{SD12}</label> Sen P., Darabi S.: On filtering the noise from the random parameters in monte carlo rendering. ACM Trans. Graph. Volume 31, Issue 3 2012, pp.18. 2, 3, 4

Digital Library

[28]

<label>{SGNS07}</label> Sloan P.J., Govindaraju N.K., Nowrouzezahrai D., Snyder J.: Image-based proxy accumulation for real-time soft global illumination</otherTitle>. In <otherTitle>Pacific Graphics 2007, pp. pp.97-105. 9

Digital Library

[29]

<label>{Shi95}</label> Shinya M.: Improvements on the pixel-tracing filter: Reflection/refraction, shadows & jittering. In Graphics Interface 1995, CANADIAN INFORMATION PROCESSING SOCIETY, pp. pp.92-92. 2

[30]

<label>{SIMP06}</label> Segovia B., Iehl J.C., Mitanchey R., Péroche B.: Non-interleaved deferred shading of interleaved sample patterns. In Proc. Graphics Hardware 2006, ACM. 2

Digital Library

[31]

<label>{ST90}</label> Saito T., Takahashi T.: Comprehensible rendering of 3-d shapes. SIGGRAPH Comp. Graph. Volume 24, Issue 4 Sept. 1990. 2

Digital Library

[32]

<label>{SYM*11}</label> Scherzer D., Yang L., Mattausch O., Nehab D., Sander P.V., Wimmer M., Eisemann E.: A survey on temporal coherence methods in real-time rendering</otherTitle>. In <otherTitle>EUROGRAPHICS 2011 State of the Art Reports 2011. 3

[33]

<label>{Vea98}</label> Veach E.: Robust Monte Carlo Methods for Light Transport Simulation. PhD thesis, Stanford, CA, USA, 1998. AAI9837162. 2, 3

Digital Library

[34]

<label>{WKB*02}</label> Wald I., Kollig T., Benthin C., Keller A., Slusallek P.: Interactive global illumination using fast ray tracing</otherTitle>. In <otherTitle>Proceedings of the 13th Eurographics Workshop on Rendering EGWR 2002, pp. pp.15-24. 2

Digital Library

[35]

<label>{WMLT07}</label> Walter B., Marschner S.R., Li H., Torrance K.E.: Microfacet models for refraction through rough surfaces</otherTitle>. In <otherTitle>Rendering Techniques 2007, pp. pp.195-206. 6

Digital Library

[36]

<label>{WRC88}</label> Ward G.J., Rubinstein F.M., Clear R.D.: A ray tracing solution for diffuse interreflection. SIGGRAPH Comp. Graph. Volume 22, Issue 4 June 1988, pp.85-92. 2, 3

Digital Library

[37]

<label>{YTS*11}</label> Yang L., Tse Y.-C., Sander P.V., Lawrence J., Nehab D., Hoppe H., Wilkins C.L.: Image-based bidirectional scene reprojection. ACM Trans. Graph. Volume 30, Issue 6 2011. 10

Digital Library

[38]

<label>{ZJL*15}</label> Zwicker M., Jarosz W., Lehtinen J., Moon B., Ramamoorthi R., Rousselle F., Sen P., Soler C., Yoon S.: Recent advances in adaptive sampling and reconstruction for monte carlo rendering</otherTitle>. <otherTitle>Comp. Graph. Forum, to appear 2015. 3

Digital Library

[39]

<label>{ZKU*04}</label> Zitnick C.L., Kang S.B., Uyttendaele M., Winder S. A. J., Szeliski R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. Volume 23, Issue 3 2004, pp.600-608. 3

Digital Library

Cited By

Chen CHe YLi T(2024)Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer BlocksACM Transactions on Graphics10.1145/365821843:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658218
Paliwal ANguyen BTsarov AKalantari N(2023)ReShader: View-Dependent Highlights for Single Image View-SynthesisACM Transactions on Graphics10.1145/361839342:6(1-9)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618393
Yu JNie YLong CXu WZhang QLi G(2021)Monte Carlo denoising via auxiliary feature guided self-attentionACM Transactions on Graphics10.1145/3478513.348056540:6(1-13)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480565
Show More Cited By

Index Terms

Path-space Motion Estimation and Decomposition for Robust Animation Filtering
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Texturing

Recommendations

Realtime Rendering Glossy to Glossy Reflections in Screen Space

Glossy to glossy reflections are lights bounced between glossy surfaces. Such directional light transports are important for humans to perceive glossy materials, but difficult to simulate. This paper proposes a new method for rendering screen-space ...
Virtual Spherical Gaussian Lights for Real-time Glossy Indirect Illumination

Virtual point lights VPLs are well established for real-time global illumination. However, this method suffers from spiky artifacts and flickering caused by singularities of VPLs, highly glossy materials, high-frequency textures, and discontinuous ...
IlluminationCut

We present a novel algorithm, IlluminationCut, for rendering images using the many-lights framework. It handles any light source that can be approximated with virtual point lights VPLs as well as highly glossy materials. The algorithm extends the ...

Comments

Information & Contributors

Information

Published In

cover image Computer Graphics Forum

Computer Graphics Forum Volume 34, Issue 4

July 2015

167 pages

ISSN:0167-7055

EISSN:1467-8659

Issue’s Table of Contents

Publisher

The Eurographs Association & John Wiley & Sons, Ltd.

Chichester, United Kingdom

Publication History

Published: 01 July 2015

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen CHe YLi T(2024)Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer BlocksACM Transactions on Graphics10.1145/365821843:4(1-14)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658218
Paliwal ANguyen BTsarov AKalantari N(2023)ReShader: View-Dependent Highlights for Single Image View-SynthesisACM Transactions on Graphics10.1145/361839342:6(1-9)Online publication date: 5-Dec-2023
https://dl.acm.org/doi/10.1145/3618393
Yu JNie YLong CXu WZhang QLi G(2021)Monte Carlo denoising via auxiliary feature guided self-attentionACM Transactions on Graphics10.1145/3478513.348056540:6(1-13)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480565
Briedis KDjelouah AMeyer MMcGonigal IGross MSchroers C(2021)Neural frame interpolation for rendered contentACM Transactions on Graphics10.1145/3478513.348055340:6(1-13)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480553
Guo JFu XLin LMa HGuo YLiu SYan L(2021)ExtraNetACM Transactions on Graphics10.1145/3478513.348053140:6(1-16)Online publication date: 10-Dec-2021
https://dl.acm.org/doi/10.1145/3478513.3480531
Xu BZhang JWang RXu KYang YLi CTang R(2019)Adversarial Monte Carlo denoising with conditioned auxiliary feature modulationACM Transactions on Graphics10.1145/3355089.335654738:6(1-12)Online publication date: 8-Nov-2019
https://dl.acm.org/doi/10.1145/3355089.3356547
Koskela MImmonen KMäkitalo MFoi AViitanen TJääskeläinen PKultala HTakala J(2019)Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing ReconstructionACM Transactions on Graphics10.1145/326997838:5(1-14)Online publication date: 17-Jun-2019
https://dl.acm.org/doi/10.1145/3269978
Christensen PFong JShade JWooten WSchubert BKensler AFriedman SKilpatrick CRamshaw CBannister MRayner BBrouillat JLiani M(2018)RenderManACM Transactions on Graphics10.1145/318216237:3(1-21)Online publication date: 7-Aug-2018
https://dl.acm.org/doi/10.1145/3182162
Schied CKaplanyan AWyman CPatney AChaitanya CBurgess JLiu SDachsbacher CLefohn ASalvi MMcGuire MPatney A(2017)Spatiotemporal variance-guided filteringProceedings of High Performance Graphics10.1145/3105762.3105770(1-12)Online publication date: 28-Jul-2017
https://dl.acm.org/doi/10.1145/3105762.3105770
Fascione LHanika JFajardo MChristensen PBurley BGreen B(2017)Path tracing in production - part 1ACM SIGGRAPH 2017 Courses10.1145/3084873.3084904(1-39)Online publication date: 30-Jul-2017
https://dl.acm.org/doi/10.1145/3084873.3084904
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents