research-article

Iterative Depth Warping

Authors:

Elmar EisemannAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 37, Issue 5

Article No.: 177, Pages 1 - 13

https://doi.org/10.1145/3190859

Published: 23 October 2018 Publication History

Abstract

This article presents an iterative backward-warping technique and its applications. It predictively synthesizes depth buffers for novel views. Our solution is based on a fixed-point iteration that converges quickly in practice. Unlike the previous techniques, our solution is a pure backward warping without using bidirectional sources. To efficiently seed the iterative process, we also propose a tight bounding method for motion vectors. Non-convergent depth holes are inpainted via deep depth buffers. Our solution works well with arbitrarily distributed motion vectors under moderate motions. Many scenarios can benefit from our depth warping. As an application, we propose a highly scalable image-based occlusion-culling technique, achieving a significant speed-up compared to the state of the art. We also demonstrate the benefit of our solution in multi-view soft-shadow generation.

Supplementary Material

lee (lee.zip)

Supplemental movie and image files for, Iterative Depth Warping

Download
96.50 MB

MP4 File (a177-lee.mp4)

Download
267.21 MB

References

[1]

Dmitry Andreev. 2010. Real-time frame rate up-conversion for video games: Or how to get from 30 to 60 fps for free. In Proceedings of the ACM SIGGRAPH 2010 Talks. ACM, 16.

Digital Library

[2]

Thaddeus Beier and Shawn Neely. 1992. Feature-based image metamorphosis. Proc. ACM SIGGRAPH 26, 2 (1992), 35--42.

Digital Library

[3]

Jiří Bittner, Michael Wimmer, Harald Piringer, and Werner Purgathofer. 2004. Coherent hierarchical culling: Hardware occlusion queries made useful. Comput. Graph. Forum 23, 3 (2004), 615--624.

[4]

Huw Bowles, Kenny Mitchell, Robert W. Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative image warping. Comput. Graph. Forum 31, 2pt1 (2012), 237--246.

[5]

Shenchang Eric Chen and Lance Williams. 1993. View interpolation for image synthesis. In Proceedings of the ACM SIGGRAPH. ACM, 279--288.

Digital Library

[6]

Adam Cichocki. 2017. Advances in Real-Time Rendering in Games. ACM SIGGRAPH Courses.

[7]

R. Cunniff, M. Craighead, D. Ginsburg, K. Lefebvre, B. Licea-Kane, and N. Triantos. 2001. ARB Occlusion Query. Technical Report. NVIDIA and ATI.

[8]

X. Décoret. 2005. N-buffers for efficient depth map query. In Proceedings of the Eurographics Conference. Wiley Online Library, 393--400.

[9]

Piotr Didyk, Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2010a. Perceptually-motivated real-time temporal upsampling of 3D content for high-refresh-rate displays. Comput. Graph. Forum 29, 2 (2010), 713--722.

[10]

Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2010b. Adaptive image-space stereo view synthesis. In Proceedings of the Conference on Vision, Modeling, and Visualization. 299--306.

[11]

Hawar Doghramachi and Jean-Normand Bucci. 2017. Deferred+: Next-Gen culling and rendering for dawn engine. In GPU Zen: Advanced Rendering Techniques, Wolfgang Engel (Ed.). Bowker Identifier Services.

[12]

Elmar Eisemann and Xavier Décoret. 2007. Visibility sampling on GPU and applications. Compu. Graph. Forum 26, 3 (2007), 535--544.

[13]

Cass Everitt. 2001. Interactive Order-Independent Transparency. NVIDIA Corporation.

[14]

Randima Fernando. 2005. Percentage-closer soft shadows. In ACM SIGGRAPH 2005 Sketches. ACM, 35.

Digital Library

[15]

Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. 1996. The lumigraph. In Proceedings of the ACM SIGGRAPH Conference. ACM, 43--54.

Digital Library

[16]

Stefan Gottschalk, Ming C. Lin, and Dinesh Manocha. 1996. OBBTree: A hierarchical structure for rapid interference detection. In Proceedings of the ACM SIGGRAPH Conference. ACM, 171--180.

Digital Library

[17]

Michael Guthe, Ákos Balázs, and Reinhard Klein. 2006. Near optimal hierarchical culling: Performance driven use of hardware occlusion queries. In Proceedings of the Eurographics Symposium on Rendering. 207--214.

Digital Library

[18]

Ulrich Haar and Sebastian Aaltonen. 2015. Advances in Real-Time Rendering in Games. ACM SIGGRAPH Courses.

[19]

Yun-Suk Kang and Yo-Sung Ho. 2010. High-quality multi-view depth generation using multiple color and depth cameras. In Proceedings of the International Conference on Multimedia and Expo. IEEE, 1405--1410.

[20]

Nickolay Kasyan, Nicolas Schulz, and Tiago Sousa. 2011. Secrets of CryENGINE 3 Graphics Technology. ACM SIGGRAPH Courses.

[21]

James T. Klosowski and Claudio T. Silva. 2000. The prioritized-layered projection algorithm for visible set estimation. IEEE Trans. Vis. Comput. Graph. 6, 2 (2000), 108--123.

Digital Library

[22]

Christoph Kubisch and Markus Tavenrath. 2014. OpenGL 4.4 Scene Rendering Techniques. GPU Technology Conference.

[23]

Sungkil Lee, Elmar Eisemann, and Hans-Peter Seidel. 2010. Real-time lens blur effects and focus control. ACM Trans. Graph. 29, 4 (2010), 65:1--7.

Digital Library

[24]

Sungkil Lee, Gerard Jounghyun Kim, and Seungmoon Choi. 2008. Real-time depth-of-field rendering using point splatting on per-pixel layers. Comput. Graph. Forum 27, 7 (2008), 1955--1962.

[25]

Marc Levoy and Pat Hanrahan. 1996. Light field rendering. In Proceedings of the ACM SIGGRAPH Conference. ACM, 31--42.

Digital Library

[26]

David Luebke and Chris Georges. 1995. Portals and mirrors: Simple, fast evaluation of potentially visible sets. In Proceedings of the ACM Symposium on Interactive 3D Graphics. ACM, 105--ff.

Digital Library

[27]

William R. Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proceedings of the Symposium on Interactive 3D Graphics. ACM, 7--ff.

Digital Library

[28]

Oliver Mattausch, Jiří Bittner, A. Jaspe, E. Gobbetti, Michael Wimmer, and Renato Pajarola. 2015. CHC+RT: Coherent hierarchical culling for ray tracing. Comput. Graph. Forum 34, 2 (2015), 537--548.

Digital Library

[29]

Oliver Mattausch, Jiří Bittner, and Michael Wimmer. 2008. CHC++: Coherent hierarchical culling revisited. Comput. Graph. Forum 27, 2 (2008), 221--230.

[30]

Oliver Mattausch, Daniel Scherzer, and Michael Wimmer. 2010. High-quality screen-space ambient occlusion using temporal coherence. Comput. Graph. Forum 29, 8 (2010), 2492--2503.

[31]

Leonard McMillan and Gary Bishop. 1995. Plenoptic modeling: An image-based rendering system. In Proceedings of the ACM SIGGRAPH Conference. ACM, 39--46.

Digital Library

[32]

Leonard McMillan Jr. 1997. An Image-Based Approach to Three-dimensional Computer Graphics. Ph.D. Dissertation. University of North Carolina Chapel Hill.

Digital Library

[33]

Diego Nehab, Pedro V. Sander, Jason Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating real-time shading with reverse reprojection caching. In Proceedings of the Conference on Graphics Hardware, Vol. 41. 61--62.

Digital Library

[34]

Ivan Neulander. 2008. Pismo: parallax-interpolated shadow map occlusion. ACM SIGGRAPH 2008 talks.

Digital Library

[35]

Matthias Nießner and Charles Loop. 2012. Patch-based occlusion culling for hardware tessellation. In Proceedings of the Computer Graphics International.

[36]

Christoph Peters and Reinhard Klein. 2015. Moment shadow mapping. In Proceedings of the Symposium on Interactive 3D Graphics and Games. ACM, 7--14.

Digital Library

[37]

Gilberto Rosado and Rainbow Studios. 2007. Motion blur as a post-processing effect. In GPU Gems 3, Hubert Nguyen (Ed.). Addison-Wesley Professional, 69--117.

[38]

Daniel Scherzer, Stefan Jeschke, and Michael Wimmer. 2007. Pixel-correct shadow maps with temporal reprojection and shadow test confidence. In Proceedings of the Eurographics Symposium on Rendering. Eurographics Association, 45--50.

Digital Library

[39]

Daniel Scherzer, Michael Schwärzler, Oliver Mattausch, and Michael Wimmer. 2009. Real-time soft shadows using temporal coherence. In Proceedings of the International Symposium on Visual Computing. Springer, 13--24.

Digital Library

[40]

Daniel Scherzer, Lei Yang, Oliver Mattausch, Diego Nehab, Pedro V. Sander, Michael Wimmer, and Elmar Eisemann. 2012. Temporal coherence methods in real-time rendering. Comput. Graph. Forum 31, 8 (2012), 2378--2408.

Digital Library

[41]

Michael Schwarz and Marc Stamminger. 2008. Quality scalability of soft shadow mapping. In Graphics Interface 2008. 147--154.

Digital Library

[42]

W Rrenl Seales, Greg Welch, and Christopher O. Jaynes. 1999. Real-time depth warping for 3-d scene reconstruction. In Proceedings of the IEEE Aerospace Conference, Vol. 3. IEEE, 413--419.

[43]

S. M. Seitz and C. R. Dyer. 1996. View morphing. In Proceedings of the ACM SIGGRAPH Conference. ACM, 21--30.

Digital Library

[44]

Li Shen, Jieqing Feng, and Baoguang Yang. 2013. Exponential soft shadow mapping. Comput. Graph. Forum 32, 4 (2013), 107--116.

Digital Library

[45]

Erik Sintorn, Elmar Eisemann, and Ulf Assarsson. 2008. Sample based visibility for soft shadows using alias-free shadow maps. Comput. Graph. Forum 27, 4 (2008), 1285--1292.

Digital Library

[46]

Pitchaya Sitthi-amorn, Jason Lawrence, Lei Yang, Pedro V. Sander, and Diego Nehab. 2008. An improved shading cache for modern GPUs. In Proceedings of the Symposium on Graphics Hardware. Eurographics Association, 95--101.

Digital Library

[47]

Sundar Vedula, Simon Baker, and Takeo Kanade. 2002. Spatio-temporal view interpolation. In Proceedings of the Rendering Techniques. 65--76.

Digital Library

[48]

Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Proc. 13, 4 (2004), 600--612.

Digital Library

[49]

Lance Williams. 1983. Pyramidal parametrics. Proc. ACM SIGGRAPH 17, 3 (1983), 1--11.

Digital Library

[50]

Peter Wonka, Michael Wimmer, and Dieter Schmalstieg. 2000. Visibility preprocessing with occluder fusion for urban walkthroughs. In Proceedings of the Eurographics Workshop on Rendering. 71--82.

Digital Library

[51]

Baoguang Yang, Zhao Dong, Jieqing Feng, Hans-Peter Seidel, and Jan Kautz. 2010. Variance soft shadow mapping. Comput. Graph. Forum 29, 7 (2010), 2127--2134.

[52]

Lei Yang, Yu-Chiu Tse, Pedro V. Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L Wilkins. 2011. Image-based bidirectional scene reprojection. ACM Trans. Graph. 30, 6 (2011), 150:1--10.

Digital Library

[53]

Xuan Yu, Rui Wang, and Jingyi Yu. 2010. Real-time depth of field rendering via dynamic light field generation and filtering. Comput. Graph. Forum 29, 7 (2010), 2099--2107.

[54]

Sveta Zinger, Luat Do, and P. H. N. de With. 2010. Free-viewpoint depth image based rendering. J. Vis. Commun. Image Represent. 21, 5 (2010), 533--541.

Digital Library

[55]

Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. 2002. EWA splatting. IEEE Trans. Vis. Comput. Graph. 8, 3 (2002), 223--238.

Digital Library

Cited By

Voglreiter PKerbl BWeinrauch AMueller JNeff TSteinberger MSchmalstieg D(2023)Trim Regions for Online Computation of From-Region Potentially Visible SetsACM Transactions on Graphics10.1145/359243442:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592434
Kim JLee S(2023)Potentially Visible Hidden-Volume Rendering for Multi-View WarpingACM Transactions on Graphics10.1145/359210842:4(1-11)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592108
Kreskowski ARendle GFroehlich B(2023)Efficient Direct Isosurface Rasterization of Scalar VolumesComputer Graphics Forum10.1111/cgf.1467041:7(215-226)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14670
Show More Cited By

Index Terms

Iterative Depth Warping
1. Computing methodologies
  1. Computer graphics
    1. Rendering
      1. Rasterization
      2. Visibility

Recommendations

Real-time watercolor illustrations of plants using a blurred depth test
NPAR '06: Proceedings of the 4th international symposium on Non-photorealistic animation and rendering

We present techniques to create convincing high-quality watercolor illustrations of plants. Mainly focusing on the real-time rendering, we introduce methods to abstract the visual and geometrical complexity of the original 3D plant models that are ...
The depth discontinuity occlusion camera
I3D '06: Proceedings of the 2006 symposium on Interactive 3D graphics and games

Rendering a scene using a single depth image suffers from disocclusion errors as the view translates away from the reference view. We present the depth discontinuity occlusion camera (DDOC), a non-pinhole camera that samples surfaces which are hidden in ...
Accelerating shadow rays using volumetric occluders and modified kd-tree traversal
HPG '09: Proceedings of the Conference on High Performance Graphics 2009

Monte Carlo ray tracing remains a simple and elegant method for generating robust shadows. This approach, however, is often hampered by the time needed to evaluate the numerous shadow ray queries required to generate a high-quality image. We propose the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 37, Issue 5

October 2018

140 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3278329

Editor:
Marc Alexa
TU Berlin

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2018

Accepted: 01 July 2018

Revised: 01 June 2018

Received: 01 September 2017

Published in TOG Volume 37, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Mid-Career Program, National Research Foundation of Korea
Global Frontier Program, National Research Foundation of Korea
NWO Vernieuwingsimpuls

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
711
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)10

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Voglreiter PKerbl BWeinrauch AMueller JNeff TSteinberger MSchmalstieg D(2023)Trim Regions for Online Computation of From-Region Potentially Visible SetsACM Transactions on Graphics10.1145/359243442:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592434
Kim JLee S(2023)Potentially Visible Hidden-Volume Rendering for Multi-View WarpingACM Transactions on Graphics10.1145/359210842:4(1-11)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592108
Kreskowski ARendle GFroehlich B(2023)Efficient Direct Isosurface Rasterization of Scalar VolumesComputer Graphics Forum10.1111/cgf.1467041:7(215-226)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14670
Ke ZZhou XJiang DYan HQiu T(2023)CollabVr: Reprojection-Based Edge-Client Collaborative Rendering for Real-Time High-Quality Mobile Virtual Reality2023 IEEE Real-Time Systems Symposium (RTSS)10.1109/RTSS59052.2023.00034(304-316)Online publication date: 5-Dec-2023
https://doi.org/10.1109/RTSS59052.2023.00034
Li FLiu SMa NLiu YXing GZhang Y(2023)A GPU-friendly hybrid occlusion culling algorithm for large scenesDisplays10.1016/j.displa.2023.10253380(102533)Online publication date: Dec-2023
https://doi.org/10.1016/j.displa.2023.102533
Chlubna TMilet TZemčík PKula M(2023)Real-Time Light Field Video Focusing and GPU Accelerated StreamingJournal of Signal Processing Systems10.1007/s11265-023-01874-895:6(703-719)Online publication date: 23-May-2023
https://dl.acm.org/doi/10.1007/s11265-023-01874-8
Schertzer JMercier CRousseau SBoubekeur T(2022)Fiblets for Real‐Time Rendering of Massive Brain TractogramsComputer Graphics Forum10.1111/cgf.1448641:2(447-460)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14486
Jeong YBaek SSeok YLee GLee S(2022)Real-Time Dynamic Bokeh Rendering With Efficient Look-Up Table SamplingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.301447428:2(1373-1384)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TVCG.2020.3014474
Xie FBieron JPeers PHanrahan P(2021)Experimental Analysis of Multiple Scattering BRDF ModelsSIGGRAPH Asia 2021 Technical Communications10.1145/3478512.3488601(1-4)Online publication date: 14-Dec-2021
https://dl.acm.org/doi/10.1145/3478512.3488601
Yin ZYang ZVan De Panne MYin K(2021)Discovering diverse athletic jumping strategiesACM Transactions on Graphics10.1145/3476576.347665040:4(1-17)Online publication date: Aug-2021
https://doi.org/10.1145/3476576.3476650
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents