Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing Reconstruction

Published: 17 June 2019 Publication History

Abstract

Path tracing produces realistic results including global illumination using a unified simple rendering pipeline. Reducing the amount of noise to imperceptible levels without post-processing requires thousands of samples per pixel (spp), while currently it is only possible to render extremely noisy 1 spp frames in real time with desktop GPUs. However, post-processing can utilize feature buffers, which contain noise-free auxiliary data available in the rendering pipeline. Previously, regression-based noise filtering methods have only been used in offline rendering due to their high computational cost. In this article we propose a novel regression-based reconstruction pipeline, called Blockwise Multi-Order Feature Regression (BMFR), tailored for path-traced 1 spp inputs that runs in real time. The high speed is achieved with a fast implementation of augmented QR factorization and by using stochastic regularization to address rank-deficient feature data. The proposed algorithm is 1.8× faster than the previous state-of-the-art real-time path-tracing reconstruction method while producing better quality frame sequences.

Supplementary Material

koskela (koskela.zip)
Supplemental movie, appendix, image and software files for, Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing Reconstruction
MP4 File (a138-koskela.mp4)

References

[1]
Anne Aaron, Zhi Li, Megha Manohara, Joe Yuchieh Lin, Eddy Chi-Hao Wu, and C.-C. Jay Kuo. 2015. Challenges in cloud based ingest and encoding for high quality streaming media. In Proceedings of the Annual Conference on Image Processing.
[2]
Timo Aila and Tero Karras. 2010. Architecture considerations for tracing incoherent rays. In Proceedings of the Annual Conference on High Performance Graphics.
[3]
Chakravarty Alla Chaitanya, Anton Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. Trans. Graph. 36, 4 (2017).
[4]
AMD. 2017. RadeonRays SDK. Online. Retrieved January 23, 2018 from https://github.com/GPUOpen-LibrariesAndSDKs/RadeonRays_SDK.
[5]
Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan NováK, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. Trans. Graph. 36, 4 (2017).
[6]
Pablo Bauszat, Martin Eisemann, and Marcus Magnor. 2011. Guided image filtering for interactive high-quality global illumination. Comput. Graph. Forum 30, 4 (2011).
[7]
Benedikt Bitterli. 2016. Rendering Resources. Retrieved from https://benedikt-bitterli.me/resources/.
[8]
Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, José A Iglesias-Guitián, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Novák. 2016. Nonlinearly weighted first-order regression for denoising monte carlo renderings. Comput. Graph. Forum 35, 4 (2016).
[9]
Peter Burt. 1981. Fast filter transform for image processing. Comput. Graph. Image Process. 16, 1 (1981).
[10]
Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik Lensch. 2010. Edge-avoiding À-Trous wavelet transform for fast global illumination filtering. In Proceedings of the Annual Conference on High Performance Graphics.
[11]
Kevin Egan, Yu-Ting Tseng, Nicolas Holzschuch, Frédo Durand, and Ravi Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. Trans. Graph. 28, 3 (2009), 93.
[12]
Luke Goddard. 2014. Silencing the noise on elysium. In Proceedings of the ACM SIGGRAPH 2014 Talks.
[13]
Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. Trans. Pattern Anal. Mach. Intell. 35, 6 (2013).
[14]
Michael Heath. 1997. Scientific Computing. McGraw–Hill.
[15]
Jorge Jimenez, Jose I. Echevarria, Tiago Sousa, and Diego Gutierrez. 2012. SMAA: Enhanced morphological antialiasing. Comput. Graph. Forum (Proc. EUROGRAPHICS 2012) 31, 2 (2012).
[16]
Jorge Jiménez, X. Wu, A. Pesce, and A. Jarabo. 2016. Practical real-time strategies for accurate indirect occlusion. SIGGRAPH 2016 Courses: Physically Based Shading in Theory and Practice (2016).
[17]
Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A machine learning approach for filtering Monte Carlo noise. Trans. Graph. 34, 4 (2015).
[18]
Anton Kaplanyan and Carsten Dachsbacher. 2013. Path space regularization for holistic and robust light transport. Comput. Graph. Forum 32, 2pt1 (2013).
[19]
Brian Karis. 2014. High-quality temporal supersampling. In Proceedings of the ACM SIGGRAPH Advances in Real-Time Rendering in Games 2014.
[20]
Samuli Laine, Tero Karras, and Timo Aila. 2013. Megakernels considered harmful: Wavefront path tracing on GPUs. In Proceedings of the Annual Conference on High Performance Graphics.
[21]
Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012. SURE-based optimization for adaptive sampling and reconstruction. Trans. Graph. 31, 6 (2012).
[22]
Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward a Practical Perceptual Video Quality Metric. Retrieved January 23, 2018 from https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652.
[23]
Yu Liu, Changwen Zheng, Quan Zheng, and Hongliang Yuan. 2017. Removing Monte Carlo noise using a Sobel operator and a guided image filter. Vis. Comput. 34, 4 (2017).
[24]
Michael Mara, Morgan McGuire, Benedikt Bitterli, and Wojciech Jarosz. 2017. An efficient denoising algorithm for global illumination. In Proceedings of the Annual Conference on High Performance Graphics.
[25]
Morgan McGuire. 2017. Computer Graphics Archive. Retrieved from https://casual-effects.com/data.
[26]
Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive rendering based on weighted local regression. Trans. Graph. 33, 5 (2014).
[27]
Bochang Moon, Jose A Iglesias-Guitian, Sung-Eui Yoon, and Kenny Mitchell. 2015. Adaptive rendering with linear predictions. Trans. Graph. 34, 4 (2015).
[28]
Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive polynomial rendering. Trans. Graph. 35, 4 (2016).
[29]
Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, et al. 2010. Optix: A general purpose ray tracing engine. Trans. Graph. 29, 4 (2010).
[30]
Amar Patel. 2018. D3D12 Raytracing Functional Spec, v0.09. Microsoft. Retrieved March 23, 2018 Available: http://forums.directxtech.com/index.php?topic=5860.0.
[31]
Matt Pharr and Greg Humphreys. 2010. Physically Based Rendering: From Theory to Implementation (2nd ed.). Morgan Kaufmann.
[32]
Amy R. Reibman and David Poole. 2007. Predicting packet-loss visibility using scene characteristics. In Proceedings of the Annual Conference on Packet Video.
[33]
Gilberto Rosado. 2007. Motion blur as a post-processing effect. In GPU Gems 3. Addison-Wesley Professional, Chapter 27.
[34]
Christoph Schied, Anton Kaplanyan, Chris Wyman, Anjul Patney, Chakravarty R. Alla Chaitanya, John Burgess, Shiqiu Liu, Carsten Dachsbacher, Aaron Lefohn, and Marco Salvi. 2017. Spatiotemporal variance-guided filtering: Real-time reconstruction for path-traced global illumination. In Proceedings of the Annual Conference on High Performance Graphics.
[35]
Christoph Schied, Christoph Peters, and Carsten Dachsbacher. 2018. Gradient estimation for real-time adaptive temporal filtering. Proceedings of the ACM Annual Conference on Computer Graphics and Interactive Techniques 1, 2 (2018), 24.
[36]
Carlo Tomasi and Roberto Manduchi. 1998. Bilateral filtering for gray and color images. In Proceedings of the Annual Conference on Computer Vision.
[37]
Eric Veach and Leonidas J Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques.
[38]
Timo Viitanen, Matias Koskela, Kalle Immonen, Markku Mäkitalo, Pekka Jääskeläinen, and Jarmo Takala. 2018. Sparse sampling for real-time ray tracing. In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP’18).
[39]
Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014. Embree: A kernel framework for efficient CPU ray tracing. Trans. Graph. 33, 4 (2014).
[40]
Yong Wang, Xiaofeng Liao, Di Xiao, and Kwok-Wo Wong. 2008. One-way hash function construction based on 2D coupled map lattices. Inf. Sci. 178, 5 (2008).
[41]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. Trans. Image Process. 13, 4 (2004).
[42]
Ling-Qi Yan, Soham Uday Mehta, Ravi Ramamoorthi, and Fredo Durand. 2015. Fast 4D sheared filtering for interactive rendering of distribution effects. Trans. Graph. 35, 1 (2015), 7.
[43]
Lei Yang, Diego Nehab, Pedro V. Sander, Pitchaya Sitthi-amorn, Jason Lawrence, and Hugues Hoppe. 2009. Amortized supersampling. Trans. Graph. 28, 5 (2009).
[44]
Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-Hornung, and Alexander Sorkine-Hornung. 2015. Path-space motion estimation and decomposition for robust animation filtering. Comput. Graph. Forum 34, 4 (2015).
[45]
Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S.-E. Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Comput. Graph. Forum 34, 2 (2015).

Cited By

View all
  • (2024)Foveated Path Culling: A mixed path tracing and radiance field approach for optimizing rendering in XR DisplaysJournal on Interactive Systems10.5753/jis.2024.435215:1(576-590)Online publication date: 18-Jun-2024
  • (2024)Physically-based Path Tracer using WebGPU and OpenPBRProceedings of the 29th International ACM Conference on 3D Web Technology10.1145/3665318.3677158(1-6)Online publication date: 25-Sep-2024
  • (2024)Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer BlocksACM Transactions on Graphics10.1145/365821843:4(1-14)Online publication date: 19-Jul-2024
  • Show More Cited By

Index Terms

  1. Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing Reconstruction

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 38, Issue 5
      October 2019
      191 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3341165
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 17 June 2019
      Accepted: 01 April 2019
      Revised: 01 January 2019
      Received: 01 April 2018
      Published in TOG Volume 38, Issue 5

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Path tracing
      2. real-time
      3. reconstruction
      4. regression

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Funding Sources

      • TUT Graduate School
      • Nokia Foundation
      • Finnish Foundation for Technology Promotion
      • FiDiPro-StreamPro
      • Business Finland
      • ECSEL JU project FitOptiVis
      • Emil Aaltonen Foundation
      • Academy of Finland

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)105
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 03 Sep 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Foveated Path Culling: A mixed path tracing and radiance field approach for optimizing rendering in XR DisplaysJournal on Interactive Systems10.5753/jis.2024.435215:1(576-590)Online publication date: 18-Jun-2024
      • (2024)Physically-based Path Tracer using WebGPU and OpenPBRProceedings of the 29th International ACM Conference on 3D Web Technology10.1145/3665318.3677158(1-6)Online publication date: 25-Sep-2024
      • (2024)Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer BlocksACM Transactions on Graphics10.1145/365821843:4(1-14)Online publication date: 19-Jul-2024
      • (2024)Denoising 1SPP Monte carlo renderings based on human visual perceptionThird International Conference on Electronic Information Engineering and Data Processing (EIEDP 2024)10.1117/12.3032848(56)Online publication date: 5-Jul-2024
      • (2024)Real-Time Monte Carlo Denoising With Adaptive Fusion NetworkIEEE Access10.1109/ACCESS.2024.336958812(29154-29165)Online publication date: 2024
      • (2024)All-optical image denoising using a diffractive visual processorLight: Science & Applications10.1038/s41377-024-01385-613:1Online publication date: 4-Feb-2024
      • (2023)Neural Partitioning Pyramids for Denoising Monte Carlo RenderingsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591562(1-11)Online publication date: 23-Jul-2023
      • (2023)Real‐time Deep Radiance Reconstruction from Imperfect CachesComputer Graphics Forum10.1111/cgf.1467541:7(267-278)Online publication date: 20-Mar-2023
      • (2023)TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-ResolutionIEEE Access10.1109/ACCESS.2023.332327311(112015-112026)Online publication date: 2023
      • (2023)Non-homogeneous denoising for virtual reality in real-time path tracing renderingGraphical Models10.1016/j.gmod.2023.101184129(101184)Online publication date: Oct-2023
      • Show More Cited By

      View Options

      Get Access

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media