research-article

Recursive Control Variates for Inverse Rendering

Authors:

Baptiste Nicolet,

Fabrice Rousselle,

Alexander Keller,

Thomas MüllerAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 42, Issue 4

Article No.: 62, Pages 1 - 13

https://doi.org/10.1145/3592139

Published: 26 July 2023 Publication History

Abstract

We present a method for reducing errors---variance and bias---in physically based differentiable rendering (PBDR). Typical applications of PBDR repeatedly render a scene as part of an optimization loop involving gradient descent. The actual change introduced by each gradient descent step is often relatively small, causing a significant degree of redundancy in this computation. We exploit this redundancy by formulating a gradient estimator that employs a recursive control variate, which leverages information from previous optimization steps. The control variate reduces variance in gradients, and, perhaps more importantly, alleviates issues that arise from differentiating loss functions with respect to noisy inputs, a common cause of drift to bad local minima or divergent optimizations. We experimentally evaluate our approach on a variety of path-traced scenes containing surfaces and volumes and observe that primal rendering efficiency improves by a factor of up to 10.

Supplementary Material

MP4 File (papers_400_VOD.mp4)

presentation

Download
172.08 MB

References

[1]

Dejan Azinovic, Tzu-Mao Li, Anton Kaplanyan, and Matthias Nießner. 2019. Inverse path tracing for joint material and lighting estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2447--2456.

[2]

Sai Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased Warped-Area Sampling for Differentiable Rendering. ACM Trans. Graph. 39, 6 (2020), 245:1--245:18.

Digital Library

[3]

Sai Praveen Bangaru, Michael Gharbi, Fujun Luan, Tzu-Mao Li, Kalyan Sunkavalli, Milos Hasan, Sai Bi, Zexiang Xu, Gilbert Bernstein, and Fredo Durand. 2022. Differentiable Rendering of Neural SDFs through Reparameterization. In SIGGRAPH Asia 2022 Conference Papers. 1--9.

Digital Library

[4]

Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. 2021. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. arXiv (2021). https://jonbarron.info/mipnerf/

[5]

Nikolaus Binder, Sascha Fricke, and Alexander Keller. 2022. Massively Parallel Path Space Filtering. In Monte Carlo and Quasi-Monte Carlo Methods, MCQMC 2020, Oxford, United Kingdom, August 10--14, Alexander Keller (Ed.). Springer, 149--168.

[6]

Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. 2020. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting. ACM Transactions on Graphics (TOG) 39, 4 (2020), 148--1.

Digital Library

[7]

Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Trans. Graph. 36, 4 (jul 2017).

[8]

Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. 2020. Towards Learning-based Inverse Subsurface Scattering. 1--12.

[9]

Miguel Crespo, Adrian Jarabo, and Adolfo Muñoz. 2021. Primary-space adaptive control variates using piecewise-polynomial approximations. ACM Transactions on Graphics (TOG) 40, 3 (2021), 1--15.

Digital Library

[10]

Ido Czerninski and Yoav Schechner. 2023. PARS - Path Recycling and Sorting for Efficient Cloud Tomography. Intelligent Computing 0, ja (2023). arXiv:https://spj.science.org/doi/pdf/10.34133/icomputing.0007

[11]

Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. 2014. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. Advances in neural information processing systems 27 (2014).

[12]

Xi Deng, Fujun Luan, Bruce Walter, Kavita Bala, and Steve Marschner. 2022. Reconstructing Translucent Objects using Differentiable Rendering. In ACM SIGGRAPH 2022 Conference Proceedings. 1--10.

Digital Library

[13]

Addis Dittebrandt, Johannes Hanika, and Carsten Dachsbacher. 2020. Temporal Sample Reuse for Next Event Estimation and Path Guiding for Real-Time Path Tracing. In Eurographics Symposium on Rendering - DL-only Track, Carsten Dachsbacher and Matt Pharr (Eds.). The Eurographics Association.

[14]

Oskar Elek, Tobias Ritschel, Alexander Wilkie, and Hans-Peter Seidel. 2012. Interactive Cloud Rendering Using Temporally-Coherent Photon Mapping. In Proceedings of Graphics Interface 2012 (Toronto, Ontario, Canada) (GI '12). Canadian Information Processing Society, CAN, 141--148.

Digital Library

[15]

Shaohua Fan, Stephen Chenney, Bo Hu, Kam-Wah Tsui, and Yu-chi Lai. 2006. Optimizing control variate estimators for rendering. In Computer Graphics Forum, Vol. 25. Wiley Online Library, 351--357.

[16]

Edgar C. Fieller and H. O. Hartley. 1954. Sampling with control variables. Biometrika 41, 3--4 (12 1954).

[17]

Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Jaroslav Křivánek, and Wojciech Jarosz. 2019. Integral formulations of volumetric transmittance. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--17.

Digital Library

[18]

Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse volume rendering with material dictionaries. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1--13.

Digital Library

[19]

Robert M. Gower, Mark Schmidt, Francis Bach, and Peter Richtárik. 2020. Variance-reduced methods for machine learning. Proc. IEEE 108, 11 (2020), 1968--1983.

[20]

Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. 2022. Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising. arXiv:2206.03380 (2022).

[21]

Jon Hasselgren, Jacob Munkberg, Jaakko Lehtinen, Miika Aittala, and Samuli Laine. 2021. Appearance-Driven Automatic 3D Model Simplification. In Eurographics Symposium on Rendering.

[22]

Jon Hasselgren, Jacob Munkberg, Marco Salvi, Anjul Patney, and Aaron Lefohn. 2020. Neural Temporal Adaptive Sampling and Denoising. Computer Graphics Forum 39, 2 (2020), 147--155. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13919

[23]

Stefan Heinrich. 1998. A Multilevel Version of the Method of Dependent Tests. In Proc. of the 3rd St. Petersburg Workshop on Simulation. St. Petersburg University Press, 31--35.

[24]

Eric Heitz, Kenneth Vanhoey, Thomas Chambon, and Laurent Belcour. 2021. A Sliced Wasserstein Loss for Neural Texture Synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9412--9420.

[25]

Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. 2022b. Mitsuba 3 renderer. https://mitsuba-renderer.org.

[26]

Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, and Delio Vicini. 2022a. Dr.Jit: A Just-In-Time Compiler for Differentiable Rendering. Transactions on Graphics (Proceedings of SIGGRAPH) 41, 4 (July 2022).

Digital Library

[27]

Johan L. W. V. Jensen. 1906. Sur les fonctions convexes et les inégualités entre les valeurs Moyennes.

[28]

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. In Computer Vision - ECCV 2016. Springer International Publishing.

[29]

Rie Johnson and Tong Zhang. 2013. Accelerating Stochastic Gradient Descent using Predictive Variance Reduction. In Advances in Neural Information Processing Systems, C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc.

[30]

Simon Kallweit, Thomas Müller, Brian Mcwilliams, Markus Gross, and Jan Novák. 2017. Deep scattering: Rendering atmospheric clouds with radiance-predicting neural networks. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1--11.

Digital Library

[31]

Malvin Kalos and Paula Whitlock. 1986. Monte Carlo Methods, Volume I: Basics. J. Wiley & Sons.

[32]

Brian Karis. 2014. High Quality Temporal Anti-Aliasing. In ACM SIGGRAPH Courses: Advances in Real-Time Rendering in Games (Vancouver, Canada). ACM, New York, NY, USA.

Digital Library

[33]

Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2017. Neural 3D Mesh Renderer. CoRR abs/1711.07566 (2017). arXiv:1711.07566 http://arxiv.org/abs/1711.07566

[34]

Csaba Kelemen, László Szirmay-Kalos, György Antal, and Ferenc Csonka. 2002. A Simple and Robust Mutation Strategy for the Metropolis Light Transport Algorithm. Comput. Graph. Forum 21, 3 (2002).

[35]

Alexander Keller. 2001. Hierarchical Monte Carlo Image Synthesis. Mathematics and Computers in Simulation 55, 1--3 (2001), 79--92.

Digital Library

[36]

Markus Kettunen, Marco Manzi, Miika Aittala, Jaakko Lehtinen, Frédo Durand, and Matthias Zwicker. 2015. Gradient-domain path tracing. ACM Transactions on Graphics (TOG) 34, 4 (2015), 1--13.

Digital Library

[37]

Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2016. Matching Real Fabrics with Micro-Appearance Models. ACM Trans. Graph. 35, 1, Article 1 (dec 2016), 26 pages.

Digital Library

[38]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster).

[39]

Ivo Kondapaneni, Petr Vévoda, Pascal Grittmann, Tomáš Skřivan, Philipp Slusallek, and Jaroslav Křivánek. 2019. Optimal multiple importance sampling. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--14.

Digital Library

[40]

Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and decomposition tracking for rendering heterogeneous volumes. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1--16.

Digital Library

[41]

Eric P. Lafortune and Yves D. Willems. 1995a. A 5D tree to reduce the variance of Monte Carlo ray tracing. In Eurographics Workshop on Rendering Techniques. Springer, 11--20.

[42]

Eric P. Lafortune and Yves D. Willems. 1995b. The ambient term as a variance reducing technique for Monte Carlo ray tracing. In Photorealistic Rendering Techniques. Springer, 168--176.

[43]

Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular primitives for high-performance differentiable rendering. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--14.

Digital Library

[44]

Stephen S. Lavenberg, Thomas L. Moeller, and Peter D. Welch. 1982. Statistical Results on Control Variables with Application to Queueing Network Simulation. Operations Research 30, 1 (1982), 182--202.

Digital Library

[45]

Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1--222:11.

[46]

Daqi Lin, Markus Kettunen, Benedikt Bitterli, Jacopo Pantaleoni, Cem Yuksel, and Chris Wyman. 2022. Generalized resampled importance sampling: foundations of ReSTIR. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1--23.

Digital Library

[47]

Shichen Liu, Weikai Chen, Tianye Li, and Hao Li. 2019. Soft Rasterizer: Differentiable Rendering for Unsupervised Single-View Mesh Reconstruction. CoRR abs/1901.05567 (2019). arXiv:1901.05567 http://arxiv.org/abs/1901.05567

[48]

Matthew M. Loper and Michael J. Black. 2014. OpenDR: An approximate differentiable renderer. In European Conference on Computer Vision. Springer.

[49]

Zander Majercik, Thomas Müller, Alexander Keller, Derek Nowrouzezahrai, and Morgan McGuire. 2022. Dynamic Diffuse Global Illumination Resampling. Computer Graphics Forum 41, 1 (2022), 158--171.

[50]

Marco Manzi, Markus Kettunen, Frédo Durand, Matthias Zwicker, and Jaakko Lehtinen. 2016. Temporal Gradient-Domain Path Tracing. ACM Trans. Graph. 35, 6, Article 246 (dec 2016), 9 pages.

Digital Library

[51]

Adam Marrs, Josef Spjut, Holger Gruen, Rahul Sathe, and Morgan McGuire. 2018. Adaptive Temporal Antialiasing. In Proceedings of the Conference on High-Performance Graphics (Vancouver, British Columbia, Canada) (HPG '18). Association for Computing Machinery, New York, NY, USA, Article 1, 4 pages.

Digital Library

[52]

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.

[53]

Bailey Miller, Iliyan Georgiev, and Wojciech Jarosz. 2019. A null-scattering path integral formulation of light transport. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--13.

Digital Library

[54]

Zackary Misso, Benedikt Bitterli, Iliyan Georgiev, and Wojciech Jarosz. 2022. Unbiased and consistent rendering using biased estimators. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1--13.

Digital Library

[55]

Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages.

Digital Library

[56]

Thomas Müller, Fabrice Rousselle, Alexander Keller, and Jan Novák. 2020. Neural control variates. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--19.

Digital Library

[57]

Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. 40, 4, Article 36 (Aug. 2021), 36:1--36:16 pages.

Digital Library

[58]

Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. 2021. Extracting Triangular 3D Models, Materials, and Lighting From Images. arXiv:2111.12503 (2021).

[59]

Barry L. Nelson. 1990. Control variate remedies. Operations Research 38, 6 (1990), 974--992.

Digital Library

[60]

Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large steps ininverse rendering of geometry. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1--13.

Digital Library

[61]

Merlin Nimier-David, Thomas Müller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased Inverse Volume Rendering with Differential Trackers. ACM Trans. Graph. 41, 4, Article 44 (July 2022), 20 pages.

Digital Library

[62]

Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering. Transactions on Graphics (Proceedings of SIGGRAPH) 39, 4 (July 2020).

Digital Library

[63]

Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A retargetable forward and inverse renderer. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--17.

Digital Library

[64]

Jan Novák, Andrew Selle, and Wojciech Jarosz. 2014. Residual ratio tracking for estimating attenuation in participating media. ACM Trans. Graph. 33, 6 (2014), 179--1.

Digital Library

[65]

Art B. Owen. 2013. Monte Carlo theory, methods and examples.

[66]

Felix Petersen, Amit H. Bermano, Oliver Deussen, and Daniel Cohen-Or. 2019. Pix2Vex: Image-to-Geometry Reconstruction using a Smooth Differentiable Renderer. CoRR abs/1903.11149 (2019). arXiv:1903.11149 http://arxiv.org/abs/1903.11149

[67]

Stanislav Pidhorskyi, Timur Bagautdinov, Shugao Ma, Jason Saragih, Gabriel Schwartz, Yaser Sheikh, and Tomas Simon. 2022. Depth of Field Aware Differentiable Rendering. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1--18.

Digital Library

[68]

Helge Rhodin, Nadia Robertini, Christian Richardt, Hans-Peter Seidel, and Christian Theobalt. 2015. A Versatile Scene Model with Differentiable Visibility Applied to Generative Pose Estimation. In Proceedings of ICCV 2015.

Digital Library

[69]

Fabrice Rousselle, Wojciech Jarosz, and Jan Novák. 2016. Image space control variates for rendering. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--12.

Digital Library

[70]

Corentin Salaün, Adrien Gruson, Binh-Son Hua, Toshiya Hachisuka, and Gurprit Singh. 2022. Regression-based Monte Carlo integration. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1--14.

Digital Library

[71]

Christoph Schied, Christoph Peters, and Carsten Dachsbacher. 2018. Gradient Estimation for Real-Time Adaptive Temporal Filtering. Proc. ACM Comput. Graph. Interact. Tech. 1, 2, Article 24 (aug 2018), 16 pages.

Digital Library

[72]

Erich Schubert and Michael Gertz. 2018. Numerically stable parallel computation of (co-) variance. In Proceedings of the 30th International Conference on Scientific and Statistical Database Management. 1--12.

Digital Library

[73]

Erich Schubert, Michael Weiler, and Hans-Peter Kriegel. 2014. SigniTrend: Scalable Detection of Emerging Topics in Textual Streams by Hashed Significance Thresholds. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 871--880.

Digital Library

[74]

Erich Schubert, Michael Weiler, and Hans-Peter Kriegel. 2016. SPOTHOT: Scalable detection of geo-spatial events in large textual streams. In Proceedings of the 28th International Conference on Scientific and Statistical Database Management. 1--12.

Digital Library

[75]

Dario Seyb, Peter-Pike Sloan, Ari Silvennoinen, Michał Iwanicki, and Wojciech Jarosz. 2020. The Design and Evolution of the UberBake Light Baking System. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 39, 4 (July 2020).

Digital Library

[76]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations.

[77]

László Szécsi, Mateu Sbert, and László Szirmay-Kalos. 2004. Combined correlated and importance sampling in direct light source computation and environment mapping. In Computer Graphics Forum, Vol. 23. Wiley Online Library, 585--593.

[78]

Justin Talbot, David Cline, and Parris Egbert. 2005. Importance Resampling for Global Illumination. In Eurographics Symposium on Rendering (2005), Kavita Bala and Philip Dutre (Eds.). The Eurographics Association.

[79]

Eric Veach. 1997. Robust Monte Carlo Methods for Light Transport Simulation. Ph. D. Dissertation. Stanford University, Stanford, CA.

[80]

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2021. Path Replay Backpropagation: Differentiating Light Paths using Constant Memory and Linear Time. Transactions on Graphics (Proceedings of SIGGRAPH) 40, 4 (Aug. 2021), 108:1--108:14.

Digital Library

[81]

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2022. Differentiable Signed Distance Function Rendering. Transactions on Graphics (Proceedings of SIGGRAPH) 41, 4 (July 2022), 125:1--125:18.

Digital Library

[82]

Thijs Vogels, Fabrice Rousselle, Brian Mcwilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Trans. Graph. 37, 4, Article 124 (jul 2018), 15 pages.

Digital Library

[83]

B. P. Welford. 1962. Note on a method for calculating corrected sums of squares and products. Technometrics 4, 3 (1962), 419--420.

[84]

E. Woodcock, T. Murphy, P. Hemmings, and S. Longworth. 1965. Techniques used in the GEM code for Monte Carlo neutronics calculations in reactors and other systems of complex geometry. In Proceedings of the Conference on Applications of Computing Methods to Reactor Problems. Argonne National Laboratory, 557.

[85]

Tiantian Xie and Marc Olano. 2021. Real-time Subsurface Control Variates: Temporally Stable Adaptive Sampling. UMBC Student Collection (2021).

Digital Library

[86]

Jiankai Xing, Fujun Luan, Ling-Qi Yan, Xuejun Hu, Houde Qian, and Kun Xu. 2022. Differentiable Rendering Using RGBXY Derivatives and Optimal Transport. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1--13.

Digital Library

[87]

Alex Yu, Sara Fridovich-Keil, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 2021. Plenoxels: Radiance Fields without Neural Networks. arXiv:2112.05131 (Dec. 2021).

[88]

Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte Carlo Estimators for Differential Light Transport. Transactions on Graphics (Proceedings of SIGGRAPH) 40, 4 (Aug. 2021).

Digital Library

[89]

Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020. Path-Space Differentiable Rendering. ACM Trans. Graph. 39, 4 (2020), 143:1--143:19.

Digital Library

[90]

Cheng Zhang, Zihan Yu, and Shuang Zhao. 2021. Path-Space Differentiable Rendering of Participating Media. ACM Trans. Graph. 40, 4 (2021), 76:1--76:15.

Digital Library

Cited By

Tran DJensen MSantafé-Gabarda PKällberg SFerrero AHannemose MFrisvad J(2024)Digitizing translucent object appearance by validating computed optical propertiesApplied Optics10.1364/AO.52197463:16(4317)Online publication date: 22-May-2024
https://doi.org/10.1364/AO.521974
Gu JBack JYoon SMoon B(2024)Target-Aware Image Denoising for Inverse Monte Carlo RenderingACM Transactions on Graphics10.1145/365818243:4(1-11)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658182
Fischer MRitschel T(2024)ZeroGrads: Learning Local Surrogates for Non-Differentiable GraphicsACM Transactions on Graphics10.1145/365817343:4(1-15)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658173
Show More Cited By

Index Terms

Recursive Control Variates for Inverse Rendering
1. Computing methodologies
  1. Computer graphics
    1. Rendering

Recommendations

Parameter-space ReSTIR for Differentiable and Inverse Rendering
SIGGRAPH '23: ACM SIGGRAPH 2023 Conference Proceedings

Differentiable rendering is frequently used in gradient descent-based inverse rendering pipelines to solve for scene parameters – such as reflectance or lighting properties – from target image inputs. Efficient computation of accurate, low variance ...
Image-space control variates for rendering

We explore the theory of integration with control variates in the context of rendering. Our goal is to optimally combine multiple estimators using their covariances. We focus on two applications, re-rendering and gradient-domain rendering, where we ...
Reconstructing Translucent Objects using Differentiable Rendering
SIGGRAPH '22: ACM SIGGRAPH 2022 Conference Proceedings

Inverse rendering is a powerful approach to modeling objects from photographs, and we extend previous techniques to handle translucent materials that exhibit subsurface scattering. Representing translucency using a heterogeneous bidirectional ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 42, Issue 4

August 2023

1912 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3609020

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2023

Published in TOG Volume 42, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
360
Total Downloads

Downloads (Last 12 months)236
Downloads (Last 6 weeks)22

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tran DJensen MSantafé-Gabarda PKällberg SFerrero AHannemose MFrisvad J(2024)Digitizing translucent object appearance by validating computed optical propertiesApplied Optics10.1364/AO.52197463:16(4317)Online publication date: 22-May-2024
https://doi.org/10.1364/AO.521974
Gu JBack JYoon SMoon B(2024)Target-Aware Image Denoising for Inverse Monte Carlo RenderingACM Transactions on Graphics10.1145/365818243:4(1-11)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658182
Fischer MRitschel T(2024)ZeroGrads: Learning Local Surrogates for Non-Differentiable GraphicsACM Transactions on Graphics10.1145/365817343:4(1-15)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658173
Fan ZShi PGuo MFu RGuo YGuo J(2024)Conditional Mixture Path Guiding for Differentiable RenderingACM Transactions on Graphics10.1145/365813343:4(1-11)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658133
Belhe YXu BBangaru SRamamoorthi RLi T(2024)Importance Sampling BRDF DerivativesACM Transactions on Graphics10.1145/364861143:3(1-21)Online publication date: 9-Apr-2024
https://dl.acm.org/doi/10.1145/3648611
Ling JYu RXu FDu CZhao S(2024)NeRF as a Non-Distant Environment Emitter in Physics-based Inverse RenderingACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657404(1-12)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657404
Li ZYang GZhao QDeng XGuibas LHariharan BWetzstein G(2024)Neural Control Variates with Automatic IntegrationACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657395(1-9)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657395
Li COno TUemori TMihara HGatto ANagahara HMoriuchi Y(2024)NeISF: Neural Incident Stokes Field for Geometry and Material Estimation2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02025(21434-21445)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02025
Dou YZheng ZJin QShi RLi YNi B(2024)Differentiable Micro-Mesh Construction2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00411(4294-4303)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00411
Salesin KKnobelspiesse KChowdhary JZhai PJarosz W(2024)Unifying radiative transfer models in computer graphics and remote sensing, Part I: A surveyJournal of Quantitative Spectroscopy and Radiative Transfer10.1016/j.jqsrt.2023.108847314(108847)Online publication date: Feb-2024
https://doi.org/10.1016/j.jqsrt.2023.108847
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents