research-article

Open access

Differentiable Rendering of Parametric Geometry

Authors:

Markus Worchel,

Marc AlexaAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 42, Issue 6

Article No.: 232, Pages 1 - 18

https://doi.org/10.1145/3618387

Published: 05 December 2023 Publication History

Abstract

We propose an efficient method for differentiable rendering of parametric surfaces and curves, which enables their use in inverse graphics problems. Our central observation is that a representative triangle mesh can be extracted from a continuous parametric object in a differentiable and efficient way. We derive differentiable meshing operators for surfaces and curves that provide varying levels of approximation granularity. With triangle mesh approximations, we can readily leverage existing machinery for differentiable mesh rendering to handle parametric geometry. Naively combining differentiable tessellation with inverse graphics settings lacks robustness and is prone to reaching undesirable local minima. To this end, we draw a connection between our setting and the optimization of triangle meshes in inverse graphics and present a set of optimization techniques, including regularizations and coarse-to-fine schemes. We show the viability and efficiency of our method in a set of image-based computer-aided design applications.

References

[1]

Salim S. Abi-Ezzi and Leon A. Shirman. 1991. Tessellation of Curved Surfaces under Highly Varying Transformations. In EG 1991-Technical Papers. Eurographics Association.

[2]

Sai Bangaru, Michael Gharbi, Tzu-Mao Li, Fujun Luan, Kalyan Sunkavalli, Milos Hasan, Sai Bi, Zexiang Xu, Gilbert Bernstein, and Fredo Durand. 2022. Differentiable Rendering of Neural SDFs through Reparameterization. In ACM SIGGRAPH Asia 2022 Conference Proceedings (Daegu, Republic of Korea) (SIGGRAPH Asia '22). Association for Computing Machinery, New York, NY, USA, Article 22, 9 pages.

Digital Library

[3]

PálBenkő, Ralph R Martin, and Tamás Várady. 2001. Algorithms for reverse engineering boundary representation models. Computer-Aided Design 33, 11 (2001), 839--851.

[4]

A. Bunge, M. Botsch, and M. Alexa. 2021. The Diamond Laplace for Polygonal and Polyhedral Meshes. Computer Graphics Forum 40, 5 (2021), 217--230. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14369

[5]

Astrid Bunge, Philipp Herholz, Misha Kazhdan, and Mario Botsch. 2020. Polygon Laplacian Made Simple. Computer Graphics Forum 39, 2 (2020), 303--313. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.13931

[6]

G. Cai, K. Yan, Z. Dong, I. Gkioulekas, and S. Zhao. 2022. Physics-Based Inverse Rendering using Combined Implicit and Explicit Geometries. Computer Graphics Forum 41, 4 (2022), 129--138. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14592

[7]

Dan Cascaval, Mira Shalah, Phillip Quinn, Ras Bodik, Maneesh Agrawala, and Adriana Schulz. 2022. Differentiable 3D CAD Programs for Bidirectional Editing. Computer Graphics Forum 41, 2 (2022), 309--323. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14476

[8]

E. Catmull and J. Clark. 1978. Recursively generated B-spline surfaces on arbitrary topological meshes. Computer-Aided Design 10, 6 (1978), 350--355.

[9]

Wenzheng Chen, Jun Gao, Huan Ling, Edward J. Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. 2019. Learning to Predict 3D Objects with an Interpolation-Based Differentiable Renderer. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, Article 862, 11 pages.

Digital Library

[10]

Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khalis, Or Litany, and Sanja Fidler. 2021. DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer. In Advances in Neural Information Processing Systems (NeurIPS).

[11]

Forrester Cole, Kyle Genova, Avneesh Sud, Daniel Vlasic, and Zhoutong Zhang. 2021. Differentiable Surface Rendering via Non-Differentiable Sampling. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV). 6068--6077.

[12]

Keenan Crane. 2023. A Simple Parametric Model of Plain-Knit Yarns. online. https://github.com/keenancrane/plain-knit-yarn

[13]

Martin de La Gorce, Nikos Paragios, and David J. Fleet. 2008. Model-based hand tracking with texture, shading and self-occlusions. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. 1--8.

[14]

Tony D. DeRose. 1990. Necessary and sufficient conditions for tangent plane continuity of Bézier surfaces. Computer Aided Geometric Design 7, 1 (1990), 165--179.

Digital Library

[15]

Anjana Deva Prasad, Aditya Balu, Harshil Shah, Soumik Sarkar, Chinmay Hegde, and Adarsh Krishnamurthy. 2022. NURBS-Diff: A Differentiable Programming Module for NURBS. Computer-Aided Design 146 (2022), 103199.

Digital Library

[16]

Matthias Eck and Hugues Hoppe. 1996. Automatic Reconstruction of B-Spline Surfaces of Arbitrary Topological Type. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). Association for Computing Machinery, New York, NY, USA, 325--334.

Digital Library

[17]

Gerald Farin and Dianne Hansford. 1999. Discrete Coons patches. Computer Aided Geometric Design 16, 7 (1999), 691--700.

Digital Library

[18]

Daniel Filip, Robert Magedson, and Robert Markot. 1986. Surface algorithms using bounds on derivatives. Computer Aided Geometric Design 3, 4 (1986), 295--311.

Digital Library

[19]

Manuel Finckh, Holger Dammertz, and Hendrik PA Lensch. 2010. Geometry construction from caustic images. In Computer Vision-ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5--11, 2010, Proceedings, Part V 11. Springer, 464--477.

[20]

Richard Franke. 1979. A critical comparison of some methods for interpolation of scattered data. Technical Report. NAVAL POSTGRADUATE SCHOOL MONTEREY CA.

[21]

Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 2022. Plenoxels: Radiance Fields without Neural Networks. In CVPR.

[22]

Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jianing Li, Michael Li, and Rev Lebaredian. 2022. Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research. https://github.com/NVIDIAGameWorks/kaolin.

[23]

Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. 2022. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. In Advances In Neural Information Processing Systems.

[24]

Yotam Gingold and Denis Zorin. 2008. Shading-Based Surface Editing. ACM Trans. Graph. 27, 3 (aug 2008), 1--9.

Digital Library

[25]

Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse Volume Rendering with Material Dictionaries. ACM Trans. Graph. 32, 6, Article 162 (nov 2013), 13 pages.

Digital Library

[26]

Nikola Guid, Črtomir Oblonšek, and Borut Žalik. 1995. Surface interrogation methods. Computers & graphics 19, 4 (1995), 557--574.

[27]

Michael Guthe, Aákos Balázs, and Reinhard Klein. 2005. GPU-Based Trimming and Tessellation of NURBS and T-Spline Surfaces. ACM Trans. Graph. 24, 3 (jul 2005), 1016--1023.

Digital Library

[28]

Hans Hagen, Stefanie Hahmann, Thomas Schreiber, Yasuo Nakajima, Burkhard Wordenweber, and Petra Hollemann-Grundstedt. 1992. Surface interrogation algorithms. IEEE Computer Graphics and Applications 12, 5 (1992), 53--60.

Digital Library

[29]

Mark A. Halstead, Brain A. Barsky, Stanley A. Klein, and Robert B. Mandell. 1996. Reconstructing Curved Surfaces from Specular Reflection Patterns Using Spline Surface Fitting of Normals. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). Association for Computing Machinery, New York, NY, USA, 335--342.

Digital Library

[30]

Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. 2022. Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising. arXiv:2206.03380 (2022).

[31]

Kai-Wen Hsiao, Jia-Bin Huang, and Hung-Kuo Chu. 2018. Multi-View Wire Art. ACM Trans. Graph. 37, 6, Article 242 (dec 2018), 11 pages.

Digital Library

[32]

T.J.R. Hughes, J.A. Cottrell, and Y. Bazilevs. 2005. Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement. Computer Methods in Applied Mechanics and Engineering 194, 39 (2005), 4135--4195.

[33]

Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '99). ACM Press/Addison-Wesley Publishing Co., USA, 409--416.

Digital Library

[34]

Wenzel Jakob, Sébastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. 2022. Mitsuba 3 renderer. https://mitsuba-renderer.org.

[35]

Yue Jiang, Dantong Ji, Zhizhong Han, and Matthias Zwicker. 2020. SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]

John K. Johnstone and James P. Williams. 1995. A rational model of the surface swept by a curve*. Computer Graphics Forum 14, 3 (1995), 77--88.

Digital Library

[37]

James T. Kajiya. 1982. Ray Tracing Parametric Patches. SIGGRAPH Comput. Graph. 16, 3 (jul 1982), 245--254.

Digital Library

[38]

Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Neural 3D Mesh Renderer. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3907--3916.

[39]

Leonid Keselman and Martial Hebert. 2022. Approximate Differentiable Rendering with Algebraic Surfaces. In Computer Vision - ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 596--614.

Digital Library

[40]

Thomas Kiser, Michael Eigensatz, Minh Man Nguyen, Philippe Bompas, and Mark Pauly. 2013. Architectural caustics---controlling light with geometry. In Advances in architectural geometry 2012. Springer, 91--106.

[41]

Reinhold Klass. 1980. Correction of local surface irregularities using reflection lines. Computer-Aided Design 12, 2 (1980), 73--77.

[42]

Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A Big CAD Model Dataset For Geometric Deep Learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]

Subodh Kumar and Dinesh Manocha. 1995. Efficient rendering of trimmed nurbs surfaces. Computer-Aided Design 27, 7 (1995), 509--521. Display and visualisation.

[44]

Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular Primitives for High-Performance Differentiable Rendering. ACM Trans. Graph. 39, 6, Article 194 (nov 2020), 14 pages.

Digital Library

[45]

Christoph Lassner and Michael Zollhöfer. 2021. Pulsar: Efficient Sphere-based Neural Rendering. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1440--1449.

[46]

Manfred Lau, Greg Saul, Jun Mitani, and Takeo Igarashi. 2010. Modeling-in-context: User design of complementary objects with a single photo. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium. 17--24.

[47]

Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J. Mitra. 2020b. Sketch2CAD: Sequential CAD Modeling by Sketching in Context. ACM Trans. Graph. 39, 6, Article 164 (nov 2020), 14 pages.

Digital Library

[48]

Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. 37, 6, Article 222 (dec 2018), 11 pages.

Digital Library

[49]

Tzu-Mao Li, Michal Lukáč, Michaël Gharbi, and Jonathan Ragan-Kelley. 2020a. Differentiable Vector Graphics Rasterization for Editing and Learning. ACM Trans. Graph. 39, 6, Article 193 (nov 2020), 15 pages.

Digital Library

[50]

Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H Taylor, Mathias Unberath, Ming-Yu Liu, and Chen-Hsuan Lin. 2023. Neuralangelo: High-Fidelity Neural Surface Reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]

Lingjie Liu, Duygu Ceylan, Cheng Lin, Wenping Wang, and Niloy J. Mitra. 2017. Image-based Reconstruction of Wire Art. ACM SIGGRAPH 2017 (2017).

[52]

Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. 2019. Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning. The IEEE International Conference on Computer Vision (ICCV) (Oct 2019).

[53]

Shaohui Liu, Yinda Zhang, Songyou Peng, Boxin Shi, Marc Pollefeys, and Zhaopeng Cui. 2020. Dist: Rendering deep implicit signed distance function with differentiable sphere tracing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019--2028.

[54]

Joachim. Loos, Günther. Greiner, and Hans-Peter Seidel. 1999. Modeling of surfaces with fair reflection line pattern. In Proceedings Shape Modeling International '99. International Conference on Shape Modeling and Applications. 256--263.

[55]

Matthew M. Loper and Michael J. Black. 2014. OpenDR: An Approximate Differentiable Renderer. In Computer Vision - ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer International Publishing, Cham, 154--169.

[56]

Fujun Luan, Shuang Zhao, Kavita Bala, and Zhao Dong. 2021. Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering. Computer Graphics Forum (2021).

[57]

Andrew Luo, Tianqin Li, Wen-Hao Zhang, and Tai Sing Lee. 2021. SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 16238--16248.

[58]

Linjie Luo, Hao Li, Sylvain Paris, Thibaut Weise, Mark Pauly, and Szymon Rusinkiewicz. 2012. Multi-view hair capture using orientation fields. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. 1490--1497.

[59]

Linjie Luo, Hao Li, and Szymon Rusinkiewicz. 2013. Structure-Aware Hair Capture. ACM Trans. Graph. 32, 4, Article 76 (jul 2013), 12 pages.

Digital Library

[60]

H. Martinsson, F. Gaspard, A. Bartoli, and J.-M. Lavest. 2007a. Adaptive Evolution of 3D Curves for Quality Control. In 2007 IEEE International Symposium on Intelligent Signal Processing. 1--6.

[61]

Hanna Martinsson, Francois Gaspard, Adrien Bartoli, and Jean-Marc Lavest. 2007b. Reconstruction of 3D Curves for Quality Control. In Image Analysis, Bjarne Kjær Ersbøll and Kim Steenstrup Pedersen (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 760--769.

[62]

Ishit Mehta, Manmohan Chandraker, and Ravi Ramamoorthi. 2022. A Level Set Theory for Neural Implicit Evolution Under Explicit Flows. In Computer Vision - ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part II (Tel Aviv, Israel). Springer-Verlag, Berlin, Heidelberg, 711--729.

Digital Library

[63]

Jocelyn Meyron, Quentin Mérigot, and Boris Thibert. 2018. Light in Power: A General and Parameter-Free Algorithm for Caustic Design. ACM Trans. Graph. 37, 6, Article 224 (dec 2018), 13 pages.

Digital Library

[64]

Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.

[65]

MJ Milroy, C Bradley, GW Vickers, and DJ Weir. 1995. G1 continuity of B-spline surface patches in reverse engineering. Computer-Aided Design 27, 6 (1995), 471--478.

[66]

Juan Monterde. 2004. Bézier surfaces of minimal area: The Dirichlet approach. Computer Aided Geometric Design 21, 2 (2004), 117--136.

Digital Library

[67]

Jan U. Müller, Michael Weinmann, and Reinhard Klein. 2022. Unbiased Gradient Estimation for Differentiable Surface Splatting via Poisson Sampling. In Computer Vision - ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 281--299.

Digital Library

[68]

Jacob Munkberg, Wenzheng Chen, Jon Hasselgren, Alex Evans, Tianchang Shen, Thomas Müller, Jun Gao, and Sanja Fidler. 2022. Extracting Triangular 3D Models, Materials, and Lighting From Images. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8270--8280.

[69]

Giljoo Nam, Chenglei Wu, Min H. Kim, and Yaser Sheikh. 2019. Strand-Accurate Multi-View Hair Capture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[70]

Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: Designing Freeform Surfaces with 3D Curves. ACM Trans. Graph. 26, 3 (jul 2007), 41--es.

Digital Library

[71]

Andrew Nealen, Olga Sorkine, Marc Alexa, and Daniel Cohen-Or. 2005. A Sketch-Based Interface for Detail-Preserving Mesh Editing. In ACM SIGGRAPH 2005 Papers (Los Angeles, California) (SIGGRAPH '05). Association for Computing Machinery, New York, NY, USA, 1142--1147.

Digital Library

[72]

Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large Steps in Inverse Rendering of Geometry. ACM Trans. Graph. 40, 6, Article 248 (dec 2021), 13 pages.

Digital Library

[73]

Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. 2020. Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[74]

Merlin Nimier-David, Thomas Müller, Alexander Keller, and Wenzel Jakob. 2022. Unbiased Inverse Volume Rendering with Differential Trackers. ACM Trans. Graph. 41, 4, Article 44 (jul 2022), 20 pages.

Digital Library

[75]

Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A Retargetable Forward and Inverse Renderer. ACM Trans. Graph. 38, 6, Article 203

Digital Library

[76]

(nov 2019), 17 pages.

Digital Library

[77]

Michael Oechsle, Songyou Peng, and Andreas Geiger. 2021. UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 5589--5599.

[78]

Marios Papas, Wojciech Jarosz, Wenzel Jakob, Szymon Rusinkiewicz, Wojciech Matusik, and Tim Weyrich. 2011. Goal-based caustics. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 503--511.

[79]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, Article 721, 12 pages.

Digital Library

[80]

Dario Pavllo, Jonas Kohler, Thomas Hofmann, and Aurelien Lucchi. 2021. Learning Generative Models of Textured 3D Meshes From Real-World Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 13879--13889.

[81]

Dario Pavllo, Graham Spinks, Thomas Hofmann, Marie-Francine Moens, and Aurelien Lucchi. 2020. Convolutional generation of textured 3d meshes. Advances in Neural Information Processing Systems 33 (2020), 870--882.

[82]

Davide Pellis, Martin Kilian, Felix Dellinger, Johannes Wallner, and Helmut Pottmann. 2019. Visual smoothness of polyhedral surfaces. ACM Transactions on Graphics (TOG) 38, 4 (2019), 1--11.

Digital Library

[83]

Felix Petersen, Bastian Goldluecke, Christian Borgelt, and Oliver Deussen. 2022. GenDR: A Generalized Differentiable Renderer. In IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR).

[84]

Bui Tuong Phong. 1998. Illumination for Computer Generated Pictures. Association for Computing Machinery, New York, NY, USA, 95--101.

Digital Library

[85]

Thomas Poeschl. 1984. Detecting surface irregularities using isophotes. Computer Aided Geometric Design 1, 2 (1984), 163--168.

Digital Library

[86]

Helmut Pottmann, Michael Eigensatz, Amir Vaxman, and Johannes Wallner. 2015. Architectural geometry. Computers and Graphics 47 (2015), 145--164.

Digital Library

[87]

Marie-Julie Rakotosaona, Noam Aigerman, Niloy J. Mitra, Maks Ovsjanikov, and Paul Guerrero. 2021. Differentiable Surface Triangulation. ACM Trans. Graph. 40, 6, Article 267 (dec 2021), 13 pages.

Digital Library

[88]

Ravi Ramamoorthi and Pat Hanrahan. 2001. An Efficient Representation for Irradiance Environment Maps. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01). Association for Computing Machinery, New York, NY, USA, 497--500.

Digital Library

[89]

Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. 2020. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501 (2020).

[90]

Daxuan Ren, Jianmin Zheng, Jianfei Cai, Jiatong Li, and Junzhe Zhang. 2022. ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsing. In Computer Vision - ECCV 2022, Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer Nature Switzerland, Cham, 482--498.

Digital Library

[91]

Tobias Ritschel, Makoto Okabe, Thorsten Thormählen, and Hans-Peter Seidel. 2009. Interactive Reflection Editing. ACM Trans. Graph. 28, 5 (dec 2009), 1--7.

Digital Library

[92]

Alyn Rockwood, Kurt Heaton, and Tom Davis. 1989. Real-Time Rendering of Trimmed Surfaces. SIGGRAPH Comput. Graph. 23, 3 (jul 1989), 107--116.

Digital Library

[93]

Radu Alexandru Rosu, Shunsuke Saito, Ziyan Wang, Chenglei Wu, Sven Behnke, and Giljoo Nam. 2022. Neural Strands: Learning Hair Geometry and Appearance from Multi-View Images. ECCV (2022).

[94]

Darius Rückert, Linus Franke, and Marc Stamminger. 2022. ADOP: Approximate Differentiable One-Pixel Point Rendering. ACM Trans. Graph. 41, 4, Article 99 (jul 2022), 14 pages.

Digital Library

[95]

Yuliy Schwartzburg, Romain Testuz, Andrea Tagliasacchi, and Mark Pauly. 2014. High-Contrast Computational Caustic Design. ACM Trans. Graph. 33, 4, Article 74 (jul 2014), 11 pages.

Digital Library

[96]

Michael Schwarz and Marc Stamminger. 2009. Fast GPU-based adaptive tessellation with CUDA. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 365--374.

[97]

Yusuke Takimoto, Hiroyuki Sato, Hikari Takehara, Keishiro Uragaki, Takehiro Tawara, Xiao Liang, Kentaro Oku, Wataru Kishimoto, and Bo Zheng. 2022. Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization. Computer Graphics Forum 41, 2 (2022), 13--27. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14455

[98]

Holger Theisel and Gerald Farin. 1997. The curvature of characteristic curves on surfaces. IEEE Computer Graphics and Applications 17, 6 (1997), 88--96.

Digital Library

[99]

Ann Torrence. 2006. Martin Newell's Original Teapot. In ACM SIGGRAPH 2006 Teapot Copyright Restrictions Prevent ACM from Providing the Full Text for the Teapot Exhibits (Boston, Massachusetts) (SIGGRAPH '06). Association for Computing Machinery, New York, NY, USA, 29--es.

Digital Library

[100]

Elif Tosun, Yotam I Gingold, Jason Reisman, and Denis Zorin. 2007. Shape optimization using reflection lines. In Symposium on Geometry Processing. Citeseer, 193--202.

[101]

Mikaela Angelina Uy, Yen-Yu Chang, Minhyuk Sung, Purvi Goel, Joseph G. Lambourne, Tolga Birdal, and Leonidas J. Guibas. 2022. Point2Cyl: Reverse Engineering 3D Objects From Point Clouds to Extrusion Cylinders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11850--11860.

[102]

Tamas Varady, Ralph R Martin, and Jordan Cox. 1997. Reverse engineering of geometric models---an introduction. Computer-aided design 29, 4 (1997), 255--268.

[103]

Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2022. Differentiable Signed Distance Function Rendering. ACM Trans. Graph. 41, 4, Article 125 (jul 2022), 18 pages.

Digital Library

[104]

Vasilis Vlassopoulos. 1990. Adaptive polygonization of parametric surfaces. The Visual Computer 6 (1990), 291--298.

Digital Library

[105]

Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. 2021. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. NeurIPS (2021).

[106]

Wenping Wang and Barry Joe. 1997. Robust computation of the rotation minimizing frame for sweep surface modeling. Computer-Aided Design 29, 5 (1997), 379--391.

[107]

Wenping Wang, Bert Jüttler, Dayue Zheng, and Yang Liu. 2008. Computation of Rotation Minimizing Frames. ACM Trans. Graph. 27, 1, Article 2 (mar 2008), 18 pages.

Digital Library

[108]

Markus Worchel, Rodrigo Diaz, Weiwen Hu, Oliver Schreer, Ingo Feldmann, and Peter Eisert. 2022. Multi-View Mesh Reconstruction With Neural Deferred Shading. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6187--6197.

[109]

Kang Wu, Renjie Chen, Xiao-Ming Fu, and Ligang Liu. 2022. Computational Mirror Cup and Saucer Art. ACM Trans. Graph. 41, 5, Article 174 (jul 2022), 15 pages.

Digital Library

[110]

Ming Yang, Hongyang Chao, Chi Zhang, Jun Guo, Lu Yuan, and Jian Sun. 2015. Effective clipart image vectorization through direct optimization of bezigons. IEEE transactions on visualization and computer graphics 22, 2 (2015), 1063--1075.

[111]

Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. 2021. Volume rendering of neural implicit surfaces. In Thirty-Fifth Conference on Neural Information Processing Systems.

[112]

Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. 2020. Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. Advances in Neural Information Processing Systems 33 (2020).

[113]

Wang Yifan, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine-Hornung. 2019. Differentiable Surface Splatting for Point-Based Geometry Processing. ACM Trans. Graph. 38, 6, Article 230 (nov 2019), 14 pages.

Digital Library

[114]

Xiaohui Zeng, Arash Vahdat, Francis Williams, Zan Gojcic, Or Litany, Sanja Fidler, and Karsten Kreis. 2022. LION: Latent Point Diffusion Models for 3D Shape Generation. In Advances in Neural Information Processing Systems (NeurIPS).

[115]

Cheng Zhang, Bailey Miller, Kai Yan, Ioannis Gkioulekas, and Shuang Zhao. 2020b. Path-Space Differentiable Rendering. ACM Trans. Graph. 39, 4, Article 143 (aug 2020), 19 pages.

Digital Library

[116]

Kai Zhang, Fujun Luan, Zhengqi Li, and Noah Snavely. 2022. IRON: Inverse Rendering by Optimizing Neural SDFs and Materials From Photometric Images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5565--5574.

[117]

Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, and Sanja Fidler. 2020a. Image gans meet differentiable rendering for inverse graphics and interpretable 3d neural rendering. arXiv preprint arXiv:2010.09125 (2020).

[118]

Johannes Zimmermann, Andrew Nealen, and Marc Alexa. 2008. Sketching contours. Computers & Graphics 32, 5 (2008), 486--499.

Digital Library

Cited By

Tojo KShamir ABickel BUmetani N(2024)Fabricable 3D Wire ArtACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657453(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657453
Vaitkus MSalvi PVárady T(2024)Interior control structure for Generalized Bézier patches over curved domainsComputers & Graphics10.1016/j.cag.2024.103952121(103952)Online publication date: Jun-2024
https://doi.org/10.1016/j.cag.2024.103952

Index Terms

Differentiable Rendering of Parametric Geometry
1. Computing methodologies
  1. Computer graphics
    1. Rendering
    2. Shape modeling

Recommendations

Projective Sampling for Differentiable Rendering of Geometry

Discontinuous visibility changes at object boundaries remain a persistent source of difficulty in the area of differentiable rendering. Left untreated, they bias computed gradients so severely that even basic optimization tasks fail.

Prior path-space ...
Reparameterizing discontinuous integrands for differentiable rendering

Differentiable rendering has recently opened the door to a number of challenging inverse problems involving photorealistic images, such as computational material design and scattering-aware reconstruction of geometry and materials from photographs. ...
Reconstructing Translucent Objects using Differentiable Rendering
SIGGRAPH '22: ACM SIGGRAPH 2022 Conference Proceedings

Inverse rendering is a powerful approach to modeling objects from photographs, and we extend previous techniques to handle translucent materials that exhibit subsurface scattering. Representing translucency using a heterogeneous bidirectional ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 42, Issue 6

December 2023

1565 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3632123

Issue’s Table of Contents

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2023

Published in TOG Volume 42, Issue 6

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,462
Total Downloads

Downloads (Last 12 months)1,462
Downloads (Last 6 weeks)232

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tojo KShamir ABickel BUmetani N(2024)Fabricable 3D Wire ArtACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657453(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657453
Vaitkus MSalvi PVárady T(2024)Interior control structure for Generalized Bézier patches over curved domainsComputers & Graphics10.1016/j.cag.2024.103952121(103952)Online publication date: Jun-2024
https://doi.org/10.1016/j.cag.2024.103952

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents