Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Fusion 360 gallery: a dataset and environment for programmatic CAD construction from human design sequences

Published: 19 July 2021 Publication History

Abstract

Parametric computer-aided design (CAD) is a standard paradigm used to design manufactured objects, where a 3D shape is represented as a program supported by the CAD software. Despite the pervasiveness of parametric CAD and a growing interest from the research community, currently there does not exist a dataset of realistic CAD models in a concise programmatic form. In this paper we present the Fusion 360 Gallery, consisting of a simple language with just the sketch and extrude modeling operations, and a dataset of 8,625 human design sequences expressed in this language. We also present an interactive environment called the Fusion 360 Gym, which exposes the sequential construction of a CAD program as a Markov decision process, making it amendable to machine learning approaches. As a use case for our dataset and environment, we define the CAD reconstruction task of recovering a CAD program from a target geometry. We report results of applying state-of-the-art methods of program synthesis with neurally guided search on this task.

Supplementary Material

VTT File (3450626.3459818.vtt)
MP4 File (a54-willis.mp4)
MP4 File (3450626.3459818.mp4)
Presentation.

References

[1]
Silvia Ansaldi, Leila De Floriani, and Bianca Falcidieno. 1985. Geometric modeling of solid objects by using a face adjacency graph representation. ACM SIGGRAPH Computer Graphics 19, 3 (1985), 131--139.
[2]
Autodesk. 2012. Inventor Feature Recognition. https://apps.autodesk.com/INVNTOR/en/Detail/Index?id=9172877436288348979
[3]
Autodesk. 2014. Fusion 360 API. http://help.autodesk.com/view/fusion360/ENU/?guid=GUID-7B5A90C8-E94C-48DA-B16B-430729B734DC
[4]
Autodesk. 2015. Autodesk Online Gallery. https://gallery.autodesk.com
[5]
Pal Benko, Geza Kos, Pál Benkö, Laszlo Andor, Géza Kós, Tamas Varady, László Andor, and Ralph Martin. 2002. Constrained Fitting in Reverse Engineering.
[6]
Suzanne Fox Buchele. 2000. Three-dimensional binary space partitioning tree and constructive solid geometry tree construction from algebraic boundary representations. (2000).
[7]
Suzanne F Buchele and Richard H Crawford. 2003. Three-dimensional halfspace constructive solid geometry tree construction from implicit boundary representations. In Proceedings of the eighth ACM symposium on Solid modeling and applications. 135--144.
[8]
Suzanne F Buchele and Angela C Roles. 2001. Binary space partitioning tree and constructive solid geometry representations for objects bounded by curved surfaces. In CCCG. Citeseer, 49--52.
[9]
Francesco Buonamici, Monica Carfagni, Rocco Furferi, Lapo Governi, Alessandro Lapini, and Yary Volpe. 2018. Reverse engineering modeling methods and tools: a survey. Computer-Aided Design and Applications 15, 3 (2018), 443--464. arXiv:https://doi.org/10.1080/16864360.2017.1397894
[10]
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015).
[11]
Zhiqin Chen, Andrea Tagliasacchi, and Hao Zhang. 2020. Bsp-net: Generating compact meshes via binary space partitioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 45--54.
[12]
Dassault. 2019. Solidworks FeatureWorks. https://help.solidworks.com/2019/english/SolidWorks/fworks/c_Overview_of_FeatureWorks.htm
[13]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[14]
Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. 2017. Robustfill: Neural program learning under noisy i/o. arXiv preprint arXiv:1703.07469 (2017).
[15]
Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. 2018. Inversecsg: Automatic conversion of 3d models to csg trees. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1--16.
[16]
Mathias Eitz, James Hays, and Marc Alexa. 2012. How do humans sketch objects? ACM Transactions on graphics (TOG) 31, 4 (2012), 1--10.
[17]
Kevin Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. 2019. Write, execute, assess: Program synthesis with a repl. In Advances in Neural Information Processing Systems. 9169--9178.
[18]
Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Josh Tenenbaum. 2018. Learning to Infer Graphics Programs from Hand-Drawn Images. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc., 6059--6068. https://proceedings.neurips.cc/paper/2018/file/6788076842014c83cedadbe6b0ba0314-Paper.pdf
[19]
Pierre-Alain Fayolle and Alexander Pasko. 2016. An evolutionary approach to the extraction of object construction trees from 3D point clouds. Computer-Aided Designs 74 (2016), 1--17.
[20]
Markus Friedrich, Pierre-Alain Fayolle, Thomas Gabor, and Claudia Linnhoff-Popien. 2019. Optimizing evolutionary CSG tree extraction. In Proceedings of the Genetic and Evolutionary Computation Conference. 1183--1191.
[21]
Lin Gao, Jie Yang, Tong Wu, Yu-Jie Yuan, Hongbo Fu, Yu-Kun Lai, and Hao Zhang. 2019. SDM-NET: Deep generative network for structured deformable mesh. ACM Transactions on Graphics (TOG) 38, 6 (2019), 1--15.
[22]
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In International Conference on Machine Learning. PMLR, 1263--1272.
[23]
Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia Pont, Fredo Durand, and Adrien Bousseau. 2019. Opensketch: A richly-annotated dataset of product design sketches. ACM Transactions on Graphics (TOG) 38, 6 (2019), 232.
[24]
Karim Hamza and Kazuhiro Saitou. 2004. Optimization of constructive solid geometry via a tree-based multi-objective genetic algorithm. In Genetic and Evolutionary Computation Conference. Springer, 981--992.
[25]
Pradeep Kumar Jayaraman, Aditya Sanghi, Joseph Lambourne, Thomas Davies, Hooman Shayani, and Nigel Morris. 2020. UV-Net: Learning from Curve-Networks and Solids. arXiv preprint arXiv:2006.10211 (2020).
[26]
R. Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy Mitra, and Daniel Ritchie. 2020. ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis. ACM Transactions on Graphics (TOG), Siggraph Asia 2020 39, 6 (2020), Article 234.
[27]
Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-guided deductive search for real-time program synthesis from examples. arXiv preprint arXiv:1804.01186 (2018).
[28]
Kacper Kania, Maciej Zięba, and Tomasz Kajdanowicz. 2020. UCSG-Net-Unsupervised Discovering of Constructive Solid Geometry Tree. arXiv preprint arXiv:2006.09102 (2020).
[29]
Sangpil Kim, Hyung-gun Chi, Xiao Hu, Qixing Huang, and Karthik Ramani. 2020. A Large-scale Annotated Mechanical Components Benchmark for Classification and Retrieval Tasks with Deep Neural Networks. In Proceedings of 16th European Conference on Computer Vision (ECCV).
[30]
Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, and Richard Zemel. 2018. Neural relational inference for interacting systems. In International Conference on Machine Learning. PMLR, 2688--2697.
[31]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
[32]
Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A big CAD model dataset for geometric deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9601--9611.
[33]
Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J. Mitra. 2020b. Sketch2CAD: Sequential CAD Modeling by Sketching in Context. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia 2020) 39, 6 (2020), 164:1--164:14.
[34]
Jun Li, Chengjie Niu, and Kai Xu. 2020a. Learning part generation and assembly for structure-aware shape synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11362--11369.
[35]
Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, Will Hamilton, David K Duvenaud, Raquel Urtasun, and Richard Zemel. 2019. Efficient graph generation with graph recurrent attention networks. In Advances in Neural Information Processing Systems. 4255--4265.
[36]
Cheng Lin, Tingxiang Fan, Wenping Wang, and Matthias Nießner. 2020. Modeling 3D Shapes by Reinforcement Learning. ECCV (2020).
[37]
Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy J. Mitra, and Leonidas J. Guibas. 2019a. StructureNet: Hierarchical Graph Networks for 3D Shape Generation. ACM Trans. Graph. 38, 6, Article 242 (Nov. 2019), 19 pages.
[38]
Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su. 2019b. Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 909--918.
[39]
Chandrakana Nandi, Anat Caspi, Dan Grossman, and Zachary Tatlock. 2017. Programming language tools and techniques for 3D printing. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
[40]
Chandrakana Nandi, James R Wilcox, Pavel Panchekha, Taylor Blau, Dan Grossman, and Zachary Tatlock. 2018. Functional programming for compiling and decompiling computer-aided design. Proceedings of the ACM on Programming Languages 2, ICFP (2018), 1--31.
[41]
Chandrakana Nandi, Max Willsey, Adam Anderson, James R. Wilcox, Eva Darulova, Dan Grossman, and Zachary Tatlock. 2020. Synthesizing Structured CAD Models with Equality Saturation and Inverse Transformations. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation. 31--44.
[42]
Charlie Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. 2020. Polygen: An autoregressive generative model of 3d meshes. In International Conference on Machine Learning. PMLR, 7220--7229.
[43]
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. 2016. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--12.
[44]
Nadav Schor, Oren Katzir, Hao Zhang, and Daniel Cohen-Or. 2019. CompoNet: Learning to generate the unseen by part synthesis and composition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8759--8768.
[45]
Adriana Schulz, Ariel Shamir, David I. W. Levin, Pitchaya Sitthi-Amorn, and Wojciech Matusik. 2014. Design and Fabrication by Example. ACM Transactions on Graphics (Proceedings SIGGRAPH 2014) 33, 4 (2014).
[46]
Ari Seff, Yaniv Ovadia, Wenda Zhou, and Ryan P. Adams. 2020. SketchGraphs: A Large-Scale Dataset for Modeling Relational Geometry in Computer-Aided Design. In ICML 2020 Workshop on Object-Oriented Learning.
[47]
Jami J Shah, David Anderson, Yong Se Kim, and Sanjay Joshi. 2001. A discourse on geometric feature recognition from CAD models. J. Comput. Inf. Sci. Eng. 1, 1 (2001), 41--51.
[48]
Vadim Shapiro and Donald L Vossler. 1993. Separation for boundary to CSG conversion. ACM Transactions on Graphics (TOG) 12, 1 (1993), 35--55.
[49]
Gopal Sharma, Rishabh Goyal, Difan Liu, Evangelos Kalogerakis, and Subhransu Maji. 2017. CSGNet: Neural Shape Parser for Constructive Solid Geometry. CoRR abs/1712.08290 (2017). arXiv preprint arXiv:1712.08290 (2017).
[50]
Binil Starly. 2020. FabWave - 3D Part Repository. https://www.dimelab.org/fabwave
[51]
O. Stava, S. Pirk, J. Kratt, B. Chen, R. Mundefinedch, O. Deussen, and B. Benes. 2014. Inverse Procedural Modelling of Trees. Comput. Graph. Forum 33, 6 (Sept. 2014), 118--131.
[52]
Minhyuk Sung, Hao Su, Vladimir G Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-supervised component suggestions for 3D modeling. ACM Transactions on Graphics (TOG) 36, 6 (2017), 1--12.
[53]
Jerry O. Talton, Yu Lou, Steve Lesser, Jared Duke, Radomír Měch, and Vladlen Koltun. 2011. Metropolis Procedural Modeling. ACM Trans. Graph. 30, 2, Article 11 (April 2011), 14 pages.
[54]
Yunhao Tang, Shipra Agrawal, and Yuri Faenza. 2019. Reinforcement learning for integer programming: Learning to cut. arXiv preprint arXiv:1906.04859 (2019).
[55]
Yonglong Tian, Andrew Luo, Xingyuan Sun, Kevin Ellis, William T. Freeman, Joshua B. Tenenbaum, and Jiajun Wu. 2019. Learning to Infer and Execute 3D Shape Programs. In International Conference on Learning Representations.
[56]
Carlos A. Vanegas, Ignacio Garcia-Dorado, Daniel G. Aliaga, Bedrich Benes, and Paul Waddell. 2012. Inverse Design of Urban Procedural Models. ACM Trans. Graph. 31, 6, Article 168 (Nov. 2012), 11 pages.
[57]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc., 5998--6008. https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
[58]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[59]
K.J. Weiler. 1986. Topological structures for geometric modeling. University Microfilms.
[60]
Daniel Weiss. 2009. Geometry-based structural optimization on CAD specification trees. Ph.D. Dissertation. ETH Zurich.
[61]
Rundi Wu, Yixin Zhuang, Kai Xu, Hao Zhang, and Baoquan Chen. 2020. Pq-net: A generative part seq2seq network for 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 829--838.
[62]
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912--1920.
[63]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2018. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
[64]
Zhibo Zhang, Prakhar Jaiswal, and Rahul Rai. 2018. FeatureNet: machining feature recognition based on 3D convolution neural network. Computer-Aided Design 101 (2018), 12--22.
[65]
Qingnan Zhou and Alec Jacobson. 2016. Thingi10K: A Dataset of 10,000 3D-Printing Models. arXiv preprint arXiv:1605.04797 (2016).
[66]
Chuhang Zou, Ersin Yumer, Jimei Yang, Duygu Ceylan, and Derek Hoiem. 2017. 3d-prnn: Generating shape primitives with recurrent neural networks. In Proceedings of the IEEE International Conference on Computer Vision. 900--909.

Cited By

View all
  • (2025)A history-based parametric CAD sketch dataset with advanced engineering commandsComputer-Aided Design10.1016/j.cad.2025.103848182(103848)Online publication date: May-2025
  • (2025)eCAD-Net: Editable Parametric CAD Models Reconstruction from Dumb B-Rep Models Using Deep Neural NetworksComputer-Aided Design10.1016/j.cad.2024.103806178(103806)Online publication date: Jan-2025
  • (2025)A survey of emerging applications of large language models for problems in mechanics, product design, and manufacturingAdvanced Engineering Informatics10.1016/j.aei.2024.10306664(103066)Online publication date: Mar-2025
  • Show More Cited By

Index Terms

  1. Fusion 360 gallery: a dataset and environment for programmatic CAD construction from human design sequences

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Graphics
    ACM Transactions on Graphics  Volume 40, Issue 4
    August 2021
    2170 pages
    ISSN:0730-0301
    EISSN:1557-7368
    DOI:10.1145/3450626
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 July 2021
    Published in TOG Volume 40, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. CAD
    2. computer aided design
    3. construction
    4. dataset
    5. geometry synthesis
    6. reconstruction

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)303
    • Downloads (Last 6 weeks)22
    Reflects downloads up to 08 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)A history-based parametric CAD sketch dataset with advanced engineering commandsComputer-Aided Design10.1016/j.cad.2025.103848182(103848)Online publication date: May-2025
    • (2025)eCAD-Net: Editable Parametric CAD Models Reconstruction from Dumb B-Rep Models Using Deep Neural NetworksComputer-Aided Design10.1016/j.cad.2024.103806178(103806)Online publication date: Jan-2025
    • (2025)A survey of emerging applications of large language models for problems in mechanics, product design, and manufacturingAdvanced Engineering Informatics10.1016/j.aei.2024.10306664(103066)Online publication date: Mar-2025
    • (2025)CF-CAD: A Contrastive Fusion Network For 3D Computer-Aided Design Generative ModelingDatabase Systems for Advanced Applications10.1007/978-981-97-5555-4_31(435-450)Online publication date: 12-Jan-2025
    • (2024)A Hypothesis Testing of Archwire Rounding for the Efficacy of Torque Springs in Orthodontics: A Finite Element StudyCureus10.7759/cureus.57292Online publication date: 30-Mar-2024
    • (2024)A parametric and feature-based CAD dataset to support human-computer interaction for advanced 3D shape learningIntegrated Computer-Aided Engineering10.3233/ICA-24074432:1(73-94)Online publication date: 18-Oct-2024
    • (2024)CatalogBank: A Structured and Interoperable Catalog Dataset with a Semi-Automatic Annotation Tool (DocumentLabeler) for Engineering System DesignProceedings of the ACM Symposium on Document Engineering 202410.1145/3685650.3685665(1-9)Online publication date: 20-Aug-2024
    • (2024)NeurCADRecon: Neural Representation for Reconstructing CAD Surfaces by Enforcing Zero Gaussian CurvatureACM Transactions on Graphics10.1145/365817143:4(1-17)Online publication date: 19-Jul-2024
    • (2024)BrepGen: A B-rep Generative Diffusion Model with Structured Latent GeometryACM Transactions on Graphics10.1145/365812943:4(1-14)Online publication date: 19-Jul-2024
    • (2024)Communicating Design Intent Using Drawing and TextProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3664261(512-519)Online publication date: 23-Jun-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media