Sketch2Stress: Sketching With Structural Stress Awareness
Authors: 
Deng Yu, Chufeng Xiao, Manfred Lau, 
Hongbo FuAuthors Info & Claims
Deng Yu, Chufeng Xiao, Manfred Lau, Hongbo FuAuthors Info & ClaimsPages 6851 - 6865
Abstract
In the process of product design and digital fabrication, the structural analysis of a designed prototype is a fundamental and essential step. However, such a step is usually invisible or inaccessible to designers at the early sketching phase. This limits the user's ability to consider a shape's physical properties and structural soundness. To bridge this gap, we introduce a novel approach <italic>Sketch2Stress</italic> that allows users to perform structural analysis of desired objects at the sketching stage. This method takes as input a 2D freehand sketch and one or multiple locations of user-assigned external forces. With the specially-designed two-branch generative-adversarial framework, it automatically predicts a normal map and a corresponding structural stress map distributed over the user-sketched underlying object. In this way, our method empowers designers to easily examine the stress sustained everywhere and identify potential problematic regions of their sketched object. Furthermore, combined with the predicted normal map, users are able to conduct a region-wise structural analysis efficiently by aggregating the stress effects of multiple forces in the same direction. Finally, we demonstrate the effectiveness and practicality of our system with extensive experiments and user studies.
References
[1]
G. Johnson, M. Gross, E. Y.-L. Do, and J. Hong, “Sketch it, make it: Sketching precise drawings for laser cutting,” in Proc. Extended Abstr. Hum. Factors Comput. Syst., 2012, pp. 1079–1082.
[2]
O. Stava, J. Vanek, B. Benes, N. Carr, and R. Měch, “Stress relief: Improving structural strength of 3D printable objects,” ACM Trans. Graph., vol. 31, no. 4, pp. 1–11, 2012.
[3]
Q. Zhou, J. Panetta, and D. Zorin, “Worst-case structural analysis,” ACM Trans. Graph., vol. 32, no. 4, pp. 137–1, 2013.
[4]
W. Wang et al., “Cost-effective printing of 3D objects with skin-frame structures,” ACM Trans. Graph., vol. 32, no. 6, pp. 1–10, 2013.
[5]
R. Prévost, E. Whiting, S. Lefebvre, and O. Sorkine-Hornung, “Make it stand: Balancing shapes for 3D fabrication,” ACM Trans. Graph., vol. 32, no. 4, pp. 1–10, 2013.
[6]
E. Ulu, J. Mccann, and L. B. Kara, “Lightweight structure design under force location uncertainty,” ACM Trans. Graph., vol. 36, no. 4, pp. 1–13, 2017.
[7]
J. Panetta, A. Rahimian, and D. Zorin, “Worst-case stress relief for microstructures,” ACM Trans. Graph., vol. 36, no. 4, pp. 1–16, 2017.
[8]
L. Lu et al., “Build-to-last: Strength to weight 3D printed objects,” ACM Trans. Graph., vol. 33, no. 4, pp. 1–10, 2014.
[9]
T. Langlois, A. Shamir, D. Dror, W. Matusik, and D. I. Levin, “Stochastic structural analysis for context-aware design and fabrication,” ACM Trans. Graph., vol. 35, no. 6, pp. 1–13, 2016.
[10]
M. Miki, T. Igarashi, and P. Block, “Parametric self-supporting surfaces via direct computation of airy stress functions,” ACM Trans. Graph., vol. 34, no. 4, pp. 1–12, 2015.
[11]
J. Zehnder, S. Coros, and B. Thomaszewski, “Designing structurally-sound ornamental curve networks,” ACM Trans. Graph., vol. 35, no. 4, pp. 1–10, 2016.
[12]
J. Dumas, A. Lu, S. Lefebvre, J. Wu, and C. Dick, “By-example synthesis of structurally sound patterns,” ACM Trans. Graph., vol. 34, no. 4, pp. 1–12, 2015.
[13]
G. Fang, T. Zhang, S. Zhong, X. Chen, Z. Zhong, and C. C. Wang, “Reinforced FDM: Multi-axis filament alignment with controlled anisotropic strength,” ACM Trans. Graph., vol. 39, no. 6, pp. 1–15, 2020.
[14]
M. Yao, Z. Chen, L. Luo, R. Wang, and H. Wang, “Level-set-based partitioning and packing optimization of a printable model,” ACM Trans. Graph., vol. 34, no. 6, pp. 1–11, 2015.
[15]
C. Schumacher, B. Bickel, J. Rys, S. Marschner, C. Daraio, and M. Gross, “Microstructures to control elasticity in 3D printing,” ACM Trans. Graph., vol. 34, no. 4, pp. 1–13, 2015.
[16]
Z. Lun, M. Gadelha, E. Kalogerakis, S. Maji, and R. Wang, “3D shape reconstruction from sketches via multi-view convolutional networks,” in Proc. Int. Conf. 3D Vis., 2017, pp. 67–77.
[17]
C. Li, H. Pan, Y. Liu, X. Tong, A. Sheffer, and W. Wang, “Robust flow-guided neural prediction for sketch-based freeform surface modeling,” ACM Trans. Graph., vol. 37, no. 6, pp. 1–12, 2018.
[18]
J. Delanoy, M. Aubry, P. Isola, A. A. Efros, and A. Bousseau, “3D sketching using multi-view deep volumetric prediction,” in Proc. ACM Comput. Graph. Interactive Techn., vol. 1, no. 1, pp. 1–22, 2018.
[19]
D. Smirnov, M. Bessmeltsev, and J. Solomon, “Learning manifold patch-based representations of man-made shapes,” in Proc. Int. Conf. Learn. Representations, 2021.
[20]
S.-H. Zhang, Y.-C. Guo, and Q.-W. Gu, “Sketch2model: View-aware 3D modeling from single free-hand sketches,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2021, pp. 6012–6021.
[21]
B. Guillard, E. Remelli, P. Yvernay, and P. Fua, “Sketch2Mesh: Reconstructing and editing 3D shapes from sketches,” in Proc. IEEE Int. Conf. Comput. Vis., 2021, pp. 13 023–13 032.
[22]
N. Wang, Y. Zhang, Z. Li, Y. Fu, W. Liu, and Y.-G. Jiang, “Pixel2Mesh: Generating 3D mesh models from single RGB images,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 52–67.
[23]
P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 1125–1134.
[24]
T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro, “High-resolution image synthesis and semantic manipulation with conditional GANs,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 8798–8807.
[25]
L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, “Occupancy networks: Learning 3D reconstruction in function space,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019, pp. 4460–4470.
[26]
A. Telea and A. Jalba, “Voxel-based assessment of printability of 3D shapes,” in Proc. Int. Symp. Math. Morphol. Appl. Signal Image Process., Springer, 2011, pp. 393–404.
[27]
T. Hughes, The Finite Element Method: Linear Static and Dynamic Finite Element Analysis. New York, NY, USA: Dover, 1987.
[28]
W. Su, D. Du, X. Yang, S. Zhou, and H. Fu, “Interactive sketch-based normal map generation with deep neural networks,” in Proc. ACM Comput. Graph. Interactive Techn., vol. 1, no. 1, pp. 1–17, 2018.
[29]
J. Jiao, Y. Cao, M. Lau, and R. Lau, “Tactile sketch saliency,” in Proc. 28th ACM Int. Conf. Multimedia, 2020, pp. 3072–3080.
[30]
J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in Proc. Int. Conf. Mach. Learn., 2015, pp. 2256–2265.
[31]
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2020, pp. 6840–6851.
[32]
Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” 2020,.
[33]
C. Meng et al., “SDEdit: Guided image synthesis and editing with stochastic differential equations,” in Proc. Int. Conf. Learn. Representations, 2021.
[34]
A. Nichol et al., “GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models,” 2021,.
[35]
A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with CLIP latents,” 2022,.
[36]
A. X. Chang et al., “ShapeNet: An information-rich 3D model repository,” 2015,.
[37]
D. Du et al., “SAniHead: Sketching animal-like 3D character heads using a view-surface collaborative mesh generative network,” IEEE Trans. Vis. Comput. Graph., vol. 28, no. 6, pp. 2415–2429, Jun. 2022.
[38]
Y. Wang, S. Asafi, O. Van Kaick, H. Zhang, D. Cohen-Or, and B. Chen, “Active co-analysis of a set of shapes,” ACM Trans. Graph., vol. 31, no. 6, pp. 1–10, 2012.
[39]
D. Stutz and A. Geiger, “Learning 3D shape completion under weak supervision,” Int. J. Comput. Vis., vol. 128, no. 5, pp. 1162–1181, 2020.
[40]
H. Fu, D. Cohen-Or, G. Dror, and A. Sheffer, “Upright orientation of man-made objects,” in Proc. ACM SIGGRAPH Papers, 2008, pp. 1–7.
[41]
J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986.
[42]
O. Sidi, O. van Kaick, Y. Kleiman, H. Zhang, and D. Cohen-Or, “Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering,” in Proc. SIGGRAPH Asia Conf., 2011, pp. 1–10.
[43]
A. Paszke et al., “PyTorch: An imperative style, high-performance deep learning library,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2019, Art. no.
[44]
X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in Proc. 13th Int. Conf. Artif. Intell. Statist., 2010, pp. 249–256.
[45]
J. Zhou, Z. Luo, Q. Yu, X. Han, and H. Fu, “GA-Sketching: Shape modeling from multi-view sketching with geometry-aligned deep implicit functions,” 2023,.
[46]
D. Yu, M. Lau, L. Gao, and H. Fu, “Sketch beautification: Learning part beautification and structure refinement for sketches of man-made objects,” 2023,.
[47]
Y. Gryaditskaya, M. Sypesteyn, J. W. Hoftijzer, S. C. Pont, F. Durand, and A. Bousseau, “OpenSketch: A richly-annotated dataset of product design sketches,” ACM Trans. Graph., vol. 38, no. 6, 2019, Art. no.
[48]
N. Umetani and R. M. Schmidt, “Cross-sectional structural analysis for 3D printing optimization,” in Proc. SIGGRAPH Asia Tech. Briefs, 2013, Art. no.
[49]
D. Yu et al., “SketchDesc: Learning local sketch descriptors for multi-view correspondence,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 5, pp. 1738–1750, May 2021.
Information & Contributors
Information
Published In
1077-2626 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 13 December 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 17 Jan 2025