research-article

Juxtaform: interactive visual summarization for exploratory shape design

Authors:

Fanny Chevalier,

Karan SinghAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 42, Issue 4

Article No.: 52, Pages 1 - 14

https://doi.org/10.1145/3592436

Published: 26 July 2023 Publication History

Abstract

We present juxtaform, a novel approach to the interactive summarization of large shape collections for conceptual shape design. We conduct a formative study to ascertain design goals for creative shape exploration tools. Motivated by a mathematical formulation of these design goals, juxtaform integrates the exploration, analysis, selection, and refinement of large shape collections to support an interactive divergence-convergence shape design workflow. We exploit sparse, segmented sketch-stroke visual abstractions of shape and a novel visual summarization algorithm to balance the needs of shape understanding, in-situ shape juxtaposition, and visual clutter. Our evaluation is three-fold: we show that existing shape and stroke clustering algorithms do not address our design goals compared to our proposed shape corpus summarization algorithm; we compare juxtaform against a structured image gallery interface for various shape design and analysis tasks; and we present multiple compelling 2D/3D applications using juxtaform.

Supplementary Material

ZIP File (papers_698-supplemental.zip)

supplemental material

Download
132.60 MB

MP4 File (papers_698_VOD.mp4)

presentation

Download
169.82 MB

References

[1]

Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, and Michael Cohen. 2004. Interactive digital photomontage. In ACM SIGGRAPH 2004 Papers. 294--302.

Digital Library

[2]

Andrés Arias-Rosales. 2022. The perceived value of human-AI collaboration in early shape exploration: An exploratory assessment. PloS one 17, 9 (2022), e0274496.

[3]

Rahul Arora, Ishan Darolia, Vinay P Namboodiri, Karan Singh, and Adrien Bousseau. 2017. Sketchsoup: Exploratory ideation using design sketches. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 302--312.

[4]

Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. 2018. Symbiosissketch: Combining 2d & 3d sketching for designing detailed 3d objects in situ. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--15.

Digital Library

[5]

Jackie Assa, Yaron Caspi, and Daniel Cohen-Or. 2005. Action synopsis: pose selection and illustration. ACM Transactions on Graphics (TOG) 24, 3 (2005), 667--676.

Digital Library

[6]

Melinos Averkiou, Vladimir G Kim, Youyi Zheng, and Niloy J Mitra. 2014. Shapesynth: Parameterizing model collections for coupled shape exploration and synthesis. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 125--134.

[7]

Pierre Bénard and Aaron Hertzmann. 2019. Line Drawings from 3D Models: A Tutorial. Found. Trends. Comput. Graph. Vis. 11, 1--2 (sep 2019), 1--159.

Digital Library

[8]

Silvia Biasotti, Andrea Cerri, Alex Bronstein, and Michael Bronstein. 2016a. Recent trends, applications, and perspectives in 3d shape similarity assessment. In Computer graphics forum, Vol. 35. Wiley Online Library, 87--119.

[9]

Silvia Biasotti, Bianca Falcidieno, Daniela Giorgi, and Michela Spagnuolo. 2016b. 3D Objects Exploration: Guidelines for Future Research. In 3DOR@ Eurographics.

[10]

Silvia Biasotti, Elia Moscoso Thompson, and Michela Spagnuolo. 2019. Context-adaptive navigation of 3D model collections. Computers & Graphics 79 (2019), 1--13.

Digital Library

[11]

Umberto Castellani and Adrien Bartoli. 2020. 3d shape registration. In 3D Imaging, Analysis and Applications. Springer, 353--411.

[12]

Tony F Chan and Luminita A Vese. 2001. Active contours without edges. IEEE Transactions on image processing 10, 2 (2001), 266--277.

Digital Library

[13]

Andy Cockburn, Amy Karlson, and Benjamin B Bederson. 2009. A review of overview+ detail, zooming, and focus+ context interfaces. ACM Computing Surveys (CSUR) 41, 1 (2009), 1--31.

Digital Library

[14]

Daniel Cohen-Or and Hao Zhang. 2016. From Inspired Modeling to Creative Modeling. Vis. Comput. 32, 1 (jan 2016), 7--14.

Digital Library

[15]

Noa Fish, Melinos Averkiou, Oliver Van Kaick, Olga Sorkine-Hornung, Daniel Cohen-Or, and Niloy J Mitra. 2014. Meta-representation of shape families. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--11.

Digital Library

[16]

Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler. 2022. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images. In Advances In Neural Information Processing Systems.

[17]

Lin Gao, Yan-Pei Cao, Yu-Kun Lai, Hao-Zhi Huang, Leif Kobbelt, and Shi-Min Hu. 2014. Active exploration of large 3d model repositories. IEEE transactions on visualization and computer graphics 21, 12 (2014), 1390--1402.

[18]

Yunhao Ge, Sami Abu-El-Haija, Gan Xin, and Laurent Itti. 2021. Zero-shot Synthesis with Group-Supervised Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=8wqCDnBmnrT

[19]

Stéphane Grabli, Frédo Durand, and Francois X Sillion. 2004. Density measure for line-drawing simplification. In 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings. IEEE, 309--318.

[20]

Yulia Gryaditskaya, Felix Hähnlein, Chenxi Liu, Alla Sheffer, and Adrien Bousseau. 2020. Lifting freehand concept sketches into 3D. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--16.

Digital Library

[21]

Felix Hähnlein, Changjian Li, Niloy J Mitra, and Adrien Bousseau. 2022. CAD2Sketch: Generating Concept Sketches from CAD Sequences. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1--18.

Digital Library

[22]

Almoctar Hassoumi, María Jesús Lobo, Gabriel Jarry, Vsevolod Peysakhovich, and Christophe Hurter. 2019. Interactive shape based brushing technique for trail sets. (2019).

[23]

James W Hennessey, Han Liu, Holger Winnemöller, Mira Dontcheva, and Niloy J Mitra. 2016. How2Sketch: generating easy-to-follow tutorials for sketching 3D objects. arXiv preprint arXiv:1607.07980 (2016).

[24]

Josh Holinaty, Alec Jacobson, and Fanny Chevalier. 2021. Supporting Reference Imagery for Digital Drawing. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2434--2442.

[25]

Min Hua. 2019. The roles of sketching in supporting creative design. The Design Journal 22, 6 (2019), 895--904.

[26]

Ruqi Huang, Panos Achlioptas, Leonidas Guibas, and Maks Ovsjanikov. 2019. Limit Shapes - A Tool for Understanding Shape Differences and Variability in 3D Model Collections. SGP (2019).

[27]

Shi-Sheng Huang, Ariel Shamir, Chao-Hui Shen, Hao Zhang, Alla Sheffer, Shi-Min Hu, and Daniel Cohen-Or. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--10.

Digital Library

[28]

Moos Hueting, Maks Ovsjanikov, and Niloy J Mitra. 2015. CrossLink: joint understanding of image and 3D model collections through shape and camera pose variations. ACM Transactions on Graphics (ToG) 34, 6 (2015), 1--13.

Digital Library

[29]

Arjun Jain, Thorsten Thormählen, Tobias Ritschel, and Hans-Peter Seidel. 2012. Exploring shape variations by 3d-model decomposition and part-based recombination. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 631--640.

[30]

Waqas Javed and Niklas Elmqvist. 2012. Exploring the design space of composite visualization. In 2012 ieee pacific visualization symposium. IEEE, 1--8.

[31]

Vladimir G Kim, Wilmot Li, Niloy J Mitra, Stephen DiVerdi, and Thomas Funkhouser. 2012. Exploring collections of 3d models using fuzzy correspondences. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--11.

Digital Library

[32]

Yanir Kleiman, Noa Fish, Joel Lanir, and Daniel Cohen-Or. 2013. Dynamic maps for exploring and browsing shapes. In Computer Graphics Forum, Vol. 32. Wiley Online Library, 187--196.

[33]

Heidi Lam and Tamara Munzner. 2010. A guide to visual multi-level interface design from synthesis of empirical study evidence. Synthesis Lectures on Visualization 1, 1 (2010), 1--117.

[34]

Yong Jae Lee, C Lawrence Zitnick, and Michael F Cohen. 2011. Shadowdraw: real-time user guidance for freehand drawing. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1--10.

Digital Library

[35]

You-En Lin, Yong-Liang Yang, and Hung-Kuo Chu. 2018. Scale-aware black-and-white abstraction of 3D shapes. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--11.

Digital Library

[36]

Chenxi Liu, Enrique Rosales, and Alla Sheffer. 2018. Strokeaggregator: Consolidating raw sketches into artist-intended curve drawings. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1--15.

Digital Library

[37]

Y-C Liu, Amaresh Chakrabarti, and T Bligh. 2003. Towards an 'ideal'approach for concept generation. Design studies 24, 4 (2003), 341--355.

[38]

Zhen-Bao Liu, Shu-Hui Bu, Kun Zhou, Shu-Ming Gao, Jun-Wei Han, and Jun Wu. 2013. A survey on partial retrieval of 3D shapes. Journal of Computer Science and Technology 28, 5 (2013), 836--851.

[39]

Justin Matejka, Michael Glueck, Erin Bradner, Ali Hashemi, Tovi Grossman, and George Fitzmaurice. 2018. Dream Lens: Exploration and Visualization of Large-Scale Generative Design Datasets. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--12.

Digital Library

[40]

Harry A Nichols. 1954. DESIGN FEATURES OF THE DOUGLAS A3D SKYWARRIOR. Technical Report. SAE Technical Paper.

[41]

Jamie R. Nuñez, Christopher R. Anderton, and Ryan S. Renslow. 2018. Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data. PLOS ONE 13, 7 (08 2018), 1--14.

[42]

Luke Olsen, Faramarz F Samavati, Mario Costa Sousa, and Joaquim A Jorge. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1 (2009), 85--103.

Digital Library

[43]

Günay Orbay, Mehmet Ersin Yümer, and Levent Burak Kara. 2012. Sketch-based aesthetic product form exploration from existing images using piecewise clothoid curves. Journal of Visual Languages & Computing 23, 6 (2012), 327--339.

Digital Library

[44]

Maks Ovsjanikov, Wilmot Li, Leonidas Guibas, and Niloy J Mitra. 2011. Exploration of continuous variability in collections of 3d shapes. ACM Transactions on Graphics (TOG) 30, 4 (2011), 1--10.

Digital Library

[45]

Patrick Paczkowski, Min H Kim, Yann Morvan, Julie Dorsey, Holly E Rushmeier, and Carol O'Sullivan. 2011. Insitu: sketching architectural designs in context. ACM Trans. Graph. 30, 6 (2011), 182.

Digital Library

[46]

Mark Pauly, Niloy J Mitra, Johannes Wallner, Helmut Pottmann, and Leonidas J Guibas. 2008. Discovering structural regularity in 3D geometry. In ACM SIGGRAPH 2008 papers. 1--11.

Digital Library

[47]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825--2830.

[48]

Miquel Prats. 2007. Shape exploration in product design: assisting transformation in pictorial representations. Open University (United Kingdom).

[49]

Miquel Prats and Chris F Earl. 2006. Exploration through drawings in the conceptual stage of product design. In Design Computing and Cognition'06. Springer, 83--102.

[50]

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021. High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752 [cs.CV]

[51]

Ryan M. Schmidt and Karan Singh. 2010. meshmixer: an interface for rapid mesh composition. In SIGGRAPH Talks, James L. Mohler (Ed.). ACM. http://dblp.unitrier.de/db/conf/siggraph/siggraph2010talks.html#SchmidtS10

[52]

Donald A Schon and Glenn Wiggins. 1992. Kinds of seeing and their functions in designing. Design studies 13, 2 (1992), 135--156.

[53]

Adriana Schulz, Ariel Shamir, Ilya Baran, David IW Levin, Pitchaya Sitthi-Amorn, and Wojciech Matusik. 2017. Retrieval on parametric shape collections. ACM Transactions on Graphics (TOG) 36, 1 (2017), 1--14.

Digital Library

[54]

Ben Shneiderman. 1999. User interfaces for creativity support tools. In Proceedings of the 3rd conference on Creativity & cognition. 15--22.

Digital Library

[55]

Tim Smithers. 2001. Is sketching an aid to memory or a kind of thinking? (2001).

[56]

Johan WH Tangelder and Remco C Veltkamp. 2008. A survey of content based 3D shape retrieval methods. Multimedia tools and applications 39, 3 (2008), 441--471.

[57]

Erdogan Taskesen. 2021. Python package clustimage is for unsupervised clustering of images. https://erdogant.github.io/clustimage

[58]

James T Todd. 2004. The visual perception of 3D shape. Trends in cognitive sciences 8, 3 (2004), 115--121.

[59]

Steve Tsang, Ravin Balakrishnan, Karan Singh, and Abhishek Ranjan. 2004. A Suggestive Interface for Image Guided 3D Sketching. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria) (CHI '04). Association for Computing Machinery, New York, NY, USA, 591--598.

Digital Library

[60]

Oliver Van Kaick, Hao Zhang, Ghassan Hamarneh, and Daniel Cohen-Or. 2011. A survey on shape correspondence. In Computer graphics forum, Vol. 30. Wiley Online Library, 1681--1707.

[61]

Dave Pagurek Van Mossel, Chenxi Liu, Nicholas Vining, Mikhail Bessmeltsev, and Alla Sheffer. 2021. StrokeStrip: joint parameterization and fitting of stroke clusters. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1--18.

Digital Library

[62]

Remco C Veltkamp. 2001. Shape matching: Similarity measures and algorithms. In Proceedings International Conference on Shape Modeling and Applications. IEEE, 188--197.

[63]

Aaron Wolin, Brian Eoff, and Tracy Hammond. 2008. ShortStraw: A Simple and Effective Corner Finder for Polylines. SBIM 8 (2008), 33--40.

[64]

Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1912--1920.

[65]

Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014a. True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics 33, 4 (2014).

Digital Library

[66]

Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu, Ariel Shamir, Daniel Cohen-Or, and Hui Huang. 2014b. Organizing heterogeneous scene collections through contextual focal points. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--12.

Digital Library

[67]

Kai Xu, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2012. Fit and diverse: Set evolution for inspiring 3d shape galleries. ACM Transactions on Graphics (TOG) 31, 4 (2012), 1--10.

Digital Library

[68]

Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A benchmark for rough sketch cleanup. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--14.

Digital Library

[69]

Hui Ye, Kin Chung Kwan, and Hongbo Fu. 2021. 3D curve creation on and around physical objects with mobile AR. IEEE transactions on visualization and computer graphics 28, 8 (2021), 2809--2821.

[70]

Jun-Yan Zhu, Yong Jae Lee, and Alexei A Efros. 2014. Averageexplorer: Interactive exploration and alignment of visual data collections. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--11.

Digital Library

Cited By

Ge LHedayati MCui YDing YBonilla KJoshi AOttley ABach BKwon BRapp DPeck EPadilla LCorrell MBorkin MHarrison LKay M(2024)Toward a More Comprehensive Understanding of Visualization LiteracyExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636289(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636289

Index Terms

Juxtaform: interactive visual summarization for exploratory shape design
1. Computing methodologies
  1. Computer graphics

Recommendations

Geometric modeling in shape space
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We present a novel framework to treat shapes in the setting of Riemannian geometry. Shapes -- triangular meshes or more generally straight line graphs in Euclidean space -- are treated as points in a shape space. We introduce useful Riemannian metrics ...
Geometric modeling in shape space

We present a novel framework to treat shapes in the setting of Riemannian geometry. Shapes -- triangular meshes or more generally straight line graphs in Euclidean space -- are treated as points in a shape space. We introduce useful Riemannian metrics ...
Interactive shape design using volumetric implicit PDEs
SM '03: Proceedings of the eighth ACM symposium on Solid modeling and applications

Solid modeling based on Partial Differential Equations (PDEs) can potentially unify both geometric constraints and functional requirements within a single design framework to model real-world objects via its explicit, direct integration with parametric ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 42, Issue 4

August 2023

1912 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3609020

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2023

Published in TOG Volume 42, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
207
Total Downloads

Downloads (Last 12 months)207
Downloads (Last 6 weeks)4

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ge LHedayati MCui YDing YBonilla KJoshi AOttley ABach BKwon BRapp DPeck EPadilla LCorrell MBorkin MHarrison LKay M(2024)Toward a More Comprehensive Understanding of Visualization LiteracyExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636289(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636289

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents