Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3173574.3173759acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ

Published: 19 April 2018 Publication History

Abstract

We present SymbiosisSketch, a hybrid sketching system that combines drawing in air (3D) and on a drawing surface (2D) to create detailed 3D designs of arbitrary scale in an augmented reality (AR) setting. SymbiosisSketch leverages the complementary affordances of 3D (immersive, unconstrained, life-sized) and 2D (precise, constrained, ergonomic) interactions for in situ 3D conceptual design. A defining aspect of our system is the ongoing creation of surfaces from unorganized collections of 3D curves. These surfaces serve a dual purpose: as 3D canvases to map strokes drawn on a 2D tablet, and as shape proxies to occlude the physical environment and hidden curves in a 3D sketch. SymbiosisSketch users draw interchangeably on a 2D tablet or in 3D within an ergonomically comfortable canonical volume, mapped to arbitrary scale in AR. Our evaluation study shows this hybrid technique to be easy to use in situ and effective in transcending the creative potential of either traditional sketching or drawing in air.

Supplementary Material

suppl.mov (pn2178-file3.mp4)
Supplemental video
suppl.mov (pn2178-file5.mp4)
Supplemental video
MP4 File (pn2178.mp4)

References

[1]
2017. Gravity Sketch - Bringing Virtual Reality Into Your Design Workflow. (2017). https://www.gravitysketch.com/
[2]
2017. MixedRealityToolkit-Unity. (Sept. 2017). https: //github.com/Microsoft/MixedRealityToolkit-Unity original-date: 2016-01--28T18:54:58Z.
[3]
2017. Pixologic : ZBrush - The all-in-one-digital sculpting solution. (2017). http://pixologic.com
[4]
2017. Tilt Brush by Google. (2017). https://www.tiltbrush.com/
[5]
Fatemeh Abbasinejad, Pushkar Joshi, and Nina Amenta. 2011. Surface Patches from Unorganized Space Curves. Computer Graphics Forum 30, 5 (Aug. 2011), 1379--1387.
[6]
Ian G. Angus and Henry A. Sowizral. 1995. Embedding the 2D interaction metaphor in a real 3D virtual environment, Vol. 2409. International Society for Optics and Photonics, 282--294.
[7]
Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5643--5654.
[8]
Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. In Proceedings of the 21st annual ACM symposium on User interface software and technology. ACM, 151--160.
[9]
Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2009. EverybodyLovesSketch: 3D Sketching for a Broader Audience. In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, New York, NY, USA, 59--68.
[10]
Katie Bassett, Ilya Baran, Johannes Schmid, Markus Gross, and Robert W. Sumner. 2013. Authoring and Animating Painterly Characters. ACM Trans. Graph. 32, 5, Article 156 (Oct. 2013), 12 pages.
[11]
Mikhail Bessmeltsev, Caoyu Wang, Alla Sheffer, and Karan Singh. 2012. Design-driven quadrangulation of closed 3D curves. ACM Transactions on Graphics 31, 6 (Nov. 2012), 1.
[12]
Richard L. Bishop. 1975. There is More than One Way to Frame a Curve. The American Mathematical Monthly 82, 3 (March 1975), 246.
[13]
D. A. Bowman, J. Wineman, L. F. Hodges, and D. Allison. 1998. Designing animal habitats within an immersive VE. IEEE Computer Graphics and Applications 18, 5 (Sept. 1998), 9--13.
[14]
D. A. Bowman and C. A. Wingrave. 2001. Design and evaluation of menu systems for immersive virtual environments. In Proceedings IEEE Virtual Reality 2001. 149--156.
[15]
Xuejin Chen, Sing Bing Kang, Ying-Qing Xu, Julie Dorsey, and Heung-Yeung Shum. 2008. Sketching Reality: Realistic Interpretation of Architectural Designs. ACM Trans. Graph. 27, 2 (May 2008), 11:1--11:15.
[16]
Xiang Anthony Chen, Jeeeun Kim, Jennifer Mankoff, Tovi Grossman, Stelian Coros, and Scott E. Hudson. 2016. Reprise: A Design Tool for Specifying, Generating, and Customizing 3D Printable Adaptations on Everyday Objects. ACM Press, 29--39.
[17]
Kelvin Cheng and Masahiro Takatsuk. 2009. Interaction Paradigms for Bare-Hand Interaction with Large Displays at a Distance. In Human-Computer Interaction, Inaki Maurtua (Ed.). InTech, 195--222.
[18]
Siu-Wing Cheng, Tamal Krishna Dey, and Jonathan Richard Shewchuk. 2013. Algorithms for constructing Delaunay triangulations. CRC Press / Taylor & Francis, Boca Raton, Fla. OCLC: 844619099.
[19]
Zeynep Cipiloglu, Abdullah Bulbul, and Tolga Capin. 2010. A framework for enhancing depth perception in computer graphics. ACM Press, 141.
[20]
Josh Constine. 2016. Oculus "Quill" Turns VR Painting Into Performance Art. (2016). http://social.techcrunch.com/2016/01/26/oculus-quill/
[21]
Chris De Paoli and Karan Singh. 2015. SecondSkin: Sketch-based Construction of Layered 3D Models. ACM Trans. Graph. 34, 4 (July 2015), 126:1--126:10.
[22]
Michael F. Deering. 1995. HoloSketch: A Virtual Reality Sketching/Animation Tool. ACM Trans. Comput.-Hum. Interact. 2, 3 (Sept. 1995), 220--238.
[23]
Gianluca Donato and Serge Belongie. 2002. Approximate Thin Plate Spline Mappings. In Computer Vision ECCV 2002, Gerhard Goos, Juris Hartmanis, Jan van Leeuwen, Anders Heyden, Gunnar Sparr, Mads Nielsen, and Peter Johansen (Eds.). Vol. 2352. Springer Berlin Heidelberg, Berlin, Heidelberg, 21--31. http://link.springer.com/10.1007/3--540--47977--5_2
[24]
Tom Dorta, Gokce Kinayoglu, and Michael Hoffmann. 2016. Hyve-3D and the 3D Cursor: Architectural co-design with freedom in Virtual Reality. International Journal of Architectural Computing 14, 2 (2016), 87--102.
[25]
David Eberly. 2000. Least squares fitting of data. (2000).
[26]
Koos Eissen and Roselien Steur. 2011. Sketching: the basics ; the prequel to Sketching: drawing techniques for product designers. BIS, Amsterdam. OCLC: 756275344.
[27]
Michele Fiorentino, Giuseppe Monno, Pietro Alexander Renzulli, and Antonio E. Uva. 2003. 3D Sketch Stroke Segmentation and Fitting in Virtual Reality. In International Conference Graphicon.
[28]
Tovi Grossman, Ravin Balakrishnan, Gordon Kurtenbach, George Fitzmaurice, Azam Khan, and Bill Buxton. 2002. Creating Principal 3D Curves with Digital Tape Drawing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, New York, NY, USA, 121--128.
[29]
Andrew J. Hanson and Hui Ma. 1995. Parallel transport approach to curve framing. Technical Report TR425. 4--7 pages. ftp://ftp.cs.indiana.edu/pub/hanson/iucs-tr425.ps
[30]
Ian P. Howard and Brian J. Rogers. 2008. Depth from monocular cues and vergence. In Seeing in Depth. Oxford University Press, 355--410. http://www.oxfordscholarship.com/view/10.1093/acprof: oso/9780195367607.001.0001/acprof-9780195367607
[31]
Kiia Kallio. 2005. 3D6B Editor: Projective 3D Sketching with Line-Based Rendering. The Eurographics Association. http://diglib.eg.org/handle/10.2312/SBM.SBM05.073-079
[32]
L. B. Kara and K. Shimada. 2007. Sketch-Based 3D-Shape Creation for Industrial Styling Design. IEEE Computer Graphics and Applications 27, 1 (Jan. 2007), 60--71.
[33]
Rubaiat Habib Kazi, Tovi Grossman, Hyunmin Cheong, Ali Hashemi, and George Fitzmaurice. 2017. DreamSketch: Early Stage 3D Design Explorations with Sketching and Generative Design. ACM, Quebec City, QC, Canada.
[34]
Daniel Keefe, Robert Zeleznik, and David Laidlaw. 2007. Drawing on air: Input techniques for controlled 3D line illustration. IEEE Transactions on Visualization and Computer Graphics 13, 5 (2007), 1067--1081. http:// ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4135646
[35]
Daniel F. Keefe, Daniel Acevedo Feliz, Tomer Moscovich, David H. Laidlaw, and Joseph J. LaViola Jr. 2001. CavePainting: a fully immersive 3D artistic medium and interactive experience. In Proceedings of the 2001 symposium on Interactive 3D graphics. ACM, 85--93.
[36]
Yongkwan Kim and Seok-Hyung Bae. 2016. SketchingWithHands: 3D Sketching Handheld Products with First-Person Hand Posture. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). ACM, New York, NY, USA, 797--808.
[37]
Vojtêch Krs, Ersin Yumer, Nathan Carr, Bedrich Benes, and Radomìr Mêch. 2017. Skippy: single view 3D curve interactive modeling. ACM Transactions on Graphics 36, 4 (July 2017), 1--12.
[38]
P. Lapides, E. Sharlin, M.C. Sousa, and L. Streit. 2006. The 3D Tractus: A Three-Dimensional Drawing Board. IEEE, 169--176.
[39]
Yuwei Li, Xi Luo, Youyi Zheng, Pengfei Xu, and Hangbo Fu. 2017. SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image. ACM, Quebec City, QC, Canada.
[40]
John Finley Lucas. 2005. Design and evaluation of 3D multiple object selection techniques. Ph.D. Dissertation. Virginia Tech.
[41]
Patrick Paczkowski, Min H. Kim, Yann Morvan, Julie Dorsey, Holly Rushmeier, and Carol O'Sullivan. 2011. Insitu: Sketching Architectural Designs in Context. In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, 182:1--182:10.
[42]
Scott Robertson and Thomas Bertling. 2013. How to draw: drawing and sketching objects and environments from your imagination. Designstudio Press, Culver City, Calif. OCLC: 897060023.
[43]
E. Sachs, A. Roberts, and D. Stoops. 1991. 3-Draw: a tool for designing 3D shapes. IEEE Computer Graphics and Applications 11, 6 (Nov. 1991), 18--26.
[44]
Bardia Sadri and Karan Singh. 2014. Flow-complex-based Shape Reconstruction from 3D Curves. ACM Trans. Graph. 33, 2 (April 2014), 20:1--20:15.
[45]
Steven Schkolne, Michael Pruett, and Peter Schruder. 2001. Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 261--268.
[46]
Ryan Schmidt, Azam Khan, Karan Singh, and Gord Kurtenbach. 2009. Analytic drawing of 3D scaffolds. ACM Press, 1.
[47]
Ryan Schmidt and Karan Singh. 2008. Sketch-Based Procedural Surface Modeling and Compositing Using Surface Trees. Computer Graphics Forum 27, 2 (April 2008), 321--330.
[48]
Karan Singh. 2006. Industrial motivation for interactive shape modeling: a case study in conceptual automotive design. ACM Press, 3.
[49]
Ivan E. Sutherland. 1963. Sketchpad: a man-machine graphical communication system. ACM Press, 329.
[50]
Zsolt Szalavàri and Michael Gervautz. 1997. The Personal Interaction Panel--a Two-Handed Interface for Augmented Reality. Computer Graphics Forum 16, 3 (Sept. 1997), C335--C346.
[51]
Shun' ichi Tano, Shinya Yamamoto, Junko Ichino, Tomonori Hashiyama, and Mitsuru Iwata. 2013. Truly Useful 3D Drawing System for Professional Designer by "Life-Sized and Operable" Feature and New Interaction. In Human-Computer Interaction--INTERACT 2013. Number 8117 in Lecture Notes in Computer Science. Springer Berlin Heidelberg, 37--55.
[52]
Yannick Thiel, Karan Singh, and Ravin Balakrishnan. 2011. Elasticurves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 383--392.
[53]
Gerold Wesche and Hans-Peter Seidel. 2001. FreeDrawer: a free-form sketching system on the responsive workbench. ACM Press, 167.
[54]
Brian Whited, Eric Daniels, Michael Kaschalk, Patrick Osborne, and Kyle Odermatt. 2012. Computer-assisted Animation of Line and Paint in Disney's Paperman. In ACM SIGGRAPH 2012 Talks (SIGGRAPH '12). ACM, New York, NY, USA, Article 19, 1 pages.
[55]
Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization. ACM Trans. Graph. 33, 4 (July 2014), 131:1--131:13.
[56]
Maria C Yang and Jorge G Cham. 2007. An analysis of sketching skill and its role in early stage engineering design. Journal of Mechanical Design 129, 5 (2007), 476--482.
[57]
Youyi Zheng, Han Liu, Julie Dorsey, and Niloy J. Mitra. 2016. SmartCanvas: Context-inferred Interpretation of Sketches for Preparatory Design Studies. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 37--48.

Cited By

View all
  • (2024)A Virtual Reality Direct-Manipulation Tool for Posing and Animation of Digital Human Bodies: An Evaluation of Creativity SupportMultimodal Technologies and Interaction10.3390/mti80700608:7(60)Online publication date: 10-Jul-2024
  • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
  • (2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
  • Show More Cited By

Index Terms

  1. SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
      April 2018
      8489 pages
      ISBN:9781450356206
      DOI:10.1145/3173574
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 April 2018

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. 3d drawing
      2. augmented reality
      3. design sketching

      Qualifiers

      • Research-article

      Conference

      CHI '18
      Sponsor:

      Acceptance Rates

      CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
      Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

      Upcoming Conference

      CHI 2025
      ACM CHI Conference on Human Factors in Computing Systems
      April 26 - May 1, 2025
      Yokohama , Japan

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)373
      • Downloads (Last 6 weeks)28
      Reflects downloads up to 12 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A Virtual Reality Direct-Manipulation Tool for Posing and Animation of Digital Human Bodies: An Evaluation of Creativity SupportMultimodal Technologies and Interaction10.3390/mti80700608:7(60)Online publication date: 10-Jul-2024
      • (2024)VRCreatIn: Taking In-Situ Pen and Tablet Interaction Beyond Ideation to 3D Modeling Lighting and TexturingProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701580(24-35)Online publication date: 1-Dec-2024
      • (2024)LeARn at Home: Comparing Augmented Reality and Video Conferencing Remote TutoringProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701577(255-263)Online publication date: 1-Dec-2024
      • (2024)VRPaperCrafting: Exploring Child-Friendly Paper Crafting in Virtual Environments using Physical Proxies and Hand GesturesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685379(1-13)Online publication date: 13-Oct-2024
      • (2024)PhoneCanvas: 3D Sketching System Using a Depth Camera-Equipped Smartphone as a CanvasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682078(1-12)Online publication date: 7-Oct-2024
      • (2024)Virtual Task Environments Factors Explored in 3D Selection StudiesProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670983(1-16)Online publication date: 3-Jun-2024
      • (2024)3D-Layers: Bringing Layer-Based Color Editing to VR PaintingACM Transactions on Graphics10.1145/365818343:4(1-15)Online publication date: 19-Jul-2024
      • (2024)OptiBasePen: Mobile Base+Pen Input on Passive Surfaces by Sensing Relative Base Motion Plus Close-Range Pen PositionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676467(1-9)Online publication date: 13-Oct-2024
      • (2024)PortalInk: 2.5D Visual Storytelling with SVG Parallax and Waypoint TransitionsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676376(1-16)Online publication date: 13-Oct-2024
      • (2024)Exploiting Physical Referent Features as Input for Multidimensional Data Selection in Augmented RealityACM Transactions on Computer-Human Interaction10.1145/364861331:4(1-40)Online publication date: 19-Sep-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media