Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3363384.3363402acmotherconferencesArticle/Chapter ViewAbstractPublication PageshttfConference Proceedingsconference-collections
research-article

Sketching & Drawing as Future Inquiry in HCI

Published: 19 November 2019 Publication History
  • Get Citation Alerts
  • Abstract

    Creating visual imagery helps us to situate ourselves within unknown worlds, processes, make connections, and find solutions. By exploring drawn ideas for novel technologies, we can examine the implications of their place in the world. Drawing, or sketching, for future inquiry in Human Computer Interaction (HCI) can be a stand-alone investigative approach, part of a wider ‘world-building’ in design fiction, or simply ideation around a concept. By examining instances of existing practice in HCI, in this paper we establish recommendations and rationales for those wishing to utilise sketching and drawing within their research. We examine approaches ranging from ideation, diagramming, scenario building, comics creation and artistic representation to create a model for sketching and drawing as future inquiry for HCI. This work also reflects on the ways in which these arts can inform and elucidate research and practice in HCI, and makes recommendations for the field, within its teaching, processes and outcomes.

    References

    [1]
    Alberto Alessi. 2001. The dream factory: Alessi since 1921. Electa/Alessi.
    [2]
    Nancy C Andreasen. 2012. Creativity in art and science: are there two cultures?Dialogues in clinical neuroscience 14, 1 (2012), 49.
    [3]
    Benjamin Bach, Nathalie Henry Riche, Sheelagh Carpendale, and Hanspeter Pfister. 2017. The emerging genre of data comics. IEEE computer graphics and applications 37, 3 (2017), 6–13.
    [4]
    Eli Blevis, Sabrina Hauser, and William Odom. 2015. Sharing the hidden treasure in pictorials. interactions (2015).
    [5]
    Mark Blythe. 2017. Research fiction: storytelling, plot and design. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5400–5411.
    [6]
    Mark Blythe, Kristina Andersen, Rachel Clarke, and Peter Wright. 2016. Anti-solutionist strategies: Seriously silly design fiction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4968–4978.
    [7]
    Adrien Bousseau, Theophanis Tsandilas, Lora Oehlberg, and Wendy E Mackay. 2016. How novices sketch and prototype hand-fabricated objects. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 397–408.
    [8]
    Virginia Braun, Victoria Clarke, Nikki Hayfield, and Gareth Terry. 2019. Thematic analysis. Handbook of Research Methods in Health Social Sciences (2019), 843–860.
    [9]
    Jeffrey Browne, Bongshin Lee, Sheelagh Carpendale, Timothy Sherwood, and Nathalie Riche. 2011. iSketchVis: Integrating Sketch-based Interaction with Computer Supported Data Analysis. GSWC 2011 (2011), 25.
    [10]
    Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan kaufmann.
    [11]
    Yang Cao, Hai Wang, Changhu Wang, Zhiwei Li, Liqing Zhang, and Lei Zhang. 2010. Mindfinder: interactive sketch-based image search on millions of images. In Proceedings of the 18th ACM international conference on Multimedia. ACM, 1605–1608.
    [12]
    Qi Chen, John Grundy, and John Hosking. 2003. An e-whiteboard application to support early design-stage sketching of UML diagrams. In IEEE Symposium on Human Centric Computing Languages and Environments, 2003. Proceedings. 2003. IEEE, 219–226.
    [13]
    Eunyoung Chung, Carlos Jensen, Koji Yatani, Victor Kuechler, and Khai N Truong. 2010. Sketching and drawing in the design of open source software. In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing. IEEE, 195–202.
    [14]
    Neil Cohn. 2012. Explaining ‘I can‘t draw’: Parallels between the structure and development of language and drawing. Human Development 55, 4 (2012), 167–192. https://doi.org/10.1159/000341842
    [15]
    Paul Coulton, Joseph Lindley, Miriam Sturdee, Mike Stead, 2017. Design fiction as world building. (2017).
    [16]
    Paul Coulton and Joseph Galen Lindley. 2019. More-Than Human Centred Design: Considering Other Things. The Design Journal (2019), 1–19.
    [17]
    Brock Craft and Paul Cairns. 2009. Sketching sketching: outlines of a collaborative design method. In Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology. British Computer Society, 65–72.
    [18]
    Alma L Culén and Asbjørn Følstad. 2014. Innovation in HCI: what can we learn from design thinking?. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational. ACM, 849–852.
    [19]
    Edward De Bono. 1986. De Bono’s thinking course. (1986).
    [20]
    Tomás Dorta. 2008. Design flow and ideation. International Journal of Architectural Computing 6, 3 (2008), 299–316.
    [21]
    Thomas Dykes, Mark Blythe, Jayne Wallace, James Thomas, and Tim Regan. 2016. RtD comics: A medium for representing research through design. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, 971–982.
    [22]
    Stephanie Foehrenbach. 2015. Learn to Sketch (Even if You Can’t Draw): Hands-on Sketching Course. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 2467–2468.
    [23]
    Sean Follmer, Daniel Leithinger, Alex Olwal, Nadia Cheng, and Hiroshi Ishii. 2012. Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In Proceedings of the 25th annual ACM symposium on User interface software and technology. ACM, 519–528.
    [24]
    Christopher Frayling. 1993. Research in art and design. (1993).
    [25]
    Kevin Gallagher, Sameer Patil, and Nasir Memon. 2017. New me: Understanding expert and non-expert perceptions and usage of the Tor anonymity network. In Thirteenth Symposium on Usable Privacy and Security ({SOUPS} 2017). 385–398.
    [26]
    William W Gaver, Jacob Beaver, and Steve Benford. 2003. Ambiguity as a resource for design. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 233–240.
    [27]
    Elisa Giaccardi, Nazli Cila, Chris Speed, and Melissa Caldwell. 2016. Thing ethnography: doing design research with non-humans. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, 377–387.
    [28]
    Gabriela Goldschmidt. 2017. Manual sketching: Why is it still relevant?In The Active Image. Springer, 77–97.
    [29]
    Simon Grand and Martin Wiedmer. 2010. Design fiction: a method toolbox for design research in a complex world. In proceedings of the DRS 2010 conference: Design and Complexity.
    [30]
    Saul Greenberg, Sheelagh Carpendale, Nicolai Marquardt, and Bill Buxton. 2011. Sketching user experiences: The workbook. Elsevier.
    [31]
    Mieke Haesen, Kris Luyten, and Karin Coninx. 2009. Get your requirements straight: Storyboarding revisited. Human-Computer Interaction–INTERACT 2009(2009), 546–549.
    [32]
    Tracy Hammond and Paul Taele. 2019. Sketching Cognition and Creativity: Leveraging Sketch Interfaces for Enhancing Creativity and Cognition. In Proceedings of the 2019 on Creativity and Cognition. ACM, 708–713.
    [33]
    Jessica Hullman and Benjamin Bach. 2018. Picturing Science: Design Patterns in Graphical Abstracts. In International Conference on Theory and Application of Diagrams. Springer, 183–200.
    [34]
    Rahinah Ibrahim and Farzad Pour Rahimian. 2010. Comparison of CAD and manual sketching tools for teaching architectural design. Automation in Construction 19, 8 (2010), 978–987.
    [35]
    Takeo Igarashi, Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 2007. Teddy: a sketching interface for 3D freeform design. In Acm siggraph 2007 courses. ACM, 21.
    [36]
    Brian David Johnson. 2011. Science fiction prototyping: Designing the future with science fiction. Synthesis Lectures on Computer Science 3, 1 (2011), 1–190.
    [37]
    Gabe Johnson, Mark Gross, Ellen Yi-Luen Do, and Jason Hong. 2012. Sketch it, make it: sketching precise drawings for laser cutting. In CHI’12 Extended Abstracts on Human Factors in Computing Systems. ACM, 1079–1082.
    [38]
    Gabe Johnson, Mark D Gross, Jason Hong, Ellen Yi-Luen Do, 2009. Computational support for sketching in design: a review. Foundations and Trends® in Human–Computer Interaction 2, 1(2009), 1–93.
    [39]
    Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design studies 26, 6 (2005), 613–624.
    [40]
    James A Landay and Brad A Myers. 1996. Sketching storyboards to illustrate interface behaviors. In Conference on Human Factors in Computing Systems: Conference companion on Human factors in computing systems: common ground, Vol. 13. 193–194.
    [41]
    Makayla Lewis and Lizzie Coles-Kemp. 2014. A Tactile Visual Library To Support User Experience Storytelling. In Proceedings of NordDesign. 386.
    [42]
    Makayla Lewis, Miriam Sturdee, Jason Alexander, Jelle Van Dijk, Majken Kirkegård Rasmussen, and Thuong Hoang. 2017. SketchingDIS: Hand-drawn Sketching in HCI. In Proceedings of the 2016 ACM Conference Companion Publication on Designing Interactive Systems. ACM, 356–359.
    [43]
    Makayla Lewis, Miriam Sturdee, and Nicolai Marquardt. 2018. Applied Sketching in HCI: Hands-on Course of Sketching Techniques. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, C08.
    [44]
    Makayla Lewis, Miriam Sturdee, and Nicolai Marquardt. 2019. Sketching in HCI: Hands-on Course of Sketching Techniques. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, C04.
    [45]
    Makayla Lewis, Miriam Sturdee, Nicolai Marquardt, and Thuong Hoang. 2018. SketCHI: Hands-On Special Interest Group on Sketching in HCI. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(CHI EA ’18). ACM. https://doi.org/10.1145/3170427.3185366
    [46]
    Makayla Lewis, Miriam Sturdee, Jagoda Walny, Nicolai Marquardt, Thuong Hoang, Joanna Foster, and Sheelagh Carpendale. 2019. SketCHI 2.0: Hands-On Special Interest Group on Sketching in HCI. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, SIG12.
    [47]
    Joseph Lindley, Paul Coulton, and Miriam Sturdee. 2017. Implications for adoption. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 265–277.
    [48]
    Joseph Galen Lindley, Paul Coulton, and Haider Akmal. 2018. Turning Philosophy with a Speculative Lathe: Object Oriented Ontology, Carpentry, and Design Fiction. (2018).
    [49]
    Conor Linehan, Ben J Kirman, Stuart Reeves, Mark A Blythe, Joshua G Tanenbaum, Audrey Desjardins, and Ron Wakkary. 2014. Alternate endings: using fiction to explore design futures. In CHI’14 Extended Abstracts on Human Factors in Computing Systems. ACM, 45–48.
    [50]
    Andrew Loomis. 1963. Successful Drawing. Three-Dimensional Drawing.(Revised Edition of Successful Drawing.) With Illustrations.Chapman & Hall.
    [51]
    Aaron Marcus. 2015. HCI goes mainstream in the comics. In HCI and User-Experience Design. Springer, 265–269.
    [52]
    Nicolai Marquardt and Saul Greenberg. 2012. Sketchnotes for Visual Thinking in HCI. In Proc. ACM Conference on Human Factors in Computing Systems: CHI Workshop on Visual Thinking and Digital Imagery.(Workshop held at the ACM CHI Conference), Vol. 5.
    [53]
    Francis C Moon. 2007. The Machines of Leonardo Da Vinci and Franz Reuleaux: kinematics of machines from the Renaissance to the 20th Century. Vol. 2. Springer Science & Business Media.
    [54]
    Takuya Nojima, Yoshiharu Ooide, and Hiroki Kawaguchi. 2013. Hairlytop interface: An interactive surface display comprised of hair-like soft actuators. In 2013 World Haptics Conference (WHC). IEEE, 431–435.
    [55]
    Mizuki Oka, Kazuhiko Kato, Yingqing Xu, Lin Liang, and Fang Wen. 2008. Scribble-a-secret: Similarity-based password authentication using sketches. In 2008 19th International Conference on Pattern Recognition. IEEE, 1–4.
    [56]
    James Pierce and Carl DiSalvo. 2017. Dark Clouds, Io&#!+, and [Crystal Ball Emoji]: Projecting Network Anxieties with Alternative Design Metaphors. In Proceedings of the 2017 Conference on Designing Interactive Systems. ACM, 1383–1393.
    [57]
    Majken Kirkegård Rasmussen, Giovanni Maria Troiano, Marianne Graves Petersen, Jakob Grue Simonsen, and Kasper Hornbæk. 2016. Sketching Shape-changing Interfaces: Exploring Vocabulary, Metaphors Use, and Affordances. In CHI. 2740–2751.
    [58]
    Pieter Jan Stappers and James M Hennessey. 1999. Toward electronic napkins and beermats: Computer support for visual ideation skills. In Visual representations and interpretations. Springer, 220–225.
    [59]
    Miriam Sturdee, Jason Alexander, Paul Coulton, and Sheelagh Carpendale. 2018. Sketch & The Lizard King: Supporting image inclusion in HCI publishing. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, alt15.
    [60]
    Miriam Sturdee, Paul Coulton, and Jason Alexander. 2017. Using Design Fiction to Inform Shape-Changing Interface Design and Use. The Design Journal 20, sup1 (2017), S4146–S4157.
    [61]
    Miriam Sturdee, Paul Coulton, Joseph G Lindley, Mike Stead, Haider Ali, and Andy Hudson-Smith. 2016. Design fiction: How to build a Voight-Kampff machine. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 375–386.
    [62]
    Miriam Sturdee, Aluna Everitt, Joseph Lindley, Paul Coulton, and Jason Alexander. 2019. Visual Methods for the Design of Shape-Changing Interfaces. In IFIP Conference on Human-Computer Interaction. Springer, xx–xx.
    [63]
    Miriam Sturdee, John Hardy, Nick Dunn, and Jason Alexander. 2015. A Public Ideation of Shape-Changing Applications. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. ACM, 219–228.
    [64]
    Miriam Sturdee, Makayla Lewis, and Nicolai Marquardt. 2018. Feeling SketCHI?: the lasting appeal of the drawn image in HCI. interactions 25, 6 (2018), 64–69.
    [65]
    Miriam Sturdee, Makayla Lewis, and Nicolai Marquardt. 2018. SketchBlog# 1: the rise and rise of the sketchnote. interactions 25, 6 (2018), 6–8.
    [66]
    Miriam Sturdee, Samuel Mann, and Sheelagh Carpendale. 2019. Sketching Sustainability in Computing. In Proceedings of the 2019 on Creativity and Cognition. ACM, 29–40.
    [67]
    Miriam Amber Sturdee. 2018. Sketching as a support mechanism for the design and development of shape-changing interfaces. Ph.D. Dissertation. Lancaster University.
    [68]
    Paul Taele, Laura Barreto, and Tracy Hammond. 2015. Maestoso: an intelligent educational sketching tool for learning music theory. In Twenty-Seventh IAAI Conference.
    [69]
    Desney Tan and Anton Nijholt. 2010. Brain-computer interfaces and human-computer interaction. In Brain-Computer Interfaces. Springer, 3–19.
    [70]
    Maryam Tohidi, William Buxton, Ronald Baecker, and Abigail Sellen. 2006. User sketches: a quick, inexpensive, and effective way to elicit more reflective user feedback. In Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles. ACM, 105–114.
    [71]
    Patrick Tresset and Frederic Fol Leymarie. 2013. Portrait drawing by Paul the robot. Computers & Graphics 37, 5 (2013), 348–363.
    [72]
    Ron Wakkary, Doenja Oogjes, Sabrina Hauser, Henry WJ Lin, Cheng Cao, Leo Ma, and Tijs Duel. 2017. Morse Things: A Design Inquiry into the Gap Between Things and Us. In Conference on Designing Interactive Systems. 503–514.
    [73]
    Jagoda Walny, Jonathan Haber, Marian Dörk, Jonathan Sillito, and Sheelagh Carpendale. 2011. Follow that sketch: Lifecycles of diagrams and sketches in software development. In Visualizing Software for Understanding and Analysis (VISSOFT), 2011 6th IEEE International Workshop on. IEEE, 1–8.
    [74]
    Blake Williford. 2017. SketchTivity: Improving Creativity by Learning Sketching with an Intelligent Tutoring System. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. ACM, 477–483.
    [75]
    Dustin Wüest, Norbert Seyff, and Martin Glinz. 2015. Sketching and notation creation with FlexiSketch Team: Evaluating a new means for collaborative requirements elicitation. In 2015 IEEE 23rd International Requirements Engineering Conference (RE). IEEE, 186–195.
    [76]
    Sarita Yardi, Pamela Krolikowski, Taneshia Marshall, and Amy Bruckman. 2008. An HCI approach to computing in the real world. Journal on Educational Resources in Computing (JERIC) 8, 3 (2008), 9.

    Cited By

    View all
    • (2024)Drinking Chai with Your (AI) Programming Partner: Value Tensions in the Tokenization of Future Human-AI Collaborative WorkProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663390(1-15)Online publication date: 25-Jun-2024
    • (2024)Art Directing Blended ExperiencesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3665970(3551-3562)Online publication date: 1-Jul-2024
    • (2024)AI Art for Self-Interest or Common Good? Uncovering Value Tensions in Artists’ Imaginaries of AI TechnologiesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661619(2897-2910)Online publication date: 1-Jul-2024
    • Show More Cited By

    Index Terms

    1. Sketching & Drawing as Future Inquiry in HCI
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      HTTF 2019: Proceedings of the Halfway to the Future Symposium 2019
      November 2019
      260 pages
      ISBN:9781450372039
      DOI:10.1145/3363384
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 November 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. comics
      2. design fiction
      3. drawing
      4. future
      5. scenarios
      6. sketching

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Funding Sources

      Conference

      HTTF 2019
      HTTF 2019: Halfway to the Future
      November 19 - 20, 2019
      Nottingham, United Kingdom

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)249
      • Downloads (Last 6 weeks)22
      Reflects downloads up to 11 Aug 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Drinking Chai with Your (AI) Programming Partner: Value Tensions in the Tokenization of Future Human-AI Collaborative WorkProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663390(1-15)Online publication date: 25-Jun-2024
      • (2024)Art Directing Blended ExperiencesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3665970(3551-3562)Online publication date: 1-Jul-2024
      • (2024)AI Art for Self-Interest or Common Good? Uncovering Value Tensions in Artists’ Imaginaries of AI TechnologiesProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661619(2897-2910)Online publication date: 1-Jul-2024
      • (2024)Taking the bizarre seriously: dreams as a material for interaction designProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661562(699-713)Online publication date: 1-Jul-2024
      • (2024)Exploring Using Personalised Comics for Healthcare Communication for Patients Living With HemodialysisProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661506(859-873)Online publication date: 1-Jul-2024
      • (2024)Dear Nature: Using data drawings to promote sensemaking in human-nature relationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660732(1426-1438)Online publication date: 1-Jul-2024
      • (2024)From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UXProceedings of the ACM on Human-Computer Interaction10.1145/36373368:CSCW1(1-41)Online publication date: 26-Apr-2024
      • (2024)Explorable Explainable AI: Improving AI Understanding for Community Health Workers in IndiaProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642733(1-21)Online publication date: 11-May-2024
      • (2024)WeaveSlicer: Expanding the Range of Printable Geometries in ClayProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642622(1-16)Online publication date: 11-May-2024
      • (2023)The Museum is Dreaming: Re-Imagining the Museum through Feminist Values and Data Practices in Design FictionProceedings of the 26th International Academic Mindtrek Conference10.1145/3616961.3616984(182-194)Online publication date: 3-Oct-2023
      • Show More Cited By

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media