Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3563657.3596014acmconferencesArticle/Chapter ViewAbstractPublication PagesdisConference Proceedingsconference-collections
research-article
Open access

Design Ideation with AI - Sketching, Thinking and Talking with Generative Machine Learning Models

Published: 10 July 2023 Publication History

Abstract

Generative machine learning models provide opportunities to support design work in various parts of the design process. This study investigates how generative machine learning and large language models may play a part in creative design processes of ideation, early prototyping and sketching. A workshop was conducted in which design practitioners and design researchers developed design concepts for a provided design case, with the help of GPT-3. The findings point to three main themes, including i) the practical usefulness and limitations of the system in design ideation processes, ii) how the form of user interaction shapes users’ expectations of the system’s capabilities and potentials, and iii), how the broader discourse around AI both limits and enables how co-creative processes involving human and AI unfolds. The discussion outlines design implications and alternative framings of this kind of co-creative design practices based on post-human perspectives on design and technology use.

References

[1]
Lea Albaugh, Scott E. Hudson, Lining Yao, and Laura Devendorf. 2020. Investigating Underdetermination Through Interactive Computational Handweaving. In Proceedings of the 2020 ACM Designing Interactive Systems Conference(DIS ’20). Association for Computing Machinery, New York, NY, USA, 1033–1046. https://doi.org/10.1145/3357236.3395538
[2]
Kristina Andersen, Ron Wakkary, Laura Devendorf, and Alex McLean. 2019. Digital crafts-machine-ship: creative collaborations with machines. interactions 27, 1 (Dec. 2019), 30–35. https://doi.org/10.1145/3373644
[3]
Karen Barad. 2007. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke University Press.
[4]
Marcelo Bernal, John R. Haymaker, and Charles Eastman. 2015. On the role of computational support for designers in action. Design Studies 41 (Nov. 2015), 163–182. https://doi.org/10.1016/j.destud.2015.08.001
[5]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs] (July 2020). http://arxiv.org/abs/2005.14165 arXiv:2005.14165.
[6]
Scott C. Chase. 2005. Generative design tools for novice designers: Issues for selection. Automation in Construction 14, 6 (Dec. 2005), 689–698. https://doi.org/10.1016/j.autcon.2004.12.004
[7]
Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A. Bharath. 2018. Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine 35, 1 (Jan. 2018), 53–65. https://doi.org/10.1109/MSP.2017.2765202 Conference Name: IEEE Signal Processing Magazine.
[8]
Laura Devendorf, Abigail De Kosnik, Kate Mattingly, and Kimiko Ryokai. 2016. Probing the Potential of Post-Anthropocentric 3D Printing. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems(DIS ’16). Association for Computing Machinery, New York, NY, USA, 170–181. https://doi.org/10.1145/2901790.2901879
[9]
Daniel Fallman. 2003. Design-oriented human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’03). Association for Computing Machinery, New York, NY, USA, 225–232. https://doi.org/10.1145/642611.642652
[10]
Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard. 2019. Mapping the Landscape of Creativity Support Tools in HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–18. https://doi.org/10.1145/3290605.3300619
[11]
Elisa Giaccardi and Johan Redström. 2020. Technology and More-Than-Human Design. Design Issues 36, 4 (Sept. 2020), 33–44. https://doi.org/10.1162/desi_a_00612
[12]
Noe Vargas Hernandez, Jami J. Shah, and Steven M. Smith. 2010. Understanding design ideation mechanisms through multilevel aligned empirical studies. Design Studies 31, 4 (July 2010), 382–410. https://doi.org/10.1016/j.destud.2010.04.001
[13]
Miwa Ikemiya and Daniela K. Rosner. 2014. Broken probes: toward the design of worn media. Pers Ubiquit Comput 18, 3 (March 2014), 671–683. https://doi.org/10.1007/s00779-013-0690-y
[14]
Ben Jonson. 2005. Design ideation: the conceptual sketch in the digital age. Design Studies 26, 6 (Nov. 2005), 613–624. https://doi.org/10.1016/j.destud.2005.03.001
[15]
Martin Jonsson and Jakob Tholander. 2022. Cracking the code: Co-coding with AI in creative programming education. In Creativity and Cognition(C&C ’22). Association for Computing Machinery, New York, NY, USA, 5–14. https://doi.org/10.1145/3527927.3532801
[16]
Lenneke Kuijer and Elisa Giaccardi. 2018. Co-performance: Conceptualizing the Role of Artificial Agency in the Design of Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173699
[17]
Bryan Lawson and Shee Ming Loke. 1997. Computers, words and pictures. Design Studies 18, 2 (April 1997), 171–183. https://doi.org/10.1016/S0142-694X(97)85459-2
[18]
Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288
[19]
Betti Marenko and Philip van Allen. 2016. Animistic design: how to reimagine digital interaction between the human and the nonhuman. Digital Creativity 27, 1 (Jan. 2016), 52–70. https://doi.org/10.1080/14626268.2016.1145127 Publisher: Routledge _eprint: https://doi.org/10.1080/14626268.2016.1145127.
[20]
Jon McCormack, Alan Dorin, and Troy Innocent. 2004. Generative Design: A Paradigm for Design Research. In DRS Biennial Conference Series. https://dl.designresearchsociety.org/drs-conference-papers/drs2004/researchpapers/171
[21]
James Mountstephens and Jason Teo. 2020. Progress and Challenges in Generative Product Design: A Review of Systems. Computers 9, 4 (Dec. 2020), 80. https://doi.org/10.3390/computers9040080 Number: 4 Publisher: Multidisciplinary Digital Publishing Institute.
[22]
J. Brian Pickering, Vegard Engen, and Paul Walland. 2017. The Interplay Between Human and Machine Agency. In Human-Computer Interaction. User Interface Design, Development and Multimodality(Lecture Notes in Computer Science), Masaaki Kurosu (Ed.). Springer International Publishing, Cham, 47–59. https://doi.org/10.1007/978-3-319-58071-5_4
[23]
Martin Porcheron, Joel E. Fischer, Moira McGregor, Barry Brown, Ewa Luger, Heloisa Candello, and Kenton O’Hara. 2017. Talking with Conversational Agents in Collaborative Action. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, Portland Oregon USA, 431–436. https://doi.org/10.1145/3022198.3022666
[24]
Martin Porcheron, Joel E. Fischer, and Sarah Sharples. 2017. "Do Animals Have Accents?": Talking with Agents in Multi-Party Conversation. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW ’17). Association for Computing Machinery, New York, NY, USA, 207–219. https://doi.org/10.1145/2998181.2998298
[25]
Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems(CHI EA ’21). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3411763.3451760
[26]
Robin Roy and James P. Warren. 2019. Card-based design tools: a review and analysis of 155 card decks for designers and designing. Design Studies 63 (July 2019), 125–154. https://doi.org/10.1016/j.destud.2019.04.002
[27]
Donald A. Schön. 1987. Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass, San Francisco, CA, US. Pages: xvii, 355.
[28]
Anna Ståhl, Jakob Tholander, Jarmo Laaksolahti, and Elsa Kosmack-Vaara. 2017. Being, Bringing and Bridging: Three Aspects of Sketching with Nature. In Proceedings of the 2017 Conference on Designing Interactive Systems(DIS ’17). Association for Computing Machinery, New York, NY, USA, 1309–1320. https://doi.org/10.1145/3064663.3064764
[29]
Lucy Suchman. 2017. Agencies in Technology Design: Feminist Reconfigurations*. In Machine Ethics and Robot Ethics. Routledge. Num Pages: 15.
[30]
Lucy A. Suchman. 1987. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press. Google-Books-ID: AJ_eBJtHxmsC.
[31]
Jakob Tholander and Maria Normark. 2020. Crafting Personal Information - Resistance, Imperfection, and Self-Creation in Bullet Journaling. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376410
[32]
Jakob Tholander, Maria Normark, and Chiara Rossitto. 2012. Understanding agency in interaction design materials. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’12). Association for Computing Machinery, New York, NY, USA, 2499–2508. https://doi.org/10.1145/2207676.2208417
[33]
Vasiliki Tsaknaki and Ylva Fernaeus. 2016. Expanding on Wabi-Sabi as a Design Resource in HCI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, New York, NY, USA, 5970–5983. https://doi.org/10.1145/2858036.2858459
[34]
Ron Wakkary. 2020. Nomadic Practices: A Posthuman Theory for Knowing Design. International Journal of Design 14 (Dec. 2020), 117–128.
[35]
Ron Wakkary. 2021. Things We Could Design: For More Than Human-Centered Worlds. MIT Press.
[36]
Danielle Wilde, Anna Vallgårda, and Oscar Tomico. 2017. Embodied Design Ideation Methods: Analysing the Power of Estrangement. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). Association for Computing Machinery, New York, NY, USA, 5158–5170. https://doi.org/10.1145/3025453.3025873

Cited By

View all
  • (2024)Inteligencia Artificial en Illustrator. Percepción y exploración en estudiantes de diseño gráficoInteligencia Artificial en Illustrator. Percepción y exploración en estudiantes de diseño gráficoEuropean Public & Social Innovation Review10.31637/epsir-2024-6709(1-19)Online publication date: 12-Sep-2024
  • (2024)All Just in Your Head? Unraveling the Side Effects of Generative AI Disclosure in Creative TaskSSRN Electronic Journal10.2139/ssrn.4782554Online publication date: 2024
  • (2024)Understanding Fashion Designers’ Behavior Using Generative AI for Early-Stage Concept Ideation and RevisionArchives of Design Research10.15187/adr.2024.07.37.3.2537:3(25-45)Online publication date: 31-Jul-2024
  • Show More Cited By

Index Terms

  1. Design Ideation with AI - Sketching, Thinking and Talking with Generative Machine Learning Models

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      DIS '23: Proceedings of the 2023 ACM Designing Interactive Systems Conference
      July 2023
      2717 pages
      ISBN:9781450398930
      DOI:10.1145/3563657
      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 10 July 2023

      Check for updates

      Author Tags

      1. ChatGPT
      2. GPT-3
      3. LLM
      4. Large Language Models
      5. co-creation
      6. computer supported ideation
      7. generative machine learning
      8. ideation
      9. post-human design

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      DIS '23
      Sponsor:
      DIS '23: Designing Interactive Systems Conference
      July 10 - 14, 2023
      PA, Pittsburgh, USA

      Acceptance Rates

      Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)2,931
      • Downloads (Last 6 weeks)382
      Reflects downloads up to 01 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Inteligencia Artificial en Illustrator. Percepción y exploración en estudiantes de diseño gráficoInteligencia Artificial en Illustrator. Percepción y exploración en estudiantes de diseño gráficoEuropean Public & Social Innovation Review10.31637/epsir-2024-6709(1-19)Online publication date: 12-Sep-2024
      • (2024)All Just in Your Head? Unraveling the Side Effects of Generative AI Disclosure in Creative TaskSSRN Electronic Journal10.2139/ssrn.4782554Online publication date: 2024
      • (2024)Understanding Fashion Designers’ Behavior Using Generative AI for Early-Stage Concept Ideation and RevisionArchives of Design Research10.15187/adr.2024.07.37.3.2537:3(25-45)Online publication date: 31-Jul-2024
      • (2024)AI and Future-Making: Design, Biases, and Human-Plant InteractionsProceedings of the 27th International Academic Mindtrek Conference10.1145/3681716.3681738(24-35)Online publication date: 8-Oct-2024
      • (2024)Investigating How Generative AI Affects Decision-Making in Participatory Design: Shifting the space to make design choicesProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685384(1-14)Online publication date: 13-Oct-2024
      • (2024)"We Are Visual Thinkers, Not Verbal Thinkers!": A Thematic Analysis of How Professional Designers Use Generative AI Image Generation ToolsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685370(1-14)Online publication date: 13-Oct-2024
      • (2024)"I'm a Solo Developer but AI is My New Ill-Informed Co-Worker": Envisioning and Designing Generative AI to Support Indie Game DevelopmentProceedings of the ACM on Human-Computer Interaction10.1145/36770828:CHI PLAY(1-26)Online publication date: 15-Oct-2024
      • (2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
      • (2024)Temporal Aspects of Human-AI Collaborations for WorkProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663397(1-7)Online publication date: 25-Jun-2024
      • (2024)Exploring the Supportive Role of Artificial Intelligence in Participatory Design: A Systematic ReviewProceedings of the Participatory Design Conference 2024: Exploratory Papers and Workshops - Volume 210.1145/3661455.3669868(37-44)Online publication date: 11-Aug-2024
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Get Access

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media