research-article

Suggestion Lists vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorship

Authors:

Florian Lehmann,

Niklas Markert,

Daniel BuschekAuthors Info & Claims

MuC '22: Proceedings of Mensch und Computer 2022

Pages 192 - 208

https://doi.org/10.1145/3543758.3543947

Published: 15 September 2022 Publication History

Abstract

Neural language models have the potential to support human writing. However, questions remain on their integration and influence on writing and output. To address this, we designed and compared two user interfaces for writing with AI on mobile devices, which manipulate levels of initiative and control: 1) Writing with continuously generated text, the AI adds text word-by-word and user steers. 2) Writing with suggestions, the AI suggests phrases and user selects from a list. In a supervised online study (N=18), participants used these prototypes and a baseline without AI. We collected touch interactions, ratings on inspiration and authorship, and interview data. With AI suggestions, people wrote less actively, yet felt they were the author. Continuously generated text reduced this perceived authorship, yet increased editing behavior. In both designs, AI increased text length and was perceived to influence wording. Our findings add new empirical evidence on the impact of UI design decisions on user experience and output with co-creative systems.

Supplementary Material

MP4 File (explainer.mp4)

Supplemental video

Download
5.19 MB

References

[1]

Khaldoun Al Faraj, Mustapha Mojahid, and Nadine Vigouroux. 2009. BigKey: A Virtual Keyboard for Mobile Devices. In Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction, Julie A. Jacko (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 3–10. https://doi.org/10.1007/978-3-642-02580-8_1

Digital Library

[2]

Ahmed Sabbir Arif and Wolfgang Stuerzlinger. 2009. Analysis of text entry performance metrics. In 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH). Institute of Electrical and Electronics Engineers, Toronto, Ontario, Canada, 100–105. https://doi.org/10.1109/TIC-STH.2009.5444533

[3]

Kenneth C. Arnold, Krysta Chauncey, and Krzysztof Z. Gajos. 2020. Predictive Text Encourages Predictable Writing. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 128–138. https://doi.org/10.1145/3377325.3377523

Digital Library

[4]

Kenneth C. Arnold, Krzysztof Z. Gajos, and Adam T. Kalai. 2016. On Suggesting Phrases vs. Predicting Words for Mobile Text Composition. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 603–608. https://doi.org/10.1145/2984511.2984584

Digital Library

[5]

Kenneth C. Arnold, April M. Volzer, and Noah G. Madrid. 2021. Generative Models can Help Writers without Writing for Them. http://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-1.pdf

[6]

Yoshua Bengio. 2008. Neural net language models. Scholarpedia 3, 1 (2008), 3881. https://doi.org/10.4249/scholarpedia.3881 revision #140963.

[7]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. https://doi.org/10.48550/ARXIV.2005.14165 arXiv:2005.14165

[8]

Daniel Buschek, Martin Zürn, and Malin Eiband. 2021. The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 732, 13 pages. https://doi.org/10.1145/3411764.3445372

Digital Library

[9]

Alex Calderwood, Vivian Qiu, Katy Ilonka Gero, and Lydia B Chilton. 2020. How Novelists Use Generative Language Models: An Exploratory User Study.http://ceur-ws.org/Vol-2848/HAI-GEN-Paper-3.pdf

[10]

Haw-Shiuan Chang, Jiaming Yuan, Mohit Iyyer, and Andrew McCallum. 2021. Changing the Mind of Transformers for Topically-Controllable Language Generation. https://doi.org/10.48550/ARXIV.2103.15335 arXiv:2103.15335

[11]

Mia Xu Chen, Benjamin N. Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M. Dai, Zhifeng Chen, Timothy Sohn, and Yonghui Wu. 2019. Gmail Smart Compose: Real-Time Assisted Writing. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (Anchorage, AK, USA) (KDD ’19). Association for Computing Machinery, New York, NY, USA, 2287–2295. https://doi.org/10.1145/3292500.3330723

Digital Library

[12]

Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan. 2021. Wordcraft: a Human-AI Collaborative Editor for Story Writing. https://doi.org/10.48550/ARXIV.2107.07430 arXiv:2107.07430

[13]

Wenzhe Cui, Suwen Zhu, Mingrui Ray Zhang, H. Andrew Schwartz, Jacob O. Wobbrock, and Xiaojun Bi. 2020. JustCorrect: Intelligent Post Hoc Text Correction Techniques on Smartphones. Association for Computing Machinery, New York, NY, USA, 487–499. https://doi.org/10.1145/3379337.3415857

Digital Library

[14]

Lei Gao. 2005. Latin squares in experimental design. http://compneurosci.com/wiki/images/9/98/Latin_square_Method.pdf

[15]

Surjya Ghosh, Kaustubh Hiware, Niloy Ganguly, Bivas Mitra, and Pradipta De. 2019. Does Emotion Influence the Use of Auto-Suggest during Smartphone Typing?. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 144–149. https://doi.org/10.1145/3301275.3302329

Digital Library

[16]

Joshua Goodman, Gina Venolia, Keith Steury, and Chauncey Parker. 2002. Language Modeling for Soft Keyboards. In Proceedings of the 7th International Conference on Intelligent User Interfaces (San Francisco, California, USA) (IUI ’02). Association for Computing Machinery, New York, NY, USA, 194–195. https://doi.org/10.1145/502716.502753

Digital Library

[17]

Asela Gunawardana, Tim Paek, and Christopher Meek. 2010. Usability Guided Key-Target Resizing for Soft Keyboards. In Proceedings of the 15th International Conference on Intelligent User Interfaces (Hong Kong, China) (IUI ’10). Association for Computing Machinery, New York, NY, USA, 111–118. https://doi.org/10.1145/1719970.1719986

Digital Library

[18]

Matthew Guzdial and Mark Riedl. 2019. An Interaction Framework for Studying Co-Creative AI. https://doi.org/10.48550/ARXIV.1903.09709 arXiv:1903.09709

[19]

Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 159–166. https://doi.org/10.1145/302979.303030

Digital Library

[20]

Torsten Hothorn, Kurt Hornik, Mark A. van de Wiel, and Achim Zeileis. 2008. Implementing a class of permutation tests: The coin package. Journal of Statistical Software 28, 8 (2008), 1–23. https://doi.org/10.18637/jss.v028.i08

[21]

Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young, and Vivek Ramavajjala. 2016. Smart Reply: Automated Response Suggestion for Email. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16). Association for Computing Machinery, New York, NY, USA, 955–964. https://doi.org/10.1145/2939672.2939801

Digital Library

[22]

Anna Kantosalo and Hannu Toivonen. 2016. Modes for Creative Human-Computer Collaboration: Alternating and Task-Divided Co-Creativity. In Proceedings of the Seventh International Conference on Computational Creativity (ICCC 2016). Sony CSL, Paris, 77–84. http://www.computationalcreativity.net/iccc2016/wp-content/uploads/2016/01/Modes-for-Creative-Human-Computer-Collaboration.pdf

[23]

Per Ola Kristensson and Keith Vertanen. 2014. The Inviscid Text Entry Rate and Its Application as a Grand Goal for Mobile Text Entry. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (Toronto, ON, Canada) (MobileHCI ’14). Association for Computing Machinery, New York, NY, USA, 335–338. https://doi.org/10.1145/2628363.2628405

Digital Library

[24]

Alexandra Kuznetsova, Per B. Brockhoff, and Rune H. B. Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software 82, 13 (2017), 1–26. https://doi.org/10.18637/jss.v082.i13

[25]

Mina Lee, Percy Liang, and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 388, 19 pages. https://doi.org/10.1145/3491102.3502030

Digital Library

[26]

Santiago Negrete-Yankelevich and Nora Morales-Zaragoza. 2014. The apprentice framework: planning and assessing creativity. In Proceedings of the Seventh International Conference on Computational Creativity (ICCC 2014). computationalcreativity.net, Ljubljana, 280–283. http://computationalcreativity.net/iccc2014/wp-content/uploads/2014/06/13.4_NegreteYankelevich.pdf

[27]

Kseniia Palin, Anna Maria Feit, Sunjun Kim, Per Ola Kristensson, and Antti Oulasvirta. 2019. How Do People Type on Mobile Devices? Observations from a Study with 37,000 Volunteers. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (Taipei, Taiwan) (MobileHCI ’19). Association for Computing Machinery, New York, NY, USA, Article 9, 12 pages. https://doi.org/10.1145/3338286.3340120

Digital Library

[28]

Philip Quinn and Shumin Zhai. 2016. A Cost-Benefit Study of Text Entry Suggestion Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 83–88. https://doi.org/10.1145/2858036.2858305

Digital Library

[29]

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9. http://www.persagen.com/files/misc/radford2019language.pdf

[30]

Melissa Roemmele. 2021. Inspiration through Observation: Demonstrating the Influence of Automatically Generated Text on Creative Writing. https://doi.org/10.48550/ARXIV.2107.04007 arXiv:2107.04007

[31]

Melissa Roemmele and Andrew S. Gordon. 2015. Creative Help: A Story Writing Assistant. In Interactive Storytelling, Henrik Schoenau-Fog, Luis Emilio Bruni, Sandy Louchart, and Sarune Baceviciute (Eds.). Springer International Publishing, Cham, 81–92.

[32]

Nikhil Singh, Guillermo Bernal, Daria Savchenko, and Elena L. Glassman. 2022. Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence. https://doi.org/10.1145/3511599 Just Accepted.

Digital Library

[33]

Paul D. Varcholik, Joseph J. LaViola, and Charles E. Hughes. 2012. Establishing a baseline for text entry for a multi-touch virtual keyboard. International Journal of Human-Computer Studies 70, 10 (2012), 657–672. https://doi.org/10.1016/j.ijhcs.2012.05.007 Special issue on Developing, Evaluating and Deploying Multi-touch Systems.

Digital Library

[34]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. https://doi.org/10.48550/ARXIV.1706.03762 arXiv:1706.03762

[35]

Jacob O. Wobbrock and Matthew Kay. 2016. Nonparametric Statistics in Human–Computer Interaction. Springer International Publishing, Cham, 135–170. https://doi.org/10.1007/978-3-319-26633-6_7

[36]

Mingrui Ray Zhang, He Wen, and Jacob O. Wobbrock. 2019. Type, Then Correct: Intelligent Text Correction Techniques for Mobile Text Entry Using Neural Networks. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 843–855. https://doi.org/10.1145/3332165.3347924

Digital Library

Cited By

Fan PJiang Q(2024)Exploring the Factors Influencing Continuance Intention to Use AI Drawing Tools: Insights from DesignersSystems10.3390/systems1203006812:3(68)Online publication date: 22-Feb-2024
https://doi.org/10.3390/systems12030068
He JHoude SGonzalez GSilva Moran DRoss SMuller MWeisz J(2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663398
Laban PVig JHearst MXiong CWu C(2024)Beyond the Chat: Executable and Verifiable Text-Editing with LLMsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676419(1-23)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676419
Show More Cited By

Index Terms

Suggestion Lists vs. Continuous Generation: Interaction Design for Writing with Generative Models on Mobile Devices Affect Text Length, Wording and Perceived Authorship
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction techniques
      1. Text input

Recommendations

The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

We present an in-depth analysis of the impact of multi-word suggestion choices from a neural language model on user behaviour regarding input and text composition in email writing. Our study for the first time compares different numbers of parallel ...
How Older Adults Learn to Use Mobile Devices: Survey and Field Investigations

Mobile computing devices, such as smart phones, offer benefits that may be especially valuable to older adults (age 65+). Yet, older adults have been shown to have difficulty learning to use these devices. In the research presented in this article, we ...
The effect of explanations on perceived control and behaviors in intelligent systems
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

Intelligent systems often violate fundamental usability principles, such as control and transparency. Explanations have been shown to have a positive effect on transparency in such systems but little research exists as to how it may affect control. We ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MuC '22: Proceedings of Mensch und Computer 2022

September 2022

624 pages

ISBN:9781450396905

DOI:10.1145/3543758

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 September 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

MuC '22

MuC '22: Mensch und Computer 2022

September 4 - 7, 2022

Darmstadt, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
440
Total Downloads

Downloads (Last 12 months)201
Downloads (Last 6 weeks)15

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fan PJiang Q(2024)Exploring the Factors Influencing Continuance Intention to Use AI Drawing Tools: Insights from DesignersSystems10.3390/systems1203006812:3(68)Online publication date: 22-Feb-2024
https://doi.org/10.3390/systems12030068
He JHoude SGonzalez GSilva Moran DRoss SMuller MWeisz J(2024)AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasProceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work10.1145/3663384.3663398(1-14)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3663384.3663398
Laban PVig JHearst MXiong CWu C(2024)Beyond the Chat: Executable and Verifiable Text-Editing with LLMsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676419(1-23)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676419
Buschek D(2024)Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI ToolsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660681(2719-2737)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660681
Draxler FWerner ALehmann FHoppe MSchmidt ABuschek DWelsch R(2024)The AI Ghostwriter Effect: When Users do not Perceive Ownership of AI-Generated Text but Self-Declare as AuthorsACM Transactions on Computer-Human Interaction10.1145/363787531:2(1-40)Online publication date: 5-Feb-2024
https://dl.acm.org/doi/10.1145/3637875
Guo APataranutaporn PMaes P(2024)Exploring the Impact of AI Value Alignment in Collaborative Ideation: Effects on Perception, Ownership, and OutputExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650892(1-11)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650892
Masson DMalacria SCasiez GVogel D(2024)DirectGPT: A Direct Manipulation Interface to Interact with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642462(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642462
Fernandes PMadaan ALiu EFarinhas AMartins PBertsch Ade Souza JZhou SWu TNeubig GMartins A(2023)Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language GenerationTransactions of the Association for Computational Linguistics10.1162/tacl_a_0062611(1643-1668)Online publication date: 19-Dec-2023
https://doi.org/10.1162/tacl_a_00626
Lehmann FKornecki IBuschek DFeit A(2023)Typing Behavior is About More than Speed: Users' Strategies for Choosing Word Suggestions Despite Slower Typing RatesProceedings of the ACM on Human-Computer Interaction10.1145/36042767:MHCI(1-26)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604276
Lehmann F(2023)Mixed-Initiative Interaction with Computational Generative SystemsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3577061(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3577061
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents