research-article

Open access

"It's like a rubber duck that talks back": Understanding Generative AI-Assisted Data Analysis Workflows through a Participatory Prompting Study

Authors:

Carina Negreanu,

Lev TankelevitchAuthors Info & Claims

CHIWORK '24: Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work

Article No.: 16, Pages 1 - 21

https://doi.org/10.1145/3663384.3663389

Published: 25 June 2024 Publication History

All formats PDF

Abstract

Generative AI tools can help users with many tasks. One such task is data analysis, which is notoriously challenging for non-expert end-users due to its expertise requirements, and where AI holds much potential, such as finding relevant data sources, proposing analysis strategies, and writing analysis code. To understand how data analysis workflows can be assisted or impaired by generative AI, we conducted a study (n=15) using Bing Chat via participatory prompting. Participatory prompting is a recently developed methodology in which users and researchers reflect together on tasks through co-engagement with generative AI. In this paper we demonstrate the value of the participatory prompting method. We found that generative AI benefits the information foraging and sensemaking loops of data analysis in specific ways, but also introduces its own barriers and challenges, arising from the difficulties of query formulation, specifying context, and verifying results.

References

[1]

Lisa P. Argyle, E. Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, Taylor Sorensen, and David Wingate. 2022. An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels. Political Analysis 31 (2022), 337 – 351. https://api.semanticscholar.org/CorpusID:252280474

[2]

Shraddha Barke, Michael B James, and Nadia Polikarpova. 2023. Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (2023), 85–111.

Digital Library

[3]

Hugh Beyer and Karen Holtzblatt. 1997. Contextual Design: Defining Customer-Centered Systems. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[4]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[5]

Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–21.

Digital Library

[6]

Tingfeng Cao, Chengyu Wang, Bingyan Liu, Ziheng Wu, Jinhui Zhu, and Jun Huang. 2023. BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesis. In Conference on Empirical Methods in Natural Language Processing. https://api.semanticscholar.org/CorpusID:265150243

[7]

George Chalhoub and Advait Sarkar. 2022. “It’s Freedom to Put Things Where My Mind Wants”: Understanding and Improving the User Experience of Structuring Data in Spreadsheets. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 585, 24 pages. https://doi.org/10.1145/3491102.3501833

Digital Library

[8]

Souti Chattopadhyay, Zixuan Feng, Emily Arteaga, Audrey Au, Gonzalo Ramos, Titus Barik, and Anita Sarma. 2023. Make It Make Sense! Understanding and Facilitating Sensemaking in Computational Notebooks. arXiv preprint arXiv:2312.11431 (2023).

[9]

Chen Chen, Jane Hoffswell, Shunan Guo, Ryan Rossi, Yeuk-Yin Chan, Fan Du, Eunyee Koh, and Zhicheng Liu. 2023. WHATSNEXT: Guidance-enriched Exploratory Data Analysis with Interactive, Low-Code Notebooks. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 209–214.

[10]

Hai Dang, Sven Goller, Florian Lehmann, and Daniel Buschek. 2023. Choice over control: How users write with large language models using diegetic and non-diegetic prompting. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[11]

Ewart J De Visser, Samuel S Monfort, Ryan McKendrick, Melissa AB Smith, Patrick E McKnight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology: Applied 22, 3 (2016), 331.

[12]

Stephen L Dorton and Robert A Hall. 2021. Collaborative human-AI sensemaking for intelligence analysis. In International conference on human-computer interaction. Springer, 185–201.

Digital Library

[13]

Gregor Engels and Martin Erwig. 2005. ClassSheets: automatic generation of spreadsheet applications from object-oriented specifications. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering. 124–133.

Digital Library

[14]

Kasra Ferdowsi, Jack Williams, Ian Drosos, Andrew D. Gordon, Carina Negreanu, Nadia Polikarpova, Advait Sarkar, and Benjamin Zorn. 2023. COLDECO: An End User Spreadsheet Inspection Tool for AI-Generated Code. In 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 82–91. https://doi.org/10.1109/VL-HCC57772.2023.00017

[15]

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv abs/2312.10997 (2023). https://api.semanticscholar.org/CorpusID:266359151

[16]

Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, and Nikolas Martelaro. 2023. Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–20.

Digital Library

[17]

Andrew D Gordon, Carina Negreanu, José Cambronero, Rasika Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson, Advait Sarkar, Stephanie Simmons, 2023. Co-audit: tools to help humans double-check AI-generated content. arXiv preprint arXiv:2310.01297 (2023).

[18]

Valentina Grigoreanu, Margaret Burnett, Susan Wiedenbeck, Jill Cao, Kyle Rector, and Irwin Kwan. 2012. End-user debugging strategies: A sensemaking perspective. ACM Transactions on Computer-Human Interaction (TOCHI) 19, 1 (2012), 1–28.

Digital Library

[19]

Ken Gu, Madeleine Grunde-McLaughlin, Andrew M McNutt, Jeffrey Heer, and Tim Althoff. 2023. How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study. arXiv preprint arXiv:2309.10108 (2023).

[20]

Ken Gu, Ruoxi Shang, Tim Althoff, Chenglong Wang, and Steven M. Drucker. 2023. How Do Analysts Understand and Verify AI-Assisted Data Analyses?arxiv:2309.10947 [cs.HC]

[21]

Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. ACM Sigplan Notices 46, 1 (2011), 317–330.

Digital Library

[22]

Amber Horvath, Brad Myers, Andrew Macvean, and Imtiaz Rahman. 2022. Using Annotations for Sensemaking About Code. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–16.

Digital Library

[23]

Dhanya Jayagopal, Justin Lubin, and Sarah E Chasins. 2022. Exploring the learnability of program synthesizers by novice programmers. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–15.

Digital Library

[24]

Theodore Jensen and Mohammad Maifi Hasan Khan. 2022. I’m Only Human: The Effects of Trust Dampening by Anthropomorphic Agents. In International Conference on Human-Computer Interaction. Springer, 285–306.

[25]

Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation. ACM Comput. Surv. 55, 12, Article 248 (mar 2023), 38 pages. https://doi.org/10.1145/3571730

Digital Library

[26]

Nima Joharizadeh, Advait Sarkar, Andrew D. Gordon, and Jack Williams. 2020. Gridlets: Reusing Spreadsheet Grids. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3334480.3382806

Digital Library

[27]

Philip N Johnson-Laird and Keith Oatley. 1998. Basic emotions, rationality, and folk theory. In Consciousness and Emotion in Cognitive Science. Routledge, 289–311.

[28]

Simon Peyton Jones, Alan Blackwell, and Margaret Burnett. 2003. A user-centred approach to functions in Excel. In Proceedings of the eighth ACM SIGPLAN international conference on Functional programming. 165–176.

Digital Library

[29]

Karl E. Weick. 1969. The Social Psychology of Organizing. Addison Wesley, Reading, MA.

[30]

Karl E. Weick. 1995. Sensemaking in Organizations. SAGE Publications, Thousand Oaks, CA.

[31]

Suzanne Kieffer. 2017. ECOVAL: Ecological Validity of Cues and Representative Design in User Experience Evaluations. AIS Transactions on Human-Computer Interaction 9, 2 (June 2017), 149–172. https://aisel.aisnet.org/thci/vol9/iss2/4

[32]

Amy J Ko, Robin Abraham, Laura Beckwith, Alan Blackwell, Margaret Burnett, Martin Erwig, Chris Scaffidi, Joseph Lawrance, Henry Lieberman, Brad Myers, 2011. The state of the art in end-user software engineering. ACM Computing Surveys (CSUR) 43, 3 (2011), 1–44.

Digital Library

[33]

Sam Lau, Sruti Srinivasa Srinivasa Ragavan, Ken Milne, Titus Barik, and Advait Sarkar. 2021. TweakIt: Supporting End-User Programmers Who Transmogrify Code. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 311, 12 pages. https://doi.org/10.1145/3411764.3445265

Digital Library

[34]

John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50–80.

[35]

Sukwon Lee, Sung-Hee Kim, Ya-Hsin Hung, Heidi Lam, Youn-ah Kang, and Ji Soo Yi. 2015. How do people make sense of unfamiliar visualizations?: A grounded model of novice’s information visualization sensemaking. IEEE transactions on visualization and computer graphics 22, 1 (2015), 499–508.

[36]

Clayton Lewis and Cathleen Wharton. 1997. Cognitive walkthroughs. In Handbook of human-computer interaction. Elsevier, 717–732.

[37]

Xingjun Li, Yizhi Zhang, Justin Leung, Chengnian Sun, and Jian Zhao. 2023. EDAssistant: Supporting Exploratory Data Analysis in Computational Notebooks with In Situ Code Search and Recommendation. ACM Trans. Interact. Intell. Syst. 13, 1, Article 1 (mar 2023), 27 pages. https://doi.org/10.1145/3545995

Digital Library

[38]

Michael Xieyang Liu, Advait Sarkar, Carina Negreanu, Benjamin Zorn, Jack Williams, Neil Toronto, and Andrew D. Gordon. 2023. “What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 598, 31 pages. https://doi.org/10.1145/3544548.3580817

Digital Library

[39]

Deborah Lupton. 2016. The quantified self. John Wiley & Sons.

[40]

Mariana Mărăşoiu, Alan F Blackwell, Advait Sarkar, and Martin Spott. 2016. Clarifying hypotheses by sketching data. In Proceedings of the Eurographics/IEEE VGTC Conference on Visualization: Short Papers. 125–129.

[41]

Matt Mccutchen, Judith Borghouts, Andrew D Gordon, Simon Peyton Jones, and Advait Sarkar. 2020. Elastic sheet-defined functions: Generalising spreadsheet functions to variable-size input arrays. Journal of Functional Programming 30 (2020), e26.

[42]

Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1–23.

Digital Library

[43]

Andrew M McNutt, Chenglong Wang, Robert A Deline, and Steven M Drucker. 2023. On the design of ai-powered code assistants for notebooks. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.

Digital Library

[44]

Jesse Mu and Advait Sarkar. 2019. Do We Need Natural Language? Exploring Restricted Language Interfaces for Complex Domains. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312975

Digital Library

[45]

Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: literature review. Microsoft Research (2022).

[46]

Peter Pirolli and Stuart Card. 1999. Information foraging.Psychological review 106, 4 (1999), 643.

[47]

Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2–4.

[48]

James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. " It’s Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers. arXiv preprint arXiv:2304.02491 (2023).

[49]

Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic Prompt Optimization with "Gradient Descent" and Beam Search. In Conference on Empirical Methods in Natural Language Processing. https://api.semanticscholar.org/CorpusID:258546785

[50]

Gregg Rothermel, Lixin Li, Christopher DuPuis, and Margaret Burnett. 1998. What you see is what you test: A methodology for testing form-based visual programs. In Proceedings of the 20th international conference on Software engineering. IEEE, 198–207.

[51]

Daniel M Russell, Mark J Stefik, Peter Pirolli, and Stuart K Card. 1993. The cost structure of sensemaking. In Proceedings of the INTERACT’93 and CHI’93 conference on Human factors in computing systems. 269–276.

Digital Library

[52]

Advait Sarkar. 2016. Constructivist Design for Interactive Machine Learning. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (San Jose, California, USA) (CHI EA ’16). Association for Computing Machinery, New York, NY, USA, 1467–1475. https://doi.org/10.1145/2851581.2892547

Digital Library

[53]

Advait Sarkar. 2016. Interactive analytical modelling. Technical Report UCAM-CL-TR-920. University of Cambridge, Computer Laboratory. https://doi.org/10.48456/tr-920

[54]

Advait Sarkar. 2022. Is explainable AI a race against model complexity?. In Workshop on Transparency and Explanations in Smart Systems (TeXSS), in conjunction with ACM Intelligent User Interfaces (IUI 2022)(CEUR Workshop Proceedings, 3124). 192–199. http://ceur-ws.org/Vol-3124/paper22.pdf

[55]

Advait Sarkar. 2023. Enough With “Human-AI Collaboration”. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 415, 8 pages. https://doi.org/10.1145/3544549.3582735

Digital Library

[56]

Advait Sarkar. 2023. Exploring Perspectives on the Impact of Artificial Intelligence on the Creativity of Knowledge Work: Beyond Mechanised Plagiarism and Stochastic Parrots. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work (Oldenburg, Germany) (CHIWORK ’23). Association for Computing Machinery, New York, NY, USA, Article 13, 17 pages. https://doi.org/10.1145/3596671.3597650

Digital Library

[57]

Advait Sarkar. 2023. Should Computers Be Easy To Use? Questioning the Doctrine of Simplicity in User Interface Design. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA ’23). Association for Computing Machinery, New York, NY, USA, Article 419, 10 pages. https://doi.org/10.1145/3544549.3582741

Digital Library

[58]

Advait Sarkar. 2023. Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?. In Proceedings of the 2023 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Cascais, Portugal) (Onward! 2023). Association for Computing Machinery, New York, NY, USA, 153–167. https://doi.org/10.1145/3622758.3622882

Digital Library

[59]

Advait Sarkar. 2024. AI Should Challenge, Not Obey. Communications of the ACM (in press) (2024).

[60]

Advait Sarkar. 2024. Large Language Models Cannot Explain Themselves. In ACM CHI 2024 Workshop on Human-Centered Explainable AI (HCXAI).

[61]

Advait Sarkar, Alan F Blackwell, Mateia Jamnik, and Martin Spott. 2014. Teach and try: A simple interaction technique for exploratory data modelling by end users. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 53–56. https://doi.org/10.1109/VLHCC.2014.6883022

[62]

Advait Sarkar, Judith W. Borghouts, Anusha Iyer, Sneha Khullar, Christian Canton, Felienne Hermans, Andrew D. Gordon, and Jack Williams. 2020. Spreadsheet Use and Programming Experience: An Exploratory Survey. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3334480.3382807

Digital Library

[63]

Advait Sarkar, Ian Drosos, Rob Deline, Andrew D. Gordon, Carina Negreanu, Sean Rintel, Jack Williams, and Ben Zorn. 2023. Participatory prompting: a user-centric research method for eliciting AI assistance opportunities in knowledge workflows. In Proceedings of the 34th Annual Conference of the Psychology of Programming Interest Group (PPIG 2023).

[64]

Advait Sarkar and Andrew D. Gordon. 2018. How do people learn to use spreadsheets? (Work in progress). In Proceedings of the 29th Annual Conference of the Psychology of Programming Interest Group (PPIG 2018). 28–35.

[65]

Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with artificial intelligence?. In Proceedings of the 33rd Annual Conference of the Psychology of Programming Interest Group (PPIG 2022).

[66]

Advait Sarkar, Mateja Jamnik, Alan F. Blackwell, and Martin Spott. 2015. Interactive visual machine learning in spreadsheets. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 159–163. https://doi.org/10.1109/VLHCC.2015.7357211

[67]

Advait Sarkar, Sruti Srinivasa Ragavan, Jack Williams, and Andrew D. Gordon. 2022. End-user encounters with lambda abstraction in spreadsheets: Apollo’s bow or Achilles’ heel?. In 2022 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–11. https://doi.org/10.1109/VL/HCC53370.2022.9833131

[68]

Advait Sarkar, Martin Spott, Alan F. Blackwell, and Mateja Jamnik. 2016. Visual discovery and model-driven explanation of time series patterns. In 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 78–86. https://doi.org/10.1109/VLHCC.2016.7739668

[69]

Douglas Schuler and Aki Namioka. 1993. Participatory design: Principles and practices. CRC Press.

Digital Library

[70]

Sina J. Semnani, Violet Z. Yao, He Zhang, and Monica S. Lam. 2023. WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia. In Conference on Empirical Methods in Natural Language Processing. https://api.semanticscholar.org/CorpusID:258841157

[71]

Floarea Serban, Joaquin Vanschoren, Jörg-Uwe Kietz, and Abraham Bernstein. 2013. A survey of intelligent assistants for data analysis. ACM Computing Surveys (CSUR) 45, 3 (2013), 1–35.

Digital Library

[72]

Ben Shneiderman. 1983. Direct manipulation: A step beyond programming languages. Computer 16, 08 (1983), 57–69.

Digital Library

[73]

Divya Siddarth, Daron Acemoglu, Danielle Allen, Kate Crawford, James Evans, Michael Jordan, and E Weyl. 2021. How AI fails us. arXiv preprint arXiv:2201.04200 (2021).

[74]

Sruti Srinivasa Ragavan, Advait Sarkar, and Andrew D Gordon. 2021. Spreadsheet Comprehension: Guesswork, Giving Up and Going Back to the Author. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 181, 21 pages. https://doi.org/10.1145/3411764.3445634

Digital Library

[75]

Robert Stalnaker. 2002. Common ground. Linguistics and philosophy 25, 5/6 (2002), 701–721.

[76]

Lev Tankelevitch, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, Advait Sarkar, Abigail Sellen, and Sean Rintel. 2023. The Metacognitive Demands and Opportunities of Generative AI. arXiv preprint arXiv:2312.10893 (2023).

[77]

Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S Bernstein, and Ranjay Krishna. 2023. Explanations can reduce overreliance on ai systems during decision-making. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–38.

Digital Library

[78]

Dakuo Wang, Josh Andres, Justin D. Weisz, Erick Oduor, and Casey Dugan. 2021. AutoDS: Towards Human-Centered Automation of Data Science. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 79, 12 pages. https://doi.org/10.1145/3411764.3445526

Digital Library

[79]

Weixuan Wang, Barry Haddow, Alexandra Birch, and Wei Peng. 2023. Assessing the Reliability of Large Language Model Knowledge. ArXiv abs/2310.09820 (2023). https://api.semanticscholar.org/CorpusID:264146357

[80]

Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, and Tomas Pfister. 2024. Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding. ArXiv abs/2401.04398 (2024). https://api.semanticscholar.org/CorpusID:266899992

[81]

Justin D Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021. Perfection not required? Human-AI partnerships in code translation. In 26th International Conference on Intelligent User Interfaces. 402–412.

Digital Library

[82]

John Wenskovitch, Corey Fallon, Kate Miller, and Aritra Dasgupta. 2021. Beyond visual analytics: Human-machine teaming for ai-driven data sensemaking. In 2021 IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX). IEEE, 40–44.

[83]

Jack Williams, Carina Negreanu, Andrew D. Gordon, and Advait Sarkar. 2020. Understanding and Inferring Units in Spreadsheets. In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). 1–9. https://doi.org/10.1109/VL/HCC50065.2020.9127254

[84]

JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can’t prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.

Digital Library

Cited By

Kazemitabaar MWilliams JDrosos IGrossman THenley ANegreanu CSarkar A(2024)Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task DecompositionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676345(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676345
Sarkar A(2024)Diversity in Study Participants and a Critique of the “Representative Sample” in Human-Computer Interaction Research2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00062(394-399)Online publication date: 2-Sep-2024
https://doi.org/10.1109/VL/HCC60511.2024.00062

Index Terms

"It's like a rubber duck that talks back": Understanding Generative AI-Assisted Data Analysis Workflows through a Participatory Prompting Study

Recommendations

Layered participatory analysis: new developments in the CARD technique
CHI '01: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

CARD (Collaborative Analysis of Requirements and Design) is an influential technique for participatory design and participatory analysis that is in use on three continents. This paper reviews three case studies that document the development of a ...
Participatory Design: How to Engage Older Adults in Participatory Design Activities

Ongoing advances in mobile technologies have the potential to improve independence and quality of life of older adults by supporting the delivery of personalised and ubiquitous healthcare solutions. The authors are actively engaged in participatory, ...
Scalable script-based data analysis workflows on clouds
WORKS '13: Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science

Data analysis workflows are often composed by many concurrent and compute-intensive tasks that can be efficiently executed only on scalable computing infrastructures, such as HPC systems, Grids and Cloud platforms. The use of Cloud services for the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CHIWORK '24: Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work

June 2024

297 pages

ISBN:9798400710179

DOI:10.1145/3663384

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2024

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

CHIWORK 2024

CHIWORK 2024: Annual Symposium on Human-Computer Interaction for Work

June 25 - 27, 2024

Newcastle upon Tyne, United Kingdom

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
350
Total Downloads

Downloads (Last 12 months)350
Downloads (Last 6 weeks)75

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kazemitabaar MWilliams JDrosos IGrossman THenley ANegreanu CSarkar A(2024)Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task DecompositionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676345(1-19)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676345
Sarkar A(2024)Diversity in Study Participants and a Critique of the “Representative Sample” in Human-Computer Interaction Research2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00062(394-399)Online publication date: 2-Sep-2024
https://doi.org/10.1109/VL/HCC60511.2024.00062

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents