research-article

Open access

Methodological Middle Spaces: Addressing the Need for Methodological Innovation to Achieve Simultaneous Realism, Control, and Scalability in Experimental Studies of AI-Mediated Communication

Authors:

Zhila Aghajari,

Eric P. S. Baumer,

Jess Hohenstein,

Dominic DiFranzoAuthors Info & Claims

Proceedings of the ACM on Human-Computer Interaction, Volume 7, Issue CSCW1

Article No.: 73, Pages 1 - 28

https://doi.org/10.1145/3579506

Published: 16 April 2023 Publication History

Abstract

As AI-mediated communication (AI-MC) becomes more prevalent in everyday interactions, it becomes increasingly important to develop a rigorous understanding of its effects on interpersonal relationships and on society at large. Controlled experimental studies offer a key means of developing such an understanding, but various complexities make it difficult for experimental AI-MC research to simultaneously achieve the criteria of experimental realism, experimental control, and scalability. After outlining these methodological challenges, this paper offers the concept of methodological middle spaces as a means to address these challenges. This concept suggests that the key to simultaneously achieving all three of these criteria is to abandon the perfect attainment of any single criterion. This concept's utility is demonstrated via its use to guide the design of a platform for conducting text-based AI-MC experiments. Through a series of three example studies, the paper illustrates how the concept of methodological middle spaces can inform the design of specific experimental methods. Doing so enabled these studies to examine research questions that would have been either difficult or impossible to investigate using existing approaches. The paper concludes by describing how future research could similarly apply the concept of methodological middle spaces to expand methodological possibilities for AI-MC research in ways that enable contributions not currently possible.

References

[1]

Omar Al-Ubaydli, John A List, and Dana L Suskind. 2017. What can we learn from experiments? Understanding the threats to the scalability of experimental results. American Economic Review, Vol. 107, 5 (2017), 282--86.

[2]

Aloha Hufana Ambe, Margot Brereton, Alessandro Soro, Laurie Buys, and Paul Roe. 2019. The Adventures of Older Authors : Exploring Futures through Co-Design Fictions. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ). ACM, Glasgow, 358. https://doi.org/10.1145/3290605.3300588

Digital Library

[3]

Kenneth C Arnold, Krysta Chauncey, and Krzysztof Z Gajos. 2018. Sentiment Bias in Predictive Text Recommendations Results in Biased Writing. In Graphics Interface. 42--49.

[4]

Arthur Aron, Elaine N Aron, and Danny Smollan. 1992. Inclusion of other in the self scale and the structure of interpersonal closeness. Journal of personality and social psychology, Vol. 63, 4 (1992), 596.

[5]

Elliot Aronson, Marylinn B. Brewer, and J. Merrill Carlsmith. 1985. Experimentation in Social Psychology. In Handbook of Social Psychology third edition ed.). Random House, New York, 441--486.

[6]

Susan B. Barnes. 2006. A Privacy Paradox : Social Networking in the United States. First Monday, Vol. 11, 9 (2006).

[7]

Eric P. S. Baumer, David Mimno, Shion Guha, Emily Quan, and Geri K. Gay. 2017. Comparing Grounded Theory and Topic Modeling: Extreme Divergence or Unlikely Convergence? Journal of the Association for Information Science and Technology (JASIST), Vol. 68, 6 (June 2017), 1397--1410. https://doi.org/10.1002/asi.23786

Digital Library

[8]

Michael S. Bernstein, Bongwon Suh, Lichan Hong, Jilin Chen, Sanjay Kairam, and Ed H. Chi. 2010. Eddi: Interactive Topic-based Browsing of Social Status Streams. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST ). ACM, New York, 303--312. https://doi.org/10.1145/1866029.1866077

Digital Library

[9]

Aparajita Bhandari, Marie Ozanne, Natalya N Bazarova, and Dominic DiFranzo. 2021. Do You Care Who Flagged This Post? Effects of Moderator Visibility on Bystander Behavior. Journal of Computer-Mediated Communication, Vol. 26, 5 (2021), 284--300.

[10]

Anol Bhattacherjee. 2012. Social science research: Principles, methods, and practices. (2012).

[11]

Oloff C Biermann, Ning F Ma, and Dongwook Yoon. 2022. From Tool to Companion: Storywriters Want AI Writers to Respect Their Personal Values and Writing Strategies. In Designing Interactive Systems Conference. 1209--1227.

Digital Library

[12]

Mark Blythe. 2014. Research through design fiction: narrative in real and imaginary abstracts. In Proceedings of the SIGCHI conference on human factors in computing systems. 703--712.

Digital Library

[13]

Kirsten Boehner, Rogério DePaula, Paul Dourish, and Phoebe Sengers. 2007. How emotion is made and measured. International Journal of Human-Computer Studies, Vol. 65, 4 (2007), 275--291.

Digital Library

[14]

Boomerang. (2018). Write Better Email. https://www.boomeranggmail.com/respondable/. (Accessed on 10/05/2020).

[15]

Tommy Bruzzese, Irena Gao, Griffin Dietz, Christina Ding, and Alyssa Romanos. 2020. Effect of Confidence Indicators on Trust in AI-Generated Profiles. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1--8.

[16]

Stephen Cave, Kanta Dihal, and Sarah Dillon. 2020. AI Narratives : A History of Imaginative Thinking about Intelligent Machines. Oxford University Press, Oxford.

[17]

Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL ). ACL, Sofia, Bulgaria, 250--259.

[18]

Dominic DiFranzo, Samuel Hardman Taylor, Franccesca Kazerooni, Olivia D Wherry, and Natalya N Bazarova. 2018. Upstanding by design: Bystander intervention in cyberbullying. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--12.

Digital Library

[19]

Steven P. Dow, Manish Mehta, Blair MacIntyre, and Michael Mateas. 2010. Eliza Meets the Wizard -of-Oz: Blending Machine and Human Control of Embodied Characters. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ). ACM, Atlanta, GA, 547--556. https://doi.org/10.1145/1753326.1753408

Digital Library

[20]

Jamie Druckman. 2022. Experimental Thinking : A Primer on Social Science Experiments. Cambridge University Press, Cambridge.

[21]

Wen Duan, Naomi Yamashita, Yoshinari Shirai, and Susan R Fussell. 2021. Bridging Fluency Disparity between Native and Nonnative Speakers in Multilingual Multiparty Collaboration Using a Clarification Agent. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW2 (2021), 1--31.

Digital Library

[22]

Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. "I Always Assumed That I Wasn 't Really That Close to [Her ]": Reasoning About Invisible Algorithms in News Feeds. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ). ACM, Seoul, 153--162. https://doi.org/10.1145/2702123.2702556

Digital Library

[23]

Lauren Figueredo and Connie K Varnhagen. 2005. Didn't you run the spell checker? Effects of type of spelling error and use of a spell checker on perceptions of the author. Reading Psychology, Vol. 26, 4--5 (2005), 441--458.

[24]

Amit Fulay. 2016. Say hello to google allo: A smarter messaging app. https://blog.google/products/allo/google-allo-smarter-messaging-app/

[25]

Amy L Gonzales and Jeffrey T Hancock. 2008. Identity shift in computer-mediated environments. Media Psychology, Vol. 11, 2 (2008), 167--185.

[26]

Google. 2020. Smart Reply. https://developers.google.com/ml-kit/language/smart-reply Accessed: 2020-02--25.

[27]

Russell Haines and Joan Ellen Cheney Mann. 2011. A new perspective on de-individuation via computer-mediated communication. European Journal of Information Systems, Vol. 20, 2 (2011), 156--167.

[28]

Kevin Hamilton, Karrie Karahalios, Christian Sandvig, and Motahhare Eslami. 2014. A Path to Understanding the Effects of Algorithm Awareness. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems (CHI EA ) (CHI EA ). ACM, Toronto, ON, 631--642. https://doi.org/10.1145/2559206.2578883

Digital Library

[29]

Jeffrey T Hancock, Mor Naaman, and Karen Levy. 2020. AI-Mediated Communication: Definition, Research Agenda, and Ethical Considerations. Journal of Computer-Mediated Communication, Vol. 25, 1 (2020), 89--100.

[30]

Susan C Herring. 2002. Computer-mediated communication on the Internet. Annual review of information science and technology, Vol. 36, 1 (2002), 109--168.

[31]

Susan C Herring. 2007. Language and the Internet. Language@ Internet, Vol. 4, 1 (2007).

[32]

Erin R. Hoffman, David W. McDonald, and Mark Zachry. 2017. Evaluating a Computational Approach to Labeling Politeness : Challenges for the Application of Machine Classification to Social Computing Data. Proc. ACM Hum.-Comput. Interact., Vol. 1, CSCW (Dec. 2017), 52:1--52:14. https://doi.org/10.1145/3134687

Digital Library

[33]

Jess Hohenstein, Dominic DiFranzo, Rene F. Kizilcec, Zhila Aghajari, Hannah Mieczkowski, Karen Levy, Mor Naaman, Jeff Hancock, and Malte Jung. 2021. Artificial intelligence in communication impacts language and social relationships. arxiv: 2102.05756 [cs.HC]

[34]

Jess Hohenstein and Malte Jung. 2018. AI-supported messaging: An investigation of human-human text conversation with AI support. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. 1--6.

Digital Library

[35]

Jess Hohenstein and Malte Jung. 2020. AI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust. Computers in Human Behavior, Vol. 106 (2020), 106190.

Digital Library

[36]

Jess Hohenstein, Lindsay E. Larson, Yoyo Tsung-Yu Hou, Alexa M. Harris, Aaron Schecter, Leslie Dechurch, Noshir Contractor, and Malte F. Jung. 2022. Vero: A Method for Remotely Studying Human-AI Collaboration. In Hawaii International Conference on System Sciences (HICSS ). Honolulu, HI, 254--263.

[37]

Maurice Jakesch, Megan French, Xiao Ma, Jeffrey T Hancock, and Mor Naaman. 2019. AI-mediated communication: How the perception that profile text was written by AI affects trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[38]

Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young, et al. 2016. Smart reply: Automated response suggestion for email. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 955--964.

Digital Library

[39]

J. F. Kelley. 1983. An Empirical Methodology for Writing User-Friendly Natural Language Computer Applications. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ). ACM, Boston, MA, 193--196. https://doi.org/10.1145/800045.801609

Digital Library

[40]

J. F. Kelley. 1984. An Iterative Design Methodology for User-Friendly Natural Language Office Information Applications. ACM Transactions on Information Systems, Vol. 2, 1 (Jan. 1984), 26--41. https://doi.org/10.1145/357417.357420

Digital Library

[41]

Soomin Kim, Jinsu Eun, Joseph Seering, and Joonhwan Lee. 2021. Moderator Chatbot for Deliberative Discussion: Effects of Discussion Structure and Discussant Facilitation. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--26.

Digital Library

[42]

Scott R. Klemmer, Anoop K. Sinha, Jack Chen, James A. Landay, Nadeem Aboobaker, and Annie Wang. 2000. Suede: A Wizard of Oz Prototyping Tool for Speech User Interfaces. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST ) (UIST ). ACM, San Diego, CA, 1--10. https://doi.org/10.1145/354401.354406

Digital Library

[43]

Peter M Krafft, Michael Macy, and Alex" Sandy" Pentland. 2017. Bots as virtual confederates: design and ethics. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 183--190.

Digital Library

[44]

Kathy J Kuipers and Stuart J Hysom. 2014. Common problems and solutions in experiments. In Laboratory experiments in the social sciences. Elsevier, 145--177.

[45]

Hui Li, Nan Chen, Minjuan Zhou, Chenyi He, Jingbo Li, and Yujie Shi. 2019. How People Browse Mobile News Feed? A Study for Mobile News Feed Design. In International Conference on Human-Computer Interaction. Springer, 248--265.

Digital Library

[46]

Wei Fang Lin, Yi Cheng Lin, Chin Lan Huang, and Lung Hung Chen. 2016. We Can Make It Better: ?We" Moderates the Relationship Between a Compromising Style in Interpersonal Conflict and Well-Being. Journal of Happiness Studies, Vol. 17, 1 (2016), 41--57. https://doi.org/10.1007/s10902-014--9582--8

[47]

Rhema Linder, Chase Hunter, Jacob McLemore, Senjuti Dutta, Fatema Akbar, Ted Grover, Thomas Breideband, Judith W Borghouts, Yuwen Lu, Gloria Mark, et al. 2022. Characterizing work-life for information work on mars: A design fiction for the new future of work on earth. Proceedings of the ACM on Human-Computer Interaction, Vol. 6, GROUP (2022), 1--27.

Digital Library

[48]

Gardner Ed Lindzey and Elliot Ed Aronson. 1968. The handbook of social psychology. (1968).

[49]

Yihe Liu, Anushk Mittal, Diyi Yang, and Amy Bruckman. 2022. Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing. In CHI Conference on Human Factors in Computing Systems. 1--13.

[50]

Philipp K Masur, Dominic James DiFranzo, and Natalya Natalie Bazarova. 2021. Behavioral Contagion on Social Media: Effects of Social Norms, Design Interventions, and Critical Media Literacy on Self-Disclosure. (2021).

[51]

Scott E Maxwell. 2004. The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychological methods, Vol. 9, 2 (2004), 147.

[52]

Hannah Mieczkowski, Jeffrey T Hancock, Mor Naaman, Malte Jung, and Jess Hohenstein. 2021. AI-Mediated Communication: Language Use and Interpersonal Effects in a Referential Communication Task. Proceedings of the ACM on Human-Computer Interaction, Vol. 5, CSCW1 (2021), 1--14.

Digital Library

[53]

Douglas G Mook. 1983. In Defense of External Invalidity. American Psychologist, Vol. 38 (1983), 379--387.

[54]

Michael Muller, Shion Guha, Eric P. S. Baumer, David Mimno, and N. Sadat Shami. 2016. Machine Learning and Grounded Theory Method : Convergence, Divergence, and Combination. In Proceedings of the ACM Conference on Supporting Group Work (GROUP ). Sanibel Island, FL.

[55]

Larissa Vivian N"agele, Merja Ryöppy, and Danielle Wilde. 2018. PDFi : Participatory Design Fiction with Vulnerable Users. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction (NordiCHI '18). ACM, Oslo, Norway, 819--831. https://doi.org/10.1145/3240167.3240272

Digital Library

[56]

Hoyeon Nam, Hankyung Kim, and Youn-kyung Lim. 2021. User Experience of Agent-Mediated Interactions with Multiple Conversational Agents. In HCI International 2021 - Posters, Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa (Eds.). Springer International Publishing, Cham, 472--476.

[57]

Renee Noortman, Britta F. Schulte, Paul Marshall, Saskia Bakker, and Anna L. Cox. 2019. HawkEye - Deploying a Design Fiction Probe. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, Glasgow, Scotland Uk, 1--14. https://doi.org/10.1145/3290605.3300652

Digital Library

[58]

Patricia A. Norberg, Daniel R. Horne, and David A. Horne. 2007. The Privacy Paradox: Personal Information Disclosure Intentions versus Behaviors. Journal of Consumer Affairs, Vol. 41, 1 (2007), 100--126.

[59]

Sladjana Nørskov, Malene F Damholdt, John P Ulhøi, Morten B Jensen, Charles Ess, and Johanna Seibt. 2020. Applicant fairness perceptions of a robot-mediated job interview: a video vignette-based experimental survey. Frontiers in Robotics and AI, Vol. 7 (2020), 586263.

[60]

Judith S Olson and Wendy A Kellogg. 2014. Ways of Knowing in HCI. Vol. 2. Springer.

[61]

W. J. Ong. 1982. Orality and literacy: the technologizing of the word. London. Methuen.

[62]

Martin T Orne. 1962. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American psychologist, Vol. 17, 11 (1962), 776.

[63]

Leysia Palen and Paul Dourish. 2003. Unpacking" privacy" for a networked world. In Proceedings of the SIGCHI conference on Human factors in computing systems. 129--136.

Digital Library

[64]

Shinyoung Park, Akira Harada, and Hiroya Igarashi. 2006. Influences of personal preference on product usability. In CHI'06 Extended Abstracts on Human Factors in Computing Systems. 87--92.

Digital Library

[65]

Souneil Park, Seungwoo Kang, Sangyoung Chung, and Junehwa Song. 2009. NewsCube : Delivering Multiple Aspects of News to Mitigate Media Bias. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ) (CHI ). ACM, Boston, 443--452. https://doi.org/10.1145/1518701.1518772

Digital Library

[66]

Seema Patel, William Bosley, David Culyba, Sabrina A. Haskell, Andrew Hosmer, TJ Jackson, Shane J. M. Liesegang, Peter Stepniewicz, James Valenti, Salim Zayat, and Brenda Harger. 2006. A Guided Performance Interface for Augmenting Social Experiences with an Interactive Animatronic Character. In Proceedings of the AAAI Conference on Artificial Intelligence in Interactive Digital Entertainment (AIIDE ). AAAI, Marina del Rey, CA, 72--79.

[67]

James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates, Vol. 71, 2001 (2001), 2001.

[68]

Kraus Rachel. (2018). Gmail smart replies may be creepy, but they're catching on like wildfire. ( (2018)). (Accessed on 10/05/2020).

[69]

Hans Radder. 2003. The philosophy of scientific experimentation. University of Pittsburgh Pre.

[70]

Sumita Sharma, Netta Iivari, Marianne Kinnula, Grace Eden, Alipta Ballav, Rocio Fatas, Ritwik Kar, Deepak Ranjan Padhi, Vahid Sadeghie, Pratiti Sarkar, Riya Sinha, Rucha Tulaskar, and Nikita Valluri. 2021. From Mild to Wild : Reimagining Friendships and Romance in the Time of Pandemic Using Design Fiction. In Designing Interactive Systems Conference 2021 (DIS '21). Association for Computing Machinery, New York, NY, USA, 64--77. https://doi.org/10.1145/3461778.3462110

Digital Library

[71]

Paschal Sheeran. 2002. Intention-behavior relations: a conceptual and empirical review. European review of social psychology, Vol. 12, 1 (2002), 1--36.

[72]

Dilruba Showkat and Eric P. S. Baumer. 2022. “It 's Like the Value System in the Loop ”: Domain Experts ' Values Expectations for NLP Automation. In Proceedings of the ACM Conference on Designing Interactive Systems (DIS ) (DIS ). ACM, Virtual Conference, 100--122. https://doi.org/10.1145/3532106.3533483

Digital Library

[73]

Choon-Ling Sia, Bernard CY Tan, and Kwok-Kee Wei. 2002. Group polarization and computer-mediated communication: Effects of communication cues, social presence, and anonymity. Information Systems Research, Vol. 13, 1 (2002), 70--90.

Digital Library

[74]

Nick Statt. 2018. Google now says controversial AI voice calling system will identify itself to humans. https://www.theverge.com/2018/5/10/17342414/google-duplex-ai-assistant-voice-calling-identify-itself-update. (Accessed on 10/05/2020).

[75]

Kristin Stecher and Scott Counts. 2008. Thin Slices of Online Profile Attributes. In Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM ). AAAI, Seattle, WA, 127--135.

[76]

Supasorn Suwajanakorn, Steven M Seitz, and Ira Kemelmacher-Shlizerman. 2017. Synthesizing obama: learning lip sync from audio. ACM Transactions on Graphics (TOG), Vol. 36, 4 (2017), 1--13.

Digital Library

[77]

Samuel Hardman Taylor, Dominic DiFranzo, Yoon Hyung Choi, Shruti Sannon, and Natalya N Bazarova. 2019. Accountability and Empathy by Design: Encouraging Bystander Intervention to Cyberbullying on Social Media. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--26.

[78]

Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2016. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2387--2395.

Digital Library

[79]

Crispin Thurlow, Laura Lengel, and Alice Tomic. 2004. Computer mediated communication. Sage.

[80]

Stephanie Tong, Joseph B Walther, et al. 2011. Relational maintenance and CMC. Computer-mediated communication in personal relationships, Vol. 53, 9 (2011), 1689--1699.

[81]

Amy Voida, Stephen Voida, Saul Greenberg, and Helen Ai He. 2008. Asymmetry in Media Spaces. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW ). ACM, San Diego, CA, 313--322. https://doi.org/10.1145/1460563.1460615

Digital Library

[82]

T Franklin Waddell. 2018. A robot wrote this? How perceived machine authorship affects news credibility. Digital journalism, Vol. 6, 2 (2018), 236--255.

[83]

Lisa Slattery Walker. 2014. Developing your experiment. In Laboratory experiments in the social sciences. Elsevier, 127--144.

[84]

Joseph B Walther and Malcolm R Parks. 2002. Cues filtered out, cues filtered in. Handbook of interpersonal communication, Vol. 3 (2002), 529--563.

[85]

Joseph Weizenbaum. 1966. ELIZA --a Computer Program for the Study of Natural Language Communication between Man and Machine. Commun. ACM, Vol. 9, 1 (Jan. 1966), 36--45. https://doi.org/10.1145/365153.365168

Digital Library

[86]

Angelina Widener and Sohye Lim. 2020. Need to belong, privacy concerns and self-disclosure in AI chatbot interaction. Journal of Digital Contents Society, Vol. 21, 12 (2020), 2203--2210.

[87]

Jerry S Wiggins, Paul Trapnell, and Norman Phillips. 1988. Psychometric and geometric characteristics of the Revised Interpersonal Adjective Scales (IAS-R). Multivariate Behavioral Research, Vol. 23, 4 (1988), 517--530.

[88]

Lotte M Willemsen, Peter C Neijens, and Fred Bronner. 2012. The ironic effect of source identification on the perceived credibility of online product reviewers. Journal of Computer-Mediated Communication, Vol. 18, 1 (2012), 16--31.

Digital Library

[89]

Richmond Y. Wong, Deirdre K. Mulligan, Ellen Van Wyk, James Pierce, and John Chuang. 2017. Eliciting Values Reflections by Engaging Privacy Futures Using Design Workbooks. Proc. ACM Hum.-Comput. Interact., Vol. 1, CSCW (Dec. 2017), 111:1--111:26. https://doi.org/10.1145/3134746

Digital Library

[90]

Nancy Viola Wuenderlich and Stefanie Paluch. 2017. A nice and friendly chat with a bot: User perceptions of AI-based service agents. (2017).

[91]

Bin Xu, Ge Gao, Susan R Fussell, and Dan Cosley. 2014. Improving machine translation by showing two outputs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3743--3746.

Digital Library

[92]

Elana. Zeide. 2015. Algorithms Can Be Lousy Fortunetellers. https://slate.com/technology/2015/05/crystal-app-algorithmic-fortunetelling-for-employers-and-potential-customers.html. (Accessed on 10/05/2020).

[93]

Justine Zhang, Jonathan P Chang, Cristian Danescu-Niculescu-Mizil, Lucas Dixon, Yiqing Hua, Nithum Thain, and Dario Taraborelli. 2018. Conversations gone awry: Detecting early signs of conversational failure. arXiv preprint arXiv:1805.05345 (2018).

Cited By

Agha ZPark JWan RAli NWang YDifranzo DBadillo-Urquiola KWisniewski P(2024)Tricky vs. Transparent: Towards an Ecologically Valid and Safe Approach for Evaluating Online Safety Nudges for TeensProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642313(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642313
Aghajari ZBaumer EDiFranzo D(2023)What’s the Norm Around Here? Individuals’ Responses Can Mitigate the Effects of Misinformation Prevalence in Shaping Perceptions of a CommunityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580946(1-27)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580946

Index Terms

Methodological Middle Spaces: Addressing the Need for Methodological Innovation to Achieve Simultaneous Realism, Control, and Scalability in Experimental Studies of AI-Mediated Communication
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Towards an integrated methodological framework for understanding embodiment in HCI
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing Systems

The third wave in HCI reveals how embodiment matters in post-WIMP computing systems. Yet it is still unclear what methods provide effective insight into the nature of embodiment in HCI in relation to both design and use. This paper presents work in ...
Methodological Review: Participatory design for computerization of clinical practice guidelines

There have been made many attempts on computerization of clinical practice guidelines (CPGs), none have, however achieved any general application in clinical work practice. The objective of this paper is: (1) to raise awareness about the impact the ...
Methodological implications of critical realism for mixed-methods research

Building on recent developments in mixed methods, we discuss the methodological implications of critical realism and explore how these can guide dynamic mixed-methods research design in information systems. Specifically, we examine the core ontological ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Human-Computer Interaction

Proceedings of the ACM on Human-Computer Interaction Volume 7, Issue CSCW1

CSCW

April 2023

3836 pages

EISSN:2573-0142

DOI:10.1145/3593053

Editor:
Jeff Nichols
Google

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 April 2023

Published in PACMHCI Volume 7, Issue CSCW1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the US Army Research Lab

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
575
Total Downloads

Downloads (Last 12 months)399
Downloads (Last 6 weeks)69

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Agha ZPark JWan RAli NWang YDifranzo DBadillo-Urquiola KWisniewski P(2024)Tricky vs. Transparent: Towards an Ecologically Valid and Safe Approach for Evaluating Online Safety Nudges for TeensProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642313(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642313
Aghajari ZBaumer EDiFranzo D(2023)What’s the Norm Around Here? Individuals’ Responses Can Mitigate the Effects of Misinformation Prevalence in Shaping Perceptions of a CommunityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580946(1-27)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580946

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents