research-article

Open access

Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems

Authors:

C. Estelle Smith,

Anjali Srivastava,

Aaron Halfaker,

Haiyi ZhuAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 14

https://doi.org/10.1145/3313831.3376783

Published: 23 April 2020 Publication History

All formats PDF

Abstract

On Wikipedia, sophisticated algorithmic tools are used to assess the quality of edits and take corrective actions. However, algorithms can fail to solve the problems they were designed for if they conflict with the values of communities who use them. In this study, we take a Value-Sensitive Algorithm Design approach to understanding a community-created and -maintained machine learning-based algorithm called the Objective Revision Evaluation System (ORES)---a quality prediction system used in numerous Wikipedia applications and contexts. Five major values converged across stakeholder groups that ORES (and its dependent applications) should: (1) reduce the effort of community maintenance, (2) maintain human judgement as the final authority, (3) support differing peoples' differing workflows, (4) encourage positive engagement with diverse editor groups, and (5) establish trustworthiness of people and algorithms within the community. We reveal tensions between these values and discuss implications for future research to improve algorithms like ORES.

Supplementary Material

MP4 File (a654-smith-presentation.mp4)

Download
48.68 MB

References

[1]

2019a. Wikipedia: Please do not bite the newcomers. (May 2019). https://en.wikipedia.org/w/index.php?title=Wikipedia: Please_do_not_bite_the_newcomers&oldid=897507618 Page Version ID: 897507618.

[2]

2019b. Wikipedia:What Wikipedia is not. (May 2019). https://en.wikipedia.org/w/index.php?title=Wikipedia: What_Wikipedia_is_not&oldid=899044833 Page Version ID: 899044833.

[3]

Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. 2018. A Reductions Approach to Fair Classification. arXiv:1803.02453 [cs] (March 2018). http://arxiv.org/abs/1803.02453 arXiv: 1803.02453.

[4]

Marco Almada. 2019. Human intervention in automated decision-making: Toward the construction of contestable systems. In International Conference on Artificial Intelligence and Law (ICAIL'19).

Digital Library

[5]

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6541--6549.

[6]

Alan Borning and Michael Muller. 2012. Next Steps for Value Sensitive Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'12). ACM, NY, NY, USA, 1125--1134. event-place: Austin, Texas, USA.

Digital Library

[7]

Eshwar Chandrasekharan, Chaitrali Gandhi, Matthew Wortley Mustelier, and Eric Gilbert. 2019. Crossmod: A Cross-Community Learning-based System to Assist Reddit Moderators. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 174.

Digital Library

[8]

Hao-Fei Cheng, Ruotong Wang, Zheng Zhang, Fiona O'Connell, Terrance Gray, F. Maxwell Harper, and Haiyi Zhu. 2019. Explaining Decision-Making Algorithms Through UI: Strategies to Help Non-Expert Stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 559:1--559:12. event-place: Glasgow, Scotland Uk.

Digital Library

[9]

Alexandra Chouldechova and Aaron Roth. 2018. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810 (2018).

[10]

Charles T Clotfelter, Helen F Ladd, and Jacob L Vigdor. 2006. Teacher-student matching and the assessment of teacher effectiveness. Journal of human Resources 41, 4 (2006), 778--820.

[11]

Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 797--806.

Digital Library

[12]

Dan Cosley, Dan Frankowski, Loren Terveen, and John Riedl. 2007. SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia. In Proceedings of the 12th International Conference on Intelligent User Interfaces (IUI '07). ACM, NY, NY, USA, 32--41. event-place: Honolulu, Hawaii, USA.

Digital Library

[13]

Quang Vinh Dang and Claudia-Lavinia Ignat. 2016. Quality assessment of wikipedia articles without feature engineering. In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries. ACM, 27--30.

Digital Library

[14]

Michael A DeVito. 2017. From editors to algorithms: A values-based approach to understanding story selection in the Facebook news feed. Digital Journalism 5, 6 (2017), 753--773.

[15]

Michael A DeVito, Jeffrey T Hancock, Megan French, Jeremy Birnholtz, Judd Antin, Karrie Karahalios, Stephanie Tong, and Irina Shklovski. 2018. The algorithm and the user: How can HCI use lay understandings of algorithmic systems?. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, panel04.

Digital Library

[16]

Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).

[17]

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness Through Awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (ITCS '12). ACM, NY, NY, USA, 214--226. event-place: Cambridge, Massachusetts.

Digital Library

[18]

Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee Rickman, Kevin Hamilton, and Alex Kirlik. 2016. First i like it, then i hide it: Folk theories of social feeds. In Proceedings of the 2016 cHI conference on human factors in computing systems. ACM, 2371--2382.

Digital Library

[19]

Andrea Forte, Vanesa Larco, and Amy Bruckman. 2009. Decentralization in Wikipedia Governance. Journal of Management Information Systems 26, 1 (July 2009), 49--72.

Digital Library

[20]

Megan French and Jeff Hancock. 2017. What's the folk theory? Reasoning about cyber-social systems. Reasoning About Cyber-Social Systems (February 2, 2017) (2017).

[21]

Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth. 2019. A Comparative Study of Fairness-enhancing Interventions in Machine Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19). ACM, NY, NY, USA, 329--338. event-place: Atlanta, GA, USA.

Digital Library

[22]

Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. 2013. Value sensitive design and information systems. In Early engagement and new technologies: Opening up the laboratory. Springer, 55--95.

[23]

R. Stuart Geiger. 2011. The lives of bots. In Critical point of view: a Wikipedia reader, Geert Lovink and Nathaniel Tkacz (Eds.). Number 7 in INC Reader. Inst. of Network Cultures, Amsterdam, 78--93. http://stuartgeiger.com/lives-of-bots-wikipedia-cpov.pdf OCLC: 762034235.

[24]

R. Stuart Geiger. 2014. Bots, bespoke, code and the materiality of software platforms. Information, Communication & Society 17, 3 (March 2014), 342--356.

[25]

R Stuart Geiger. 2017. Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture. Big Data & Society 4, 2 (Dec. 2017), 2053951717730735.

[26]

R. Stuart Geiger and Aaron Halfaker. 2013. When the Levee Breaks: Without Bots, What Happens to Wikipedia's Quality Control Processes?. In Proceedings of the 9th International Symposium on Open Collaboration (WikiSym '13). ACM, NY, NY, USA, 6:1--6:6. event-place: Hong Kong, China.

Digital Library

[27]

R Stuart Geiger and Aaron Halfaker. 2016. Open algorithmic systems: lessons on opening the black box from Wikipedia. AoIR Selected Papers of Internet Research 6 (2016).

[28]

R. Stuart Geiger and Aaron Halfaker. 2017. Operationalizing Conflict and Cooperation Between Automated Software Agents in Wikipedia: A Replication and Expansion of 'Even Good Bots Fight'. Proc. ACM Hum.-Comput. Interact. 1, CSCW (Dec. 2017), 49:1--49:33.

Digital Library

[29]

Aaron Halfaker. 2017. Interpolating quality dynamics in Wikipedia and demonstrating the Keilana effect. In Proceedings of the 13th International Symposium on Open Collaboration. ACM, 19.

Digital Library

[30]

Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013. The rise and decline of an open collaboration system: How Wikipedia's reaction to popularity is causing its decline. American Behavioral Scientist 57, 5 (2013), 664--688.

[31]

Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, Amir Sarabadani, and Adam Wight. 2018. ORES: Facilitating re-mediation of Wikipedia's socio-technical problems. Computer-Supported Cooperative Work and Social Computing (CSCW) (2018).

[32]

Aaron Halfaker, Aniket Kittur, Robert Kraut, and John Riedl. 2009. A Jury of Your Peers: Quality, Experience and Ownership in Wikipedia. In Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym '09). ACM, NY, NY, USA, 15:1--15:10. event-place: Orlando, Florida.

Digital Library

[33]

Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't Bite the Newbies: How Reverts Affect the Quantity and Quality of Wikipedia Work. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration (WikiSym '11). ACM, NY, NY, USA, 163--172. event-place: Mountain View, California.

Digital Library

[34]

A. Halfaker and J. Riedl. 2012. Bots and Cyborgs: Wikipedia's Immune System. Computer 45, 3 (March 2012), 79--82.

Digital Library

[35]

Moritz Hardt, Eric Price, Nati Srebro, and others. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323.

[36]

Eszter Hargittai and Aaron Shaw. 2015. Mind the skills gap: the role of Internet know-how and gender in differentiated contributions to Wikipedia. Information, Communication & Society 18, 4 (April 2015), 424--442.

[37]

Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. 2016. Generating visual explanations. In European Conference on Computer Vision. Springer, 3--19.

[38]

Ibrahim Hooper. 2009. Coalition to Stop Cradle to Prison Algorithm Welcome Decision to Dissolve Joint Powers Agreement. (2009). https://www.cair.com/cair_ mn_coalition_to_stop_cradle_to_prison_algorithm_ welcome_decision_to_dissolve_joint_powers_agreement

[39]

Lilly C. Irani and M. Six Silberman. 2013. Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 611--620. event-place: Paris, France.

Digital Library

[40]

Terry Judd and Gregor Kennedy. 2010. A five-year study of on-campus Internet use by undergraduate biomedical students. Computers & Education 55, 4 (Dec. 2010), 1564--1571.

Digital Library

[41]

Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2019. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. ACM, 100--109.

Digital Library

[42]

Charles Kiene, Andrés Monroy-Hernández, and Benjamin Mako Hill. 2016. Surviving an eternal september: How an online community managed a surge of newcomers. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1152--1156.

Digital Library

[43]

Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 1885--1894.

Digital Library

[44]

Min Kyung Lee, Ji Tae Kim, and Leah Lizarondo. 2017. A Human-Centered Approach to Algorithmic Services: Considerations for Fair and Motivating Smart Community Service Management That Allocates Donations to Non-Profit Organizations. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 3365--3376. event-place: Denver, Colorado, USA.

Digital Library

[45]

Aditya Krishna Menon and Robert C Williamson. 2018. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency. 107--118.

[46]

Tim Miller. 2018. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence (2018).

[47]

Shira Mitchell, Eric Potash, Solon Barocas, Alexander D'Amour, and Kristian Lum. 2018. Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions. arXiv:1811.07867 [stat] (Nov. 2018). http://arxiv.org/abs/1811.07867 arXiv: 1811.07867.

[48]

Claudia Muller-Birn, Leonhard Dobusch, and James D. Herbsleb. 2013. Work-to-rule: The Emergence of Algorithmic Governance in Wikipedia. In Proceedings of the 6th International Conference on Communities and Technologies (C&T '13). ACM, NY, NY, USA, 80--89. event-place: Munich, Germany.

Digital Library

[49]

Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680--5689.

[50]

Reid Priedhorsky, Jilin Chen, Shyong (Tony) K. Lam, Katherine Panciera, Loren Terveen, and John Riedl. 2007. Creating, Destroying, and Restoring Value in Wikipedia. In Proceedings of the 2007 International ACM Conference on Supporting Group Work (GROUP '07). ACM, NY, NY, USA, 259--268. event-place: Sanibel Island, Florida, USA.

Digital Library

[51]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Model-agnostic interpretability of machine learning. arXiv preprint arXiv:1606.05386 (2016).

[52]

Joseph Seering, Tony Wang, Jina Yoon, and Geoff Kaufman. 2019. Moderator engagement and community development in the age of algorithms. New Media & Society 21, 7 (July 2019), 1417--1443.

[53]

Irving Seidman. 2006. Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences. Teachers College Press.

[54]

Tom Simonite. 2013. The decline of Wikipedia. Technology Review 116, 6 (2013), 50--56.

[55]

Una Titz. 2018. Facebook, AI censorship & content moderation. (May 2018). https://medium.com/beluga-team/ facebook-ai-censorship-content-moderation-5205d30f5063

[56]

Michael Veale, Max Van Kleek, and Reuben Binns. 2018. Fairness and Accountability Design Needs for Algorithmic Support in High-Stakes Public Sector Decision-Making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, 440:1--440:14. event-place: Montreal QC, Canada.

Digital Library

[57]

Nicholas Vincent, Isaac Johnson, and Brent Hecht. 2018. Examining Wikipedia with a broader lens: Quantifying the value of Wikipedia's relationships with other large-scale online communities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 566.

Digital Library

[58]

Lauren Weber and RE Silverman. 2012. Your resume vs. oblivion. The Wall Street Journal 24 (2012).

[59]

Wikipedia. 2019. Help:New filters for edit review/Quality and Intent Filters - MediaWiki. (2019). https://www.mediawiki.org/wiki/Help:New_filters_for_ edit_review/Quality_and_Intent_Filters

[60]

Wikipedia. 2019. ORES - MediaWiki. (2019). https://www.mediawiki.org/wiki/ORES

[61]

Wikipedia. 2019a. ORES/Applications - MediaWiki. (2019). https://www.mediawiki.org/wiki/ORES/Applications

[62]

Wikipedia. 2019b. Page Curation - MediaWiki. (2019). https://www.mediawiki.org/wiki/Page_Curation

[63]

Wikipedia. 2019c. Research:Applying Value-Sensitive Algorithm Design to ORES - Meta. (2019). https://meta.wikimedia.org/wiki/Research: Applying_Value-Sensitive_Algorithm_Design_to_ORES

[64]

Wikipedia. 2019d. User:SuggestBot. (Feb. 2019). https://en.wikipedia.org/w/index.php?title=User:SuggestBot&oldid=881992394 Page Version ID: 881992394.

[65]

Wikipedia. 2019e. Wikimedia Scoring Platform team MediaWiki. (2019). https://www.mediawiki.org/wiki/Wikimedia_Scoring_Platform_team

Cited By

Kallina ESingh J(2024)Stakeholder Involvement for Responsible AI Development: A Process FrameworkProceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization10.1145/3689904.3694698(1-14)Online publication date: 29-Oct-2024
https://dl.acm.org/doi/10.1145/3689904.3694698
Kawakami ACoston AHeidari HHolstein KZhu H(2024)Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and UseProceedings of the ACM on Human-Computer Interaction10.1145/36869898:CSCW2(1-24)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686989
Chauhan ASarkar DAgrawaal TSoden R(2024)Value Tensions in OpenStreetMap: Openness, Membership, and Policy in Online CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/36869198:CSCW2(1-25)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686919
Show More Cited By

Index Terms

Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems
1. Human-centered computing

Recommendations

Predicting Member Productivity and Withdrawal from Pre-Joining Attachments in Online Production Groups
CSCW '17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing

Productive and dedicated members are critical to the success of online production communities like Wikipedia. Many communities organize in subgroups where members voluntarily work on projects of shared interest. In this paper, we investigate how members'...
Out With The Old, In With The New?: Unpacking Member Turnover in Online Production Groups

Nearly any group is subject to turnover : some people leave, while others join. Turnover can be especially high in online groups, since participation typically is strictly voluntary. We investigated the effects of member turnover in online groups, ...
Examining Wikipedia With a Broader Lens: Quantifying the Value of Wikipedia's Relationships with Other Large-Scale Online Communities
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

The extensive Wikipedia literature has largely considered Wikipedia in isolation, outside of the context of its broader Internet ecosystem. Very recent research has demonstrated the significance of this limitation, identifying critical relationships ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
2,776
Total Downloads

Downloads (Last 12 months)609
Downloads (Last 6 weeks)71

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kallina ESingh J(2024)Stakeholder Involvement for Responsible AI Development: A Process FrameworkProceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization10.1145/3689904.3694698(1-14)Online publication date: 29-Oct-2024
https://dl.acm.org/doi/10.1145/3689904.3694698
Kawakami ACoston AHeidari HHolstein KZhu H(2024)Studying Up Public Sector AI: How Networks of Power Relations Shape Agency Decisions Around AI Design and UseProceedings of the ACM on Human-Computer Interaction10.1145/36869898:CSCW2(1-24)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686989
Chauhan ASarkar DAgrawaal TSoden R(2024)Value Tensions in OpenStreetMap: Openness, Membership, and Policy in Online CommunitiesProceedings of the ACM on Human-Computer Interaction10.1145/36869198:CSCW2(1-25)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686919
Berkel NPohl H(2024)Collaborating with Bots and Automation on OpenStreetMapACM Transactions on Computer-Human Interaction10.1145/366532631:3(1-30)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3665326
Yoo DWoo HPendse SLu NBirnbaum MAbowd GDe Choudhury M(2024)Missed Opportunities for Human-Centered AI Research: Understanding Stakeholder Collaboration in Mental Health AI ResearchProceedings of the ACM on Human-Computer Interaction10.1145/36373728:CSCW1(1-24)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637372
Prinster GSmith CTan CKeegan B(2024)Community Archetypes: An Empirical Framework for Guiding Research Methodologies to Reflect User Experiences of Sense of Virtual Community on RedditProceedings of the ACM on Human-Computer Interaction10.1145/36373108:CSCW1(1-33)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637310
Chancellor SKonstan JTerveen LYarosh S(2024)Do Know Harm: Considering the Ethics of Online Community ResearchCommunications of the ACM10.1145/363530367:6(35-38)Online publication date: 23-May-2024
https://doi.org/10.1145/3635303
Kuo THalfaker ACheng ZKim JWu MWu THolstein KZhu H(2024)Wikibench: Community-Driven Data Curation for AI Evaluation on WikipediaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642278(1-24)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642278
Sadek MMougenot C(2024)Challenges in Value-Sensitive AI Design: Insights from AI Practitioner InterviewsInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2439021(1-18)Online publication date: 18-Dec-2024
https://doi.org/10.1080/10447318.2024.2439021
Chen QCai JJacucci G(2024)“People are Way too Obsessed with Rank”: Trust System in Social Virtual RealityComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09498-7Online publication date: 4-May-2024
https://doi.org/10.1007/s10606-024-09498-7
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents