Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2441776.2441920acmconferencesArticle/Chapter ViewAbstractPublication PagescscwConference Proceedingsconference-collections
research-article

Crowd vs. crowd: large-scale cooperative design through open team competition

Published: 23 February 2013 Publication History

Abstract

Following the recent remarkable successes of crowdsourcing, there have been attempts to apply it to design. However a design problem is often too complex and difficult to break down into simpler, distributable tasks as required by the conventional crowdsourcing model. In this paper, we present Crowd vs. Crowd (CvC), a novel design crowdsourcing method, where several design teams made up of designers and crowd compete with each other. In each team, a designer coordinates effective communication between the crowd members and takes responsibility for the final design output, and the crowd contributes at different stages of design. We conducted an initial evaluation of CvC in comparison with other collaborative design methods, and found that: CvC can attract more people to participate; the crowd can make useful contribution in CvC; CvC can produce competent design outputs. We then applied CvC to two real-life design problems: first, designing a new logo for a university department; second, for a small tech company. With quantitative and qualitative analyses on these applications, we observed that the elements of competition and collaboration helped to sustain the crowd's motivation to participate, and to produce quality design outcomes with higher level of satisfaction for the stakeholders.

References

[1]
Be a Martian, http://beamartian.jpl.nasa.gov/
[2]
Boudreau, K., Lacetera, N., and Lakhani, K. (2011). Incentives and problem uncertainty in innovation contests: an empirical analysis. Management Science 57, 5, 843--863.
[3]
Brandes, U., Stich, S., and Wender, M. (2008). Design by Use: The Everyday Metamorphosis of Things, Birkhäuser.
[4]
Buxton, B. (2007). Sketching User Experiences: Getting the Design Right and the Right Design, Morgan Kaufmann.
[5]
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., and Popovi, Z. (2010). Predicting protein structures with a multiplayer online game. Nature 466, 7307, 756--760.
[6]
Cranshaw, J. and Kittur, A. (2011). The polymath project: lessons from a successful online collaboration in mathematics. In Proc.CHI 2011, 1865--1874.
[7]
Dow, S., Kulkarni, A. Klemmer, S., and Hartmann, B. (2012). Shepherding the crowd yields better work. In Proc. CSCW 2012, 1013--1022.
[8]
Heimerl, K., Gawalt, B., Chen, K., Parikh, T., and Hartmann, B. (2012). CommunitySourcing: engaging local crowds to perform expert work via physical kiosks. In Proc.CHI 2012, 1539--154
[9]
Joseph, M. and Reagle, J. (2007). Do as I do: authorial leadership in wikipedia. In Proc. WikiSym 2007, 143--156.
[10]
Kanefsky, B., Barlow, N., and Gulick, V. (2000). Can distributed volunteers accomplish massive data analysis tasks? Lunar and Planetary Science 32, 1272.
[11]
1Kensing, F. and Blomberg, J. (1998). Participatory design: issues and concerns. Computer Supported Cooperative Work 7, 3, 167--185.
[12]
Kittur, A., E. H. Chi., and Suh, B. (2007). Crowdsourcing user studies with Mechanical Turk. In Proc.CHI 2008, 453--456.
[13]
Kittur, A. (2010). Crowdsourcing, collaboration and creativity. XRDS 17, 2, 22--26.
[14]
Kittur, A., Smus, B., Khamkar, S., and Kraut, R. (2011). CrowdForge: crowdsourcing complex work. In Proc.UIST 2011, 43--52.
[15]
Kittur, A., Khamkar, S., André, P., and Kraut, R. (2012). CrowdWeaver: visually managing complex crowd work. In Proc.CSCW 2012, 1033--1036.
[16]
Kulkarni, A., Cam, M., and Hartmann, B. (2012). Collaboratively crowdsourcing workflows with turkomatic. In Proc.CSCW 2012, 1003--1012.
[17]
Lakhani, K., Hila, L., and Tushman, M. (2012). Open innovation and organizational boundaries: the impact of task decomposition and knowledge distribution on the locus of innovation. HBS Working Paper 12-057, 1--53.
[18]
Little, G., Chilton, L., Goldman, M., and Miller, R. (2009). TurKit: tools for iterative tasks on mechanical Turk. In Proc.HCOMP 2009, 29--30.
[19]
Luther, K. and Bruckman, A. (2008). Leadership in online creative collaboration. In Proc.CSCW 2008, 343--352.
[20]
Piller, F., Schubert, P., Koch, M., and Möslein, K. (2005). Overcoming mass confusion: collaborative customer co-design in online communities. Journal of Computer-Mediated Communication 10, 4, 1--25.
[21]
2RECAPCHA System, http://recaptcha.net/
[22]
2Ryan, R. and Deci, E. (2000). Intrinsic and extrinsic motivations: classic definitions and new directions. Contemporary Educational Psychology 25, 1, 54--67.
[23]
Salganik, M. and Levy, K. (2012). Wiki surveys: Open and quantifiable social data collection, ArXiv e-prints. 1--29. http://arxiv.org/abs/1202.0500/
[24]
Sullivan, B, Wood, C., Iliff, M., Bonney, R., Fink, D., and Kelling, S. (2009). eBird: A citizen-based bird observation network in the biological sciences. Biological Conservation 142, 10, 2282--2292.
[25]
Tang, A. and Boring, S. (2012). #EpicPlay: crowdsourcing sports video highlights. In Proc. CHI 2012, 1569--1572.
[26]
The Johnny Cash Project, http://www.thejohnnycashproject.com/
[27]
Xu, A. and Bailey, B. (2012). What do you think?: a case study of benefit, expectation, and interaction in a large online critique community. In Proc. CSCW 2012, 295--304.
[28]
Yu, L. and Nickerson, J. (2011). Cooks or cobblers?: crowd creativity through combination. In Proc.CHI 2011, 1393--1402.
[29]
99 Designs, http://99designs.com/

Cited By

View all
  • (2024)Team Collaboration Vs Team Competition: Pandavas Vs Kauravas - A Literature Evidence from Kurukshetra (Mahabharata) WarSSRN Electronic Journal10.2139/ssrn.4674713Online publication date: 2024
  • (2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
  • (2024)COR Themes for Readability from Iterative FeedbackProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642108(1-23)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Crowd vs. crowd: large-scale cooperative design through open team competition

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work
    February 2013
    1594 pages
    ISBN:9781450313315
    DOI:10.1145/2441776
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 February 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. collaboration
    2. competition
    3. crowdsourcing
    4. design process

    Qualifiers

    • Research-article

    Conference

    CSCW '13
    Sponsor:
    CSCW '13: Computer Supported Cooperative Work
    February 23 - 27, 2013
    Texas, San Antonio, USA

    Acceptance Rates

    Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

    Upcoming Conference

    CSCW '24

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)23
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 15 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Team Collaboration Vs Team Competition: Pandavas Vs Kauravas - A Literature Evidence from Kurukshetra (Mahabharata) WarSSRN Electronic Journal10.2139/ssrn.4674713Online publication date: 2024
    • (2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
    • (2024)COR Themes for Readability from Iterative FeedbackProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642108(1-23)Online publication date: 11-May-2024
    • (2023)Team Collaboration Vs Team Competition: Pandavas Vs Kauravas - A Literature Evidence from Kurukshetra (Mahabharata) WarInternational Journal of Philosophy and Languages (IJPL)10.47992/IJPL.2583.9934.0017(28-42)Online publication date: 18-Oct-2023
    • (2023)Conceptual Architectural Design at Scale: A Case Study of Community Participation Using CrowdsourcingBuildings10.3390/buildings1301022213:1(222)Online publication date: 13-Jan-2023
    • (2023)THERIF: Themes for Readability from Iterative FeedbackExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585679(1-11)Online publication date: 19-Apr-2023
    • (2022)Involving Crowdworkers with Lived Experience in Content-Development for Push-Based Digital Mental Health Tools: Lessons Learned from Crowdsourcing Mental Health MessagesProceedings of the ACM on Human-Computer Interaction10.1145/35129466:CSCW1(1-30)Online publication date: 7-Apr-2022
    • (2022)The Impact of Digital Nudging Techniques on the Formation of Self-Assembled Crowd Project TeamsProceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3503252.3531298(265-275)Online publication date: 4-Jul-2022
    • (2022)Stochastic Sequential Allocations for Creative CrowdsourcingProduction and Operations Management10.1111/poms.1357331:2(697-714)Online publication date: 1-Feb-2022
    • (2021)Spacewalker: Rapid UI Design Exploration Using Lightweight Markup Enhancement and Crowd Genetic ProgrammingProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445326(1-11)Online publication date: 6-May-2021
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media