research-article

The jabberwocky programming environment for structured social computing

Authors:

Sepander KamvarAuthors Info & Claims

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

Pages 53 - 64

https://doi.org/10.1145/2047196.2047203

Published: 16 October 2011 Publication History

Abstract

We present Jabberwocky, a social computing stack that consists of three components: a human and machine resource management system called Dormouse, a parallel programming framework for human and machine computation called ManReduce, and a high-level programming language on top of ManReduce called Dog. Dormouse is designed to enable cross-platform programming languages for social computation, so, for example, programs written for Mechanical Turk can also run on other crowdsourcing platforms. Dormouse also enables a programmer to easily combine crowdsourcing platforms or create new ones. Further, machines and people are both first-class citizens in Dormouse, allowing for natural parallelization and control flows for a broad range of data-intensive applications. And finally and importantly, Dormouse includes notions of real identity, heterogeneity, and social structure. We show that the unique properties of Dormouse enable elegant programming models for complex and useful problems, and we propose two such frameworks. ManReduce is a framework for combining human and machine computation into an intuitive parallel data flow that goes beyond existing frameworks in several important ways, such as enabling functions on arbitrary communication graphs between human and machine clusters. And Dog is a high-level procedural language written on top of ManReduce that focuses on expressivity and reuse. We explore two applications written in Dog: bootstrapping product recommendations without purchase data, and expert labeling of medical images.

References

[1]

S Bamford and et al. Galaxy Zoo.

[2]

M. Bernstein, G. Little, R. Miller, B. Hartmann, M. Ackerman, D. Karger, D. Crowell, and K. Panovich. Soylent: a word processor with a crowd inside. In Proc. UIST (2010).

Digital Library

[3]

S Cooper, F. Khatib, A. Treuille, J. Barbero, J. Lee, M. Beenen, A. Leaver-Fay, D. Baker, and Z. Popovic. Predicting protein structures with a multiplayer online game. Nature, June 2010.

[4]

J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. Communications ACM, January 2008.

Digital Library

[5]

FJ Diez. Parameter adjustment in bayes networks. the generalized noisy or-gate. In Proc. UAI (1993).

Digital Library

[6]

Michael J. Franklin, Donald Kossmann, Tim Kraska, Sukriti Ramesh, and Reynold Xin. CrowdDB: answering queries with crowdsourcing. In Proc. SIGMOD (2011), pages 61--72.

Digital Library

[7]

William Gropp, Ewing Lusk, Nathan Doss, and Anthony Skjellum. A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing, 1996.

Digital Library

[8]

B. Hartmann. Amazing but True Cat Stories. http://bjoern.org/projects/catbook/, April 2009.

[9]

P. Heymann and H. Garcia-Molina. Human processing. Technical report.

[10]

R. Hickey. The clojure programming language. In Proc. DLS (2008).

Digital Library

[11]

D. Horowitz and S.D. Kamvar. The anatomy of a large-scale social search engine. In Proc. WWW (2010).

Digital Library

[12]

M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. SIGOPS Operating Systems Review, March 2007.

Digital Library

[13]

A. Kittur, B. Smus, and R. E. Kraut. CrowdForge: Crowdsourcing Complex Work.

[14]

W. Lasecki, K. Murray, S. White, R. Miller, and F. Bigham. Legion: closed-loop crowd control of existing interfaces. In Proc. UIST (2011).

Digital Library

[15]

T. Lindholm and F. Yellin. Java Virtual Machine Specification. Addison-Wesley Longman Publishing Co., Inc., 2nd edition, 1999.

Digital Library

[16]

G. Little, L. Chilton, M. Goldman, and R. Miller. TurKit: Tools for Iterative Tasks on Mechanical Turk. In Proc. HCOMP (2009).

Digital Library

[17]

A. Marcus, E. Wu, S. Madden, and R. C. Miller. Crowdsourced Databases: Query Processing with People. In Proc. CIDR (2011).

[18]

M. Odersky, P. Altherr, V. Cremet, B. Emir, S. Maneth, S. Micheloud, N. Mihaylov, M. Schinz, E. Stenman, and M. Zenger. An overview of the Scala programming language. Technical report, EPFL Lausanne, Switzerland.

[19]

C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A Not-So-Foreign Language for Data Processing. In Proc. SIGMOD (2008).

Digital Library

[20]

A. Parameswaran and N Polyzotis. Answering Queries using Humans, Algorithms and Databases. Technical report.

[21]

R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with Sawzall. Scientific Programming, October 2005.

Digital Library

[22]

D. Tarditi, S. Puri, and J. Oglesby. Accelerator: using data parallelism to program GPUs for general-purpose uses. SIGOPS Operating Systems Review, October 2006.

Digital Library

[23]

L. von Ahn. Games with a Purpose. Computer, June 2006.

Digital Library

[24]

Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. Gunda, and J. Currey. DryadLINQ: a system for general-purpose distributed data-parallel computing using a high-level language. In Proc. OSDI (2008).

Digital Library

[25]

L. Zhang, M. Shimoji, B. Thomas, D. Moore, S. Yu, N. Marupudi, R. Torp, I. Torgner, O. Ottersen, T. Dawson, and V. Dawson. Mitochondrial localization of the Parkinson's disease related protein DJ-1. Human Molecular Genetics, June 2005.

Cited By

Pradhan VSchaekermann MLease M(2022)In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd WorkersFrontiers in Artificial Intelligence10.3389/frai.2022.8281875Online publication date: 18-May-2022
https://doi.org/10.3389/frai.2022.828187
Xiong TPan MYu YLou D(2022)A Low-Code Framework for Complex Crowdsourcing Work Based on Process ModelingComputational Intelligence and Neuroscience10.1155/2022/94967412022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/9496741
Smirnov AShilov NPonomarev A(2022)Context-Aware Knowledge Management as an Enabler for Human-Machine Collective IntelligenceKnowledge Discovery, Knowledge Engineering and Knowledge Management10.1007/978-3-031-14602-2_5(94-116)Online publication date: 7-Sep-2022
https://doi.org/10.1007/978-3-031-14602-2_5
Show More Cited By

Index Terms

The jabberwocky programming environment for structured social computing
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces

Recommendations

Parametric polymorphism in an object-oriented distributed computing programming environment
Advanced structured COBOL programming
Special issue seventh technical symposium on computer science education

There is a large market demand for COBOL programmers. Can a single introductory course in COBOL programming provide the students with sufficient professional expertise to qualify them for these job openings? If not, is it feasible to offer a second ...
Advanced structured COBOL programming
SIGCSE '77: Proceedings of the seventh SIGCSE technical symposium on Computer science education

There is a large market demand for COBOL programmers. Can a single introductory course in COBOL programming provide the students with sufficient professional expertise to qualify them for these job openings? If not, is it feasible to offer a second ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technology

October 2011

654 pages

ISBN:9781450307161

DOI:10.1145/2047196

General Chair:
Jeff Pierce
IBM Research, USA
,
Program Chairs:
Maneesh Agrawala
University of California, Berkeley, USA
,
Scott Klemmer
Stanford University, USA

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '11

Sponsor:

UIST '11: The 24th Annual ACM Symposium on User Interface Software and Technology

October 16 - 19, 2011

California, Santa Barbara, USA

Acceptance Rates

UIST '11 Paper Acceptance Rate 67 of 262 submissions, 26%;

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

113
Total Citations
View Citations
688
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pradhan VSchaekermann MLease M(2022)In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd WorkersFrontiers in Artificial Intelligence10.3389/frai.2022.8281875Online publication date: 18-May-2022
https://doi.org/10.3389/frai.2022.828187
Xiong TPan MYu YLou D(2022)A Low-Code Framework for Complex Crowdsourcing Work Based on Process ModelingComputational Intelligence and Neuroscience10.1155/2022/94967412022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/9496741
Smirnov AShilov NPonomarev A(2022)Context-Aware Knowledge Management as an Enabler for Human-Machine Collective IntelligenceKnowledge Discovery, Knowledge Engineering and Knowledge Management10.1007/978-3-031-14602-2_5(94-116)Online publication date: 7-Sep-2022
https://doi.org/10.1007/978-3-031-14602-2_5
Ponomarev AShilov N(2021)Human-Computer Cloud and Its Applications in E-TourismResearch Anthology on Decision Support Systems and Decision Management in Healthcare, Business, and Engineering10.4018/978-1-7998-9023-2.ch055(1143-1176)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-9023-2.ch055
Smirnov APonomarev AShilov NKashevnik ATeslya N(2021)Ontology-Based Human-Computer Cloud for Decision SupportResearch Anthology on Decision Support Systems and Decision Management in Healthcare, Business, and Engineering10.4018/978-1-7998-9023-2.ch019(410-430)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-9023-2.ch019
Kamali MMalek MSaeedi SLiang S(2021)A Blockchain-Based Spatial Crowdsourcing System for Spatial Information Collection Using a Reward DistributionSensors10.3390/s2115514621:15(5146)Online publication date: 29-Jul-2021
https://doi.org/10.3390/s21155146
Coblenz MKambhatla GKoronkevich PWise JBarnaby CSunshine JAldrich JMyers B(2021)PLIERSACM Transactions on Computer-Human Interaction10.1145/345237928:4(1-53)Online publication date: 23-Jul-2021
https://dl.acm.org/doi/10.1145/3452379
Louie RGarg KWerner JSun AGergle DZhang H(2021)Opportunistic Collective ExperiencesProceedings of the ACM on Human-Computer Interaction10.1145/34341784:CSCW3(1-32)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3434178
Roh YHeo GWhang S(2021)A Survey on Data Collection for Machine Learning: A Big Data - AI Integration PerspectiveIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2019.294616233:4(1328-1347)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TKDE.2019.2946162
Smirnov APonomarev A(2021)Stimulating Self-Organization in Human-Machine Collective Intelligence Environment2021 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA)10.1109/CogSIMA51574.2021.9475937(94-102)Online publication date: 14-May-2021
https://doi.org/10.1109/CogSIMA51574.2021.9475937
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents