research-article

Public Access

AutoMan: a platform for integrating human-based and digital computation

Authors:

Daniel W. Barowy,

Charlie Curtsinger,

Emery D. Berger,

Andrew McGregorAuthors Info & Claims

Communications of the ACM, Volume 59, Issue 6

Pages 102 - 109

https://doi.org/10.1145/2927928

Published: 23 May 2016 Publication History

All formats PDF

References

[1]

Ahmad, S., Battle, A., Malkani, Z., Kamvar, S. The Jabberwocky programming environment for structured social computing. In UIST 2011, 53--64.

Digital Library

Google Scholar

[2]

Barowy, D.W., Curtsinger, C., Berger, E.D., McGregor, A. AutoMan: A platform for integrating human-based and digital computation. In OOPSLA 2012, 639--654.

Digital Library

Google Scholar

[3]

Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K., Soylent: A word processor with a crowd inside. In UIST 2010, 313--322.

Digital Library

Google Scholar

[4]

Chang, G.-L., Zou, N. ITS Applications in Work Zones to Improve Traffic Operation and Performance Measurements. Technical Report MD-09-SP708B4G, Maryland Department of Transportation State Highway Administration, May.

Google Scholar

[5]

Davies, P., Emmott, N., Ayland, N. License plate recognition technology for toll violation enforcement. In IEE Colloquium on Image Analysis for Transport Applications (Feb 1990), 7/1--7/5.

Google Scholar

[6]

Douceur, J.R., The Sybil attack. In IPTPS 2001, 251--260.

Digital Library

Google Scholar

[7]

Dow, S., Kulkarni, A., Bunge, B., Nguyen, T., Klemmer, S., Hartmann, B. Shepherding the crowd: Managing and providing feedback to crowd workers. In CHI 2011, 1669--1674.

Digital Library

Google Scholar

[8]

Due, S., Ibrahim, M., Shehata, M., Badawy, W. Automatic license plate recognition (ALPR): A state of the art review. IEEE Trans. Circ. Syst. Video Technol. 23 (2012), 311--325.

Digital Library

Google Scholar

[9]

Franklin, M.J., Kossmann, D., Kraska, T., Ramesh, S., Xin, R. CrowdDB: Answering queries with crowdsourcing. In SIGMOD 2011, 61--72.

Digital Library

Google Scholar

[10]

Holm, S. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 2 (1979), 65--70.

Google Scholar

[11]

Ipeirotis, P.G. Demographics of Mechanical Turk. Technical Report CeDER-10-01, NYU Center for Digital Economy Research, 2010.

Google Scholar

[12]

Ipeirotis, P.G., Provost, F., Wang, J. Quality management on Amazon mechanical turk. In HCOMP 2010, 64--67.

Digital Library

Google Scholar

[13]

Kittur, A., Smus, B., Khamkar, S., Kraut, R.E. CrowdForge: Crowdsourcing Complex Work.

Google Scholar

[14]

Kulkarni, A.P., Can, M., Hartmann, B. Turkomatic: Automatic recursive task and workflow design for mechanical turk. In CHI 2011, 2053--2058.

Digital Library

Google Scholar

[15]

Little, G., Chilton, L.B., Goldman, M., Miller, R.C. TurKit: Human computation algorithms on mechanical turk. In UIST 2010, 57--66.

Digital Library

Google Scholar

[16]

Marge, M., Banerjee, S., Rudnicky, A. Using the Amazon mechanical turk for transcription of spoken language. In ICASSP 2010, 5270--5273, Mar.

Crossref

Google Scholar

[17]

Oleson, D., Hester, V., Sorokin, A., Laughlin, G., Le, J., Biewald, L. Programmatic gold: Targeted and scalable quality assurance in crowdsourcing. In HCOMP 2011, 43--48.

Digital Library

Google Scholar

[18]

Parikh, D., Zitnick, L. Human-debugging of machines. In NIPS CSS 2011.

Google Scholar

[19]

Shahaf, D., Amir, E. Towards a theory of AI completeness. In Commonsense 2007.

Google Scholar

[20]

Tamir, D., Kanth, P., Ipeirotis, P. Mechanical turk: Now with 40.92% spam, Dec 2010. www.behind-the-enemy-lines.com.

Google Scholar

Cited By

View all

Pradhan VSchaekermann MLease M(2022)In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd WorkersFrontiers in Artificial Intelligence10.3389/frai.2022.8281875Online publication date: 18-May-2022
https://doi.org/10.3389/frai.2022.828187
Estrada RMizouni ROtrok HOuali ABentahar J(2020)A Crowd-Sensing Framework for Allocation of Time-Constrained and Location-Based TasksIEEE Transactions on Services Computing10.1109/TSC.2017.272583513:5(769-785)Online publication date: 1-Sep-2020
https://doi.org/10.1109/TSC.2017.2725835
Sabou MWinkler DBiffl S(2020)Empirical Software Engineering Experimentation with Human ComputationContemporary Empirical Methods in Software Engineering10.1007/978-3-030-32489-6_7(173-215)Online publication date: 28-Aug-2020
https://doi.org/10.1007/978-3-030-32489-6_7
Show More Cited By

Index Terms

AutoMan: a platform for integrating human-based and digital computation

Recommendations

AutoMan: a platform for integrating human-based and digital computation
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Humans can perform many tasks with ease that remain difficult or impossible for computers. Crowdsourcing platforms like Amazon's Mechanical Turk make it possible to harness human-based computational power at an unprecedented scale. However, their ...
AutoMan: a platform for integrating human-based and digital computation
OOPSLA '12

Humans can perform many tasks with ease that remain difficult or impossible for computers. Crowdsourcing platforms like Amazon's Mechanical Turk make it possible to harness human-based computational power at an unprecedented scale. However, their ...
AutoMan: Resource-efficient provisioning with tail latency guarantees for microservices
Abstract
Modern user-facing services are progressively evolving from large monolithic applications to complex graphs of loosely-coupled microservices. While microservice architecture greatly improves the efficiency of development and operation, ...
Highlights
- We propose an automatic and SLO-aware resource manager for microservices.
- We ...

Reviews

Reviewer: Amos O Olagunju

Programming digital computers to perform complicated concurrent tasks usually executed by several human beings is important in business intelligence decisions, but not easy. How should multifaceted or interconnected tasks be automated on computers to schedule and monitor human workers in timely and cost-efficient ways Barowy et al. succinctly present automatic algorithms for the effective scheduling, pricing, and quality control of concurrent activities that humans perform to accomplish a task, such as the recognition of alternative license plates at a tollbooth. Can a single programming system be automatically and effectively used to schedule employees who work in parallel to achieve quality tasks The authors present a domain-specific language called AutoMan for coalescing and making transparent the efficient schedules of designed human and digital computation activities for accomplishing alternative, complex real-world tasks. AutoMan provides an easy interface that allows users to specify efficient functions for incorporating crowdsourcing into computations. Users can create functions to define the nature of questions and responses for human workers in AutoMan. The programmer can specify the desirable degree of accuracy of the computational results of all human intelligence tasks in the AutoMan environment. The authors succinctly present algorithms for effectively advertising jobs with rewards for workers to accept the duties on time, and for certifying the quality of job allocations. The authors clearly and convincingly present evidence to illustrate the effective use of AutoMan in applications such as the automatic recognition of license plates. Without a doubt, this paper will promote new research in the areas of the Internet of Things (IoT) and digital computation. I strongly encourage all current and future software engineers to read and contribute to the unique, insightful ideas presented. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 59, Issue 6

June 2016

106 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/2942427

Editor:
Moshe Y. Vardi
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 May 2016

Published in CACM Volume 59, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
16,796
Total Downloads

Downloads (Last 12 months)907
Downloads (Last 6 weeks)97

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pradhan VSchaekermann MLease M(2022)In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd WorkersFrontiers in Artificial Intelligence10.3389/frai.2022.8281875Online publication date: 18-May-2022
https://doi.org/10.3389/frai.2022.828187
Estrada RMizouni ROtrok HOuali ABentahar J(2020)A Crowd-Sensing Framework for Allocation of Time-Constrained and Location-Based TasksIEEE Transactions on Services Computing10.1109/TSC.2017.272583513:5(769-785)Online publication date: 1-Sep-2020
https://doi.org/10.1109/TSC.2017.2725835
Sabou MWinkler DBiffl S(2020)Empirical Software Engineering Experimentation with Human ComputationContemporary Empirical Methods in Software Engineering10.1007/978-3-030-32489-6_7(173-215)Online publication date: 28-Aug-2020
https://doi.org/10.1007/978-3-030-32489-6_7
Phuttharak JLoke S(2019)A Review of Mobile Crowdsourcing Architectures and Challenges: Toward Crowd-Empowered Internet-of-ThingsIEEE Access10.1109/ACCESS.2018.28853537(304-324)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2018.2885353
Amaxilatis DMylonas GTheodoridis EDiez LDeligiannidou K(2019)LearningCity: Knowledge Generation for Smart CitiesSmart Cities Performability, Cognition, & Security10.1007/978-3-030-14718-1_2(17-41)Online publication date: 21-May-2019
https://doi.org/10.1007/978-3-030-14718-1_2
Heartfield RLoukas G(2018)Detecting semantic social engineering attacks with the weakest link: Implementation and empirical evaluation of a human-as-a-security-sensor frameworkComputers & Security10.1016/j.cose.2018.02.02076(101-127)Online publication date: Jul-2018
https://doi.org/10.1016/j.cose.2018.02.020
Barowy DBerger EGoldstein DSuri SMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)VoxPLProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3026025(2347-2358)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3026025
Timofeeva A(2017)Visualization-based analysis of multiple response survey data10.1063/1.5009891(060020)Online publication date: 2017
https://doi.org/10.1063/1.5009891
Lee JLee DHwang S(2017)CrowdKInformation Sciences: an International Journal10.1016/j.ins.2017.03.010399:C(98-120)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1016/j.ins.2017.03.010
Lazar JFeng JHochheiser H(2017)Online and ubiquitous HCI researchResearch Methods in Human Computer Interaction10.1016/B978-0-12-805390-4.00014-5(411-453)Online publication date: 2017
https://doi.org/10.1016/B978-0-12-805390-4.00014-5

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

References

Cited By

Index Terms

Recommendations