research-article

Collaborative workflow for crowdsourcing translation

Authors:

Vamshi Ambati,

Stephan Vogel,

Jaime CarbonellAuthors Info & Claims

CSCW '12: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work

Pages 1191 - 1194

https://doi.org/10.1145/2145204.2145382

Published: 11 February 2012 Publication History

Get Access

Abstract

In this paper we explore the challenges in crowdsourcing the task of translation over the web in which remotely located translators work on providing translations independent of each other. We then propose a collaborative workflow for crowdsourcing translation to address some of these challenges. In our pipeline model, the translators are working in phases where output from earlier phases can be enhanced in the subsequent phases. We also highlight some of the novel contributions of the pipeline model like assistive translation and translation synthesis that can leverage monolingual and bilingual speakers alike. We evaluate our approach by eliciting translations for both a minority-to-majority language pair and a minority-to-minority language pair. We observe that in both scenarios, our workflow produces better quality translations in a cost-effective manner, when compared to the traditional crowdsourcing workflow.

References

[1]

V. Ambati, S. Vogel, and J. Carbonell. Active learning and crowd-sourcing for machine translation. In Proceedings of the LREC 2010, Malta, May 2010.

Google Scholar

[2]

R. E. K. Aniket Kittur, Boris Smus. Crowdforge: Crowdsourcing complex work. Technical report, Human-Computer Interaction Institute, SCS, Carnegie Mellon University, January 2011.

Google Scholar

[3]

M. S. Bernstein, G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: a word processor with a crowd inside. In Proceedings of the 23rd UIST, UIST '10, pages 313--322. ACM, 2010.

Digital Library

Google Scholar

[4]

C. Callison-Burch. Fast, cheap, and creative: Evaluating translation quality using Amazon's Mechanical Turk. In EMNLP 2009, pages 286--295, Singapore, August 2009. Association for Computational Linguistics.

Digital Library

Google Scholar

[5]

C. Hu, B. B. Bederson, and P. Resnik. Translation by iterative collaboration between monolingual users. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP '10, pages 54--55. ACM, 2010.

Digital Library

Google Scholar

[6]

G. Little, L. B. Chilton, M. Goldman, and R. C. Miller. Turkit: tools for iterative tasks on mechanical turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP '09, pages 29--30, New York, NY, USA, 2009. ACM.

Digital Library

Google Scholar

[7]

S. Nirenburg and R. Frederking. Toward multi-engine machine translation. In HLT '94: Proceedings of Human Language Technology, pages 147--151, Morristown, NJ, USA, 1994. ACL.

Digital Library

Google Scholar

[8]

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL 2002, pages 311--318, Morristown, NJ, USA, 2002.

Digital Library

Google Scholar

[9]

R. Snow, B. O'Connor, D. Jurafsky, and A. Ng. Cheap and fast - but is it good? evaluating non-expert annotations for natural language tasks. In Proceedings of the EMNLP 2008, pages 254--263, Honolulu, Hawaii, October 2008.

Digital Library

Google Scholar

Cited By

View all

Yang TMiceli M(2024)"Guilds" as Worker Empowerment and Control in a Chinese Data Work PlatformProceedings of the ACM on Human-Computer Interaction10.1145/36869048:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686904
Fulker ZRiedl C(2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637314
Guntha RA APresannakumar M(2024)Harmonizing Languages: A Hybrid Translation Architecture for Multilingual Interfaces in the Layamritam App2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C63560.2024.00012(32-39)Online publication date: 4-Jun-2024
https://doi.org/10.1109/ICSA-C63560.2024.00012
Show More Cited By

Index Terms

Collaborative workflow for crowdsourcing translation
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Computer supported cooperative work
2. Social and professional topics
  1. Professional topics
    1. Computing and business
      1. Computer supported cooperative work

Recommendations

Dialogue Machine Translation System Using Multiple Translation Processors
DEXA '00: Proceedings of the 11th International Workshop on Database and Expert Systems Applications

In natural dialogues, speakers may make some kinds of mistakes or so called irregular expressions. One of the most difficult problems which Dialogue Machine Translations must deal with is translating irregular expressions in the natural conversation, ...
Comparison of crowdsourcing translation with Machine Translation

In this paper we examine the model of crowdsourcing for translation and compare it with Machine Translation (MT). The large volume of material to be translated, the translation of this material into many languages combined with tight deadlines lead ...
Using targeted paraphrasing and monolingual crowdsourcing to improve translation
Special Sections on Paraphrasing; Intelligent Systems for Socially Aware Computing; Social Computing, Behavioral-Cultural Modeling, and Prediction

Targeted paraphrasing is a new approach to the problem of obtaining cost-effective, reasonable quality translation, which makes use of simple and inexpensive human computations by monolingual speakers in combination with machine translation. The key ...

Comments

Information & Contributors

Information

Published In

CSCW '12: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work

February 2012

1460 pages

ISBN:9781450310864

DOI:10.1145/2145204

General Chairs:
Steven Poltrock
Boeing Research & Technology
,
Carla Simone
University of Milano-Biccoca
,
Program Chairs:
Jonathan Grudin
Microsoft Research
,
Gloria Mark
University of California, Irvine
,
John Riedl
University of Minnesota

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 February 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CSCW '12

Sponsor:

SIGCHI

CSCW '12: Computer Supported Cooperative Work

February 11 - 15, 2012

Washington, Seattle, USA

Acceptance Rates

CSCW '12 Paper Acceptance Rate 164 of 415 submissions, 40%;

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
1,075
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yang TMiceli M(2024)"Guilds" as Worker Empowerment and Control in a Chinese Data Work PlatformProceedings of the ACM on Human-Computer Interaction10.1145/36869048:CSCW2(1-27)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.1145/3686904
Fulker ZRiedl C(2024)Cooperation in the Gig Economy: Insights from Upwork FreelancersProceedings of the ACM on Human-Computer Interaction10.1145/36373148:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637314
Guntha RA APresannakumar M(2024)Harmonizing Languages: A Hybrid Translation Architecture for Multilingual Interfaces in the Layamritam App2024 IEEE 21st International Conference on Software Architecture Companion (ICSA-C)10.1109/ICSA-C63560.2024.00012(32-39)Online publication date: 4-Jun-2024
https://doi.org/10.1109/ICSA-C63560.2024.00012
Ez-Zaouia MCarrillo R(2023)The Group Folding Effect: The Role of Collaborative Process Structuring and Social Interaction in Group WorkACM Transactions on Computer-Human Interaction10.1145/362278331:2(1-44)Online publication date: 4-Sep-2023
https://dl.acm.org/doi/10.1145/3622783
Angelucci MMarshall HTebourbi MSeguin JVarghese DOlivier PBartindale T(2023)Action Translate: Supporting Students in Translation VolunteeringProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581129(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581129
Wang SDang D(2023)A Generative Answer Aggregation Model for Sentence-Level Crowdsourcing TasksIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.314282135:4(3299-3312)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TKDE.2022.3142821
Wu JSong CChang W(2023)Crowdsourcing as a Future Collaborative Computing ParadigmMobile Crowdsourcing10.1007/978-3-031-32397-3_1(3-32)Online publication date: 21-Apr-2023
https://doi.org/10.1007/978-3-031-32397-3_1
Hettiachchi DKostakos VGoncalves J(2022)A Survey on Task Assignment in CrowdsourcingACM Computing Surveys10.1145/349452255:3(1-35)Online publication date: 3-Feb-2022
https://dl.acm.org/doi/10.1145/3494522
Hettiachchi DSchaekermann MMcKinney TLease M(2021)The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can HelpProceedings of the ACM on Human-Computer Interaction10.1145/34760735:CSCW2(1-26)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3476073
Itoh YMatsubara S(2021)Adaptive Budget Allocation for Cooperative Task Solving in Crowdsourcing2021 IEEE International Conference on Big Data (Big Data)10.1109/BigData52589.2021.9671713(3525-3533)Online publication date: 15-Dec-2021
https://doi.org/10.1109/BigData52589.2021.9671713
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Dialogue Machine Translation System Using Multiple Translation Processors

Comparison of crowdsourcing translation with Machine Translation

Using targeted paraphrasing and monolingual crowdsourcing to improve translation