research-article

Task allocation for crowdsourcing using AI planning

Authors:

Leticia Machado,

Rafael Prikladnicki,

Felipe Meneguzzi,

Cleidson R. B. de Souza,

Erran CarmelAuthors Info & Claims

CSI-SE '16: Proceedings of the 3rd International Workshop on CrowdSourcing in Software Engineering

Pages 36 - 40

https://doi.org/10.1145/2897659.2897666

Published: 14 May 2016 Publication History

Get Access

Abstract

Crowdsourcing is a relatively new phenomenon in computer science and software engineering. In crowdsourcing a task is delivered to a crowd of participants who will work on this task. Task allocation is then an important aspect in the context of crowdsourcing. If done properly, it delivers successful results based on the answers provided by the crowd. However, task allocation in crowdsourcing is not a trivial problem. Factors like a task's requirements, the knowledge required for its resolution, and the size and heterogeneity of the participants in the crowd all impact task allocation, and therefore, the expected quality of the task results. In this case, the execution of actions from a plan, which assist the dynamic tasks' allocation in crowdsourcing systems, become relevant as an alternative solution. This paper formalizes task allocation in crowdsourcing scenarios as an artificial intelligence planning problem. Our results suggest that task allocation has several challenges when it is observed in distributed, undefined and dynamic environments, like in crowdsourcing scenarios. Our goal is to evaluate if automated planning is appropriate for providing a plan to match skills of crowd workers for the right tasks in software engineering projects. Preliminary results are presented in this paper.

References

[1]

Zhao, Y. and Zhu, Q. "Evaluation on crowdsourcing research: Current status and future direction". Springer Science+Business Media, LLC, 2012.

Google Scholar

[2]

Vukovic, M. "Crowdsourcing for enterprises". In Proceedings of the Congress on Services, pp. 686--692. IEEE Computer Society, 2009.

Digital Library

Google Scholar

[3]

Machado, L., Pereira, G., Prikladnicki, R., Carmel, E., & de Souza, C. R. "Crowdsourcing in the Brazilian IT industry: what we know and what we don't know". In Proceedings of the 1st International Workshop on Crowd-based Software Development Methods and Technologies (pp. 7--12). ACM, November 2014.

Digital Library

Google Scholar

[4]

Hetmank, L. "Towards a Semantic Standard for Enterprise Crowdsourcing -- A Scenario-based Evaluation of a Conceptual Prototype". In 21st European Conference on In-formation Systems (ECIS). Utrecht, 2013.

Google Scholar

[5]

Lykourentzou, I., Vergados, D. J., Papadaki, K., & Naudet, Y. "Guided crowdsourcing for collective work coordination in corporate environments". In Computational Collective Intelligence. Technologies and Applications (pp. 90--99). Springer Berlin Heidelberg, 2013.

Digital Library

Google Scholar

[6]

E. Gerevini, P. Haslum, D. Long, A. Saett amd Y. Dimopoulos, "Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the planners", Artificial Intelligence, Volume 173, Issues 5-6, April 2009, Pages 619--668.

Digital Library

Google Scholar

[7]

S. Srivastava, E. Fang, L. R. R. Chitnis, S. Russell, and P. Abbeel. "Combined task and motion planning through an extensible planner-independent interface layer" In Proceedings of the IEEE Conference on Robotics and Automation (ICRA), 2014, pages 639--646.

Google Scholar

[8]

J. Hoffmann, I. Weber and F. M. Kraft "SAP Speaks PDDL: Exploiting a Software-Engineering Model for Planning in Business Process Management", Journal of Artificial Intelligence Research, Volume 44, July, 2012, pages 587--632.

Digital Library

Google Scholar

[9]

Talamadupula, K., et al. "Herding the crowd: Automated planning for crowdsourced planning." First AAAI Conference on Human Computation and Crowdsourcing. 2013.

Google Scholar

[10]

K. Mao et al. "Developer Recommendation for Crowdsourced Software Development Tasks." Service-Oriented System Engineering (SOSE), 2015 IEEE Symposium on. IEEE, 2015.

Digital Library

Google Scholar

Cited By

View all

Rehman Khan SYounus MIqbal JBasit Ur Rahim M(2024)A Fuzzy AHP-based Quantitative Framework to Prioritize the Crowd-Based Requirements2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00090(680-691)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00090
Rafiei Oskooei AEngin Can BYeldan GMacit Ö(2023)Artificial Intelligence Based Store ManagementThe European Journal of Research and Development10.56038/ejrnd.v3i4.3863:4(240-248)Online publication date: 31-Dec-2023
https://doi.org/10.56038/ejrnd.v3i4.386
Giedrimas VLieponiene JRagalyte BSinickas HPaulaviciene JValinskas ASriubas EPaukstiene A(2023)AI-enchanced Crowdsourcing as an Element of Information Systems DevelopmentInformation Technology and Systems10.1007/978-3-031-33261-6_27(309-318)Online publication date: 20-Aug-2023
https://doi.org/10.1007/978-3-031-33261-6_27
Show More Cited By

Index Terms

Task allocation for crowdsourcing using AI planning
1. Software and its engineering

Recommendations

Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature review
Abstract
Crowdsourcing is simply the outsourcing of different tasks or work to a diverse group of individuals in an open call for the purpose of utilizing human intelligence. Crowdsourcing nowadays used to support and enhance software engineering in ...
Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing
GROUP '18: Proceedings of the 2018 ACM International Conference on Supporting Group Work

In micro-task crowdsourcing markets such as Amazon's Mechanical Turk, how to obtain high quality result without exceeding the limited budgets is one main challenge. The existing theory and practice of crowdsourcing suggests that uneven task difficulty ...
Task Matching in Crowdsourcing
ITHINGSCPSCOM '11: Proceedings of the 2011 International Conference on Internet of Things and 4th International Conference on Cyber, Physical and Social Computing

Crowd sourcing is evolving as a distributed problem-solving and business production model in recent years. In crowd sourcing paradigm, tasks are distributed to networked people to complete such that a company s production cost can be greatly reduced. A ...

Comments

Information & Contributors

Information

Published In

CSI-SE '16: Proceedings of the 3rd International Workshop on CrowdSourcing in Software Engineering

May 2016

55 pages

ISBN:9781450341585

DOI:10.1145/2897659

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

CNPq

Conference

ICSE '16

Sponsor:

ACM
SIGSOFT
IEEE-CS\DATC
TCSE

ICSE '16: 38th International Conference on Software Engineering

May 14 - 22, 2016

Texas, Austin

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)48
Downloads (Last 6 weeks)8

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Rehman Khan SYounus MIqbal JBasit Ur Rahim M(2024)A Fuzzy AHP-based Quantitative Framework to Prioritize the Crowd-Based Requirements2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00090(680-691)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS-C63300.2024.00090
Rafiei Oskooei AEngin Can BYeldan GMacit Ö(2023)Artificial Intelligence Based Store ManagementThe European Journal of Research and Development10.56038/ejrnd.v3i4.3863:4(240-248)Online publication date: 31-Dec-2023
https://doi.org/10.56038/ejrnd.v3i4.386
Giedrimas VLieponiene JRagalyte BSinickas HPaulaviciene JValinskas ASriubas EPaukstiene A(2023)AI-enchanced Crowdsourcing as an Element of Information Systems DevelopmentInformation Technology and Systems10.1007/978-3-031-33261-6_27(309-318)Online publication date: 20-Aug-2023
https://doi.org/10.1007/978-3-031-33261-6_27
Gilal ATunio MWaqas AAlmomani MKhan SGilal R(2022)Task Assignment and PersonalityResearch Anthology on Agile Software, Software Development, and Testing10.4018/978-1-6684-3702-5.ch086(1795-1809)Online publication date: 2022
https://doi.org/10.4018/978-1-6684-3702-5.ch086
Wang GAli FYang JNazir SYang TKhan AImtiaz M(2021)Multicriteria-Based Crowd Selection Using Ant Colony OptimizationComplexity10.1155/2021/66222312021(1-11)Online publication date: 22-Jan-2021
https://doi.org/10.1155/2021/6622231
Rozas DSaldivar JZelickson E(2021)The platform belongs to those who work on it! Co-designing worker-centric task distribution modelsProceedings of the 17th International Symposium on Open Collaboration10.1145/3479986.3479987(1-12)Online publication date: 15-Sep-2021
https://dl.acm.org/doi/10.1145/3479986.3479987
Zhen YKhan ANazir SHuiqi ZAlharbi AKhan S(2021)Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature reviewJournal of Software: Evolution and Process10.1002/smr.2368Online publication date: 30-Jun-2021
https://doi.org/10.1002/smr.2368
Wang YCai ZZhan ZZhao BTong XQi L(2020)Walrasian Equilibrium-Based Multiobjective Optimization for Task Allocation in Mobile CrowdsourcingIEEE Transactions on Computational Social Systems10.1109/TCSS.2020.29957607:4(1033-1046)Online publication date: Aug-2020
https://doi.org/10.1109/TCSS.2020.2995760
Gong DPeng CYao XTian T(2020)A Model of New Workers’ Accurate Acceptance of Tasks Using Capable SensingSwarm and Evolutionary Computation10.1016/j.swevo.2020.100732(100732)Online publication date: Jul-2020
https://doi.org/10.1016/j.swevo.2020.100732
Gilal ATunio MWaqas AAlmomani MKhan SGilal R(2019)Task Assignment and PersonalityHuman Factors in Global Software Engineering10.4018/978-1-5225-9448-2.ch001(1-19)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-9448-2.ch001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature review

Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing

Task Matching in Crowdsourcing

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Crowdsourcing usage, task assignment methods, and crowdsourcing platforms: A systematic literature review

Enabling Uneven Task Difficulty in Micro-Task Crowdsourcing

Task Matching in Crowdsourcing

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations