research-article

Studying user browsing behavior through gamified search tasks

Authors:

Jiyin He,

Marc Bron,

Leif Azzopardi, and

Arjen de VriesAuthors Info & Claims

GamifIR '14: Proceedings of the First International Workshop on Gamification for Information Retrieval

April 2014

Pages 49 - 52

https://doi.org/10.1145/2594776.2594787

Published: 13 April 2014 Publication History

Get Access

Abstract

Typical crowdsourcing tasks ask workers to label images or make relevance judgements, as a low cost alternative to lab based user studies. More recently, gamification has been employed as a way to make these tasks more appealing and so users play, rather than work. One observation is that differences in task design and incentives elicits different player behavior. In this paper we discuss a new type of task, where we aim at eliciting player behavior that resembles user behavior when performing a search task. Care should be taken in the design of a gamified version of such a task to allow players to complete tasks with a limited amount of effort and time, without changing the behavior to be studied. We discuss the motivation of the abstractions and design choices we have made in achieving this goal. We then analyze whether and how these abstractions and design choices influence our observations of player behaviors.

References

[1]

T. Demeester, D. Trieschnigg, D. Nguyen, and D. Hiemstra. Overview of the trec 2013 federated web search track. In TREC'14. TREC, 2014.

Google Scholar

[2]

C. Eickhoff, C. G. Harris, A. P. de Vries, and P. Srinivasan. Quality through flow and immersion: gamifying crowdsourced relevance assessments. In SIGIR'12, pages 871--880. ACM, 2012.

Digital Library

Google Scholar

[3]

J. Hamari, J. Koivisto, and H. Sarsa. Does gamification work?-a literature review of empirical studies on gamification. In Proceedings of the 47th Hawaii International Conference on System Sciences. HICSS, 2014.

Digital Library

Google Scholar

[4]

J. He, J. van Ossenbruggen, and A. P. de Vries. Do you need experts in the crowd?: a case study in image annotation for marine biology. In Proceedings of the 10th Conference on Open Research Areas in Information Retrieval, pages 57--60, 2013.

Digital Library

Google Scholar

[5]

M. Hildebrand, M. Brinkerink, R. Gligorov, M. van Steenbergen, J. Huijkman, and J. Oomen. Waisda?: video labeling game. In Proceedings of the 21st ACM international conference on Multimedia, pages 823--826. ACM, 2013.

Digital Library

Google Scholar

[6]

J. J. Horton and L. B. Chilton. The labor economics of paid crowdsourcing. In Proceedings of the 11th ACM conference on Electronic commerce, pages 209--218. ACM, 2010.

Digital Library

Google Scholar

[7]

J. Huang, R. W. White, and S. Dumais. No clicks, no problem: using cursor movements to understand and improve search. In SIGCHI'11, pages 1225--1234. ACM, 2011.

Digital Library

Google Scholar

[8]

L. Von Ahn. Games with a purpose. Computer, 39(6):92--94, 2006.

Digital Library

Google Scholar

[9]

E. M. Voorhees. Variations in relevance judgments and the measurement of retrieval effectiveness. Information processing & management, 36(5):697--716, 2000.

Digital Library

Google Scholar

Cited By

View all

Medlar AGłowacka D(2022)Game over?ACM SIGIR Forum10.1145/3527546.352755155:2(1-18)Online publication date: 17-Mar-2022
https://dl.acm.org/doi/10.1145/3527546.3527551
Jin YCarman MZhu YXiang Y(2020)A technical survey on statistical modelling and design methods for crowdsourcing quality controlArtificial Intelligence10.1016/j.artint.2020.103351(103351)Online publication date: Jun-2020
https://doi.org/10.1016/j.artint.2020.103351
Mackenzie JGupta KQiao FAwadallah AShokouhi M(2019)Exploring User Behavior in Email Re-Finding TasksThe World Wide Web Conference10.1145/3308558.3313450(1245-1255)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313450
Show More Cited By

Index Terms

Studying user browsing behavior through gamified search tasks
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
2. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Understanding user behavior in naturalistic information search tasks

Understanding users' search behavior has largely relied on the information available from search engine logs, which provide limited information about the contextual factors affecting users' behavior. Consequently, questions such as how users' intentions, ...
Read More
On Utilizing Player Models to Predict Behavior in Crowdsourcing Tasks
Social Informatics
Abstract
Player Modeling is a research field that studies player characteristics by analyzing in-game behavior. We aim to develop independent models, which are transferable and useful beyond a game’s context. We shall demonstrate the feasibility of this ...
Read More
Time use behavior in single and time-sharing tasks

Human errors in aviation, process plants and other critical industries can result in dire consequences and hence it is essential to understand the operator behavior and task characteristics in order to improve task performance and safety. The time ...
Read More

Comments

Information & Contributors

Information

Published In

GamifIR '14: Proceedings of the First International Workshop on Gamification for Information Retrieval

April 2014

68 pages

ISBN:9781450328920

DOI:10.1145/2594776

Conference Chairs:
Frank Hopfgartner
TU Berlin
,
Gabriella Kazai
Microsoft Research
,
Udo Kruschwitz
University of Essex
,
Michael Meder
TU Berlin

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Conference

GamifIR '14

Sponsor:

Technische Universitat Berlin
Microsoft Research

GamifIR '14: Workshop on Gamification for Information Retrieval

April 13, 2014

Amsterdam, The Netherlands

Acceptance Rates

GamifIR '14 Paper Acceptance Rate 14 of 18 submissions, 78%;

Overall Acceptance Rate 14 of 18 submissions, 78%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
287
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

View all

Medlar AGłowacka D(2022)Game over?ACM SIGIR Forum10.1145/3527546.352755155:2(1-18)Online publication date: 17-Mar-2022
https://dl.acm.org/doi/10.1145/3527546.3527551
Jin YCarman MZhu YXiang Y(2020)A technical survey on statistical modelling and design methods for crowdsourcing quality controlArtificial Intelligence10.1016/j.artint.2020.103351(103351)Online publication date: Jun-2020
https://doi.org/10.1016/j.artint.2020.103351
Mackenzie JGupta KQiao FAwadallah AShokouhi M(2019)Exploring User Behavior in Email Re-Finding TasksThe World Wide Web Conference10.1145/3308558.3313450(1245-1255)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313450
Ferreira AAraújo AFernandes SMiguel I(2017)Gamification in the Workplace: A Systematic Literature ReviewRecent Advances in Information Systems and Technologies10.1007/978-3-319-56541-5_29(283-292)Online publication date: 28-Mar-2017
https://doi.org/10.1007/978-3-319-56541-5_29
Rokicki MZerr SSiersdorfer SBourdeau JHendler JNkambou RHorrocks IZhao B(2016)Just in TimeProceedings of the 25th International Conference on World Wide Web10.1145/2872427.2883075(817-827)Online publication date: 11-Apr-2016
https://dl.acm.org/doi/10.1145/2872427.2883075
Morschheuser BHamari JKoivisto J(2016)Gamification in CrowdsourcingProceedings of the 2016 49th Hawaii International Conference on System Sciences (HICSS)10.1109/HICSS.2016.543(4375-4384)Online publication date: 5-Jan-2016
https://dl.acm.org/doi/10.1109/HICSS.2016.543
Rokicki MZerr SSiersdorfer SGangemi ALeonardi SPanconesi A(2015)GroupsourcingProceedings of the 24th International Conference on World Wide Web10.1145/2736277.2741097(906-915)Online publication date: 18-May-2015
https://dl.acm.org/doi/10.1145/2736277.2741097
Rokicki MChelaru SZerr SSiersdorfer SLi JWang XGarofalakis MSoboroff ISuel TWang M(2014)Competitive Game Designs for Improving the Cost Effectiveness of CrowdsourcingProceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management10.1145/2661829.2661946(1469-1478)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1145/2661829.2661946

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Understanding user behavior in naturalistic information search tasks

On Utilizing Player Models to Predict Behavior in Crowdsourcing Tasks

Time use behavior in single and time-sharing tasks

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Understanding user behavior in naturalistic information search tasks

On Utilizing Player Models to Predict Behavior in Crowdsourcing Tasks

Time use behavior in single and time-sharing tasks

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations