research-article

Public Access

Bridging Gaps: Predicting User and Task Characteristics from Partial User Information

Authors:

Matthew Mitsui,

Chirag ShahAuthors Info & Claims

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 415 - 424

https://doi.org/10.1145/3331184.3331221

Published: 18 July 2019 Publication History

Abstract

Interactive information retrieval (IIR) researchers often conduct laboratory studies to understand the relationship between people seeking information and information retrieval systems. They develop extensive data collection methods and tools create new understanding about the relationship between observable behaviors, searcher context, and underlying cognition, to better support people's information seeking. Yet aside from the problems of data size, realism, and demographics, laboratory studies are limited in the number and nature of phenomena they can study. Hence, data collected in laboratories contains different searcher populations and collects non-overlapping user and task characteristics. While research analyses and collection methods are isolated, how can we further IIR's mission of broad understanding? We approach this as a structure learning problem on incomplete data, determining the extent to which incomplete data can be used to predict user and task characteristics from interactions. In particular, we examine whether combining heterogeneous data sets is more effective than using a single data set alone in prediction. Our results indicate that adding external data significantly improves predictions of searcher characteristics, task characteristics, and behaviors, even when the data does not contain identical information about searchers.

Supplementary Material

MP4 File (cite3-11h20-d2.mp4)

Download
587.29 MB

References

[1]

Pia Borlund and Peter Ingwersen. 1997. The development of a method for the evaluation of interactive information retrieval systems. Journal of documentation, Vol. 53, 3 (1997), 225--250.

[2]

Pia Borlund and Jesper W. Schneider. 2010. Reconsideration of the Simulated Work Task Situation: A Context Instrument for Evaluation of Information Retrieval Interaction. In Proceedings of the Third Symposium on Information Interaction in Context (IIiX '10). ACM, New York, NY, USA, 155--164.

Digital Library

[3]

D. J. Campbell. 1988. Task Complexity: A Review and Analysis. Academy of Management Review, Vol. 13, 1 (1988).

[4]

Rob Capra, Jaime Arguello, and Yinglong Zhang. 2017. The effects of search task determinability on search behavior. In Advances in Information Retrieval - 39th European Conference on IR Research, ECIR 2017, Proceedings (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Springer Verlag, Germany, 108--121.

[5]

Michael J. Cole, Jacek Gwizdka, Chang Liu, Nicholas J. Belkin, and Xiangmin Zhang. 2013. Inferring user knowledge level from eye movement patterns. Information Processing & Management, Vol. 49, 5 (2013), 1075--1091.

Digital Library

[6]

Anita Crescenzi, Diane Kelly, and Leif Azzopardi. 2015. Time Pressure and System Delays in Information Search. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '15). ACM, New York, NY, USA, 767--770.

Digital Library

[7]

Nir Friedman. 1997. Learning Belief Networks in the Presence of Missing Values and Hidden Variables. In Proceedings of the Fourteenth International Conference on Machine Learning (ICML '97). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 125--133. http://dl.acm.org/citation.cfm?id=645526.657145

Digital Library

[8]

Maxime Gasse, Alex Aussem, and Haytham Elghazel. 2012. An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning. In Proceedings of the 2012th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I (ECMLPKDD'12). Springer-Verlag, Berlin, Heidelberg, 58--73.

[9]

Ahmed Hassan Awadallah, Ryen W. White, Patrick Pantel, Susan T. Dumais, and Yi-Min Wang. 2014. Supporting Complex Search Tasks. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). ACM, New York, NY, USA, 829--838.

Digital Library

[10]

Jiyin He and Emine Yilmaz. 2017. User Behaviour and Task Characteristics: A Field Study of Daily Information Behaviour. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (CHIIR '17). ACM, New York, NY, USA, 67--76.

Digital Library

[11]

Daniel Hienert, Matthew Mitsui, Philipp Mayr, Chirag Shah, and Nicholas J. Belkin. 2018. The Role of the Task Topic in Web Search of Different Task Types. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR '18). ACM, New York, NY, USA, 72--81.

Digital Library

[12]

Rong Hu, Kun Lu, and Soohyung Joo. 2013. Effects of Topic Familiarity and Search Skills on Query Reformulation Behavior. In Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries (ASIST '13). American Society for Information Science, Silver Springs, MD, USA, Article 66, 9 pages. http://dl.acm.org/citation.cfm?id=2655780.2655846

Digital Library

[13]

Jiepu Jiang, Daqing He, and James Allan. 2014. Searching, Browsing, and Clicking in a Search Session: Changes in User Behavior by Task and over Time. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '14). ACM, New York, NY, USA, 607--616.

Digital Library

[14]

Liangda Li, Hongbo Deng, Anlei Dong, Yi Chang, and Hongyuan Zha. 2014. Identifying and Labeling Search Tasks via Query-based Hawkes Processes. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '14). ACM, New York, NY, USA, 731--740.

Digital Library

[15]

Yuelin Li and Nicholas J. Belkin. 2008. A Faceted Approach to Conceptualizing Tasks in Information Seeking. Inf. Process. Manage., Vol. 44, 6 (Nov. 2008), 1822--1837.

Digital Library

[16]

Chang Liu, Nicholas J. Belkin, and Michael J. Cole. 2012. Personalization of Search Results Using Interaction Behaviors in Search Sessions. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '12). ACM, New York, NY, USA, 205--214.

Digital Library

[17]

Chang Liu, Jacek Gwizdka, and Nicholas J. Belkin. 2010. Analysis of Query Reformulation Types on Different Search Tasks (iConference '10). 4.

[18]

Chang Liu, Jingjing Liu, Michael Cole, Nicholas J. Belkin, and Xiangmin Zhang. {n. d.}. Task difficulty and domain knowledge effects on information search behaviors. Proceedings of the American Society for Information Science and Technology, Vol. 49, 1 ( {n. d.}), 1--10. https://doi.org/10.1016/j.apergo.2016.05.006

[19]

Chang Liu and Yiming Wei. 2016. The Impacts of Time Constraint on Users' Search Strategy During Search Process. In Proceedings of the 79th ASIS&T Annual Meeting: Creating Knowledge, Enhancing Lives Through Information & Technology (ASIST '16). American Society for Information Science, Silver Springs, MD, USA, Article 51, 9 pages. http://dl.acm.org/citation.cfm?id=3017447.3017498.

Digital Library

[20]

Jingjing Liu, Michael J. Cole, Chang Liu, Ralf Bierig, Jacek Gwizdka, Nicholas J. Belkin, Jun Zhang, and Xiangmin Zhang. 2010. Search Behaviors in Different Task Types. In Proceedings of the 10th Annual Joint Conference on Digital Libraries (JCDL '10). ACM, New York, NY, USA, 69--78.

Digital Library

[21]

Jingjing Liu, Jacek Gwizdka, Chang Liu, and Nicholas J. Belkin. 2010. Predicting Task Difficulty for Different Task Types. In Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem - Volume 47 (ASIS&T '10). American Society for Information Science, Silver Springs, MD, USA, Article 16, 10 pages. http://dl.acm.org/citation.cfm?id=1920331.1920355.

Digital Library

[22]

Claudio Lucchese, Salvatore Orlando, Raffaele Perego, Fabrizio Silvestri, and Gabriele Tolomei. 2011. Identifying Task-based Sessions in Search Engine Query Logs. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (WSDM '11). ACM, New York, NY, USA, 277--286.

Digital Library

[23]

Gary Marchionini. 1989. Information-seeking strategies of novices using a full-text electronic encyclopedia. Journal of the American Society for Information Science 40, 1 (1989), 54--66.

Digital Library

[24]

Rishabh Mehrotra and Emine Yilmaz. 2015. Terms, Topics & Tasks: Enhanced User Modelling for Better Personalization. In Proceedings of the 2015 International Conference on The Theory of Information Retrieval (ICTIR '15). ACM, New York, NY, USA, 131--140.

Digital Library

[25]

Matthew Mitsui, Jiqun Liu, and Chirag Shah. 2018. The Paradox of Personalization: Does Task Prediction Require Individualized Models? In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR '18). ACM, New York, NY, USA, 277--280.

Digital Library

[26]

Matthew Mitsui and Chirag Shah. 2018. The Broad View of Task Type Using Path Analysis. In Proceedings of The 8th International Conference on the Theory of Information Retrieval (ICTIR '18). ACM, New York, NY, USA, 8.

Digital Library

[27]

Iraj Mohammadfam, Fakhradin Ghasemi, Omid Kalatpour, and Abbas Moghimbeigi. 2017. Constructing a Bayesian network model for improving safety behavior of employees at workplaces. Applied Ergonomics 58 (2017), 35--47.

[28]

R. Neapolitan, X. Jiang, D. P. Ladner, and B. Kaplan. 2016. A Primer on Bayesian Decision Analysis With an Application to a Kidney Transplant Decision. In Transplantation. 489--96.

[29]

Filip Radlinski and Nick Craswell. 2017. A Theoretical Framework for Conversational Search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (CHIIR '17). ACM, New York, NY, USA, 117--126.

Digital Library

[30]

G. A. P. Saptawati and B. Sitohang. 2005. Hybrid algorithm for learning structure of Bayesian network from incomplete databases. In IEEE International Symposium on Communications and Information Technology, 2005. ISCIT 2005., Vol. 1. 741--744.

[31]

M. Berkan Sesen, Ann E. Nicholson, Rene Banares-Alcantara, Timor Kadir, and Michael Brady. 2013. Bayesian Networks for Clinical Decision Support in Lung Cancer Care. PLOS ONE, Vol. 8, 12 (12 2013).

[32]

Joshua B. Tenenbaum and Thomas L. Griffiths. 2001. Structure Learning in Human Causal Induction. In Advances in Neural Information Processing Systems 13, T. K. Leen, T. G. Dietterich, and V. Tresp (Eds.). MIT Press, 59--65. http://papers.nips.cc/paper/1845-structure-learning-in-human-causal-induction.pdf

Digital Library

[33]

Elaine G. Toms, Heather O'Brien, Tayze Mackenzie, Chris Jordan, Luanne Freund, Sandra Toze, Emilie Dawe, and Alexandra MacNutt. 2008. Task Effects on Interactive Search: The Query Factor. In Focused Access to XML Documents, Norbert Fuhr, Jaap Kamps, Mounia Lalmas, and Andrew Trotman (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 359--372.

Digital Library

[34]

Manisha Verma and Emine Yilmaz. 2016. Category Oriented Task Extraction. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (CHIIR '16). ACM, New York, NY, USA, 333--336.

Digital Library

[35]

Ryen W. White, Susan T. Dumais, and Jaime Teevan. 2009. Characterizing the Influence of Domain Expertise on Web Search Behavior. In Proceedings of the Second ACM International Conference on Web Search and Data Mining (WSDM '09). ACM, New York, NY, USA, 132--141.

Digital Library

[36]

Ryen W. White and Diane Kelly. 2006. A Study on the Effects of Personalization and Task Information on Implicit Feedback Performance. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management (CIKM '06). ACM, New York, NY, USA, 297--306.

Digital Library

[37]

Hong (Iris) Xie. 2002. Patterns between interactive intentions and information-seeking strategies. Information Processing & Management, Vol. 38, 1 (2002), 55--77. http://www.sciencedirect.com/science/article/pii/S0306457301000188

Digital Library

Cited By

Shah CWhite RThomas PMitra BSarkar SBelkin N(2023)Taking Search to TaskProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578288(1-13)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578288
Sarkar SAmirizaniani MShah C(2023)Representing Tasks with a Graph-Based Method for Supporting Users in Complex Search TasksProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578279(378-382)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578279
Shah CWhite R(2021)Task Intelligence for Search and RecommendationSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S01103ED1V01Y202105ICR07413:3(1-160)Online publication date: 9-Jun-2021
https://doi.org/10.2200/S01103ED1V01Y202105ICR074
Show More Cited By

Index Terms

Bridging Gaps: Predicting User and Task Characteristics from Partial User Information

Recommendations

The Broad View of Task Type Using Path Analysis
ICTIR '18: Proceedings of the 2018 ACM SIGIR International Conference on Theory of Information Retrieval

Many past research efforts have examined the relationships between observable search behaviors, the task that drives searching, and user characteristics such as search expertise and topic familiarity. These studies often look at pairwise relationships ...
A faceted approach to conceptualizing tasks in information seeking

The nature of the task that leads a person to engage in information interaction, as well as of information seeking and searching tasks, have been shown to influence individuals' information behavior. Classifying tasks in a domain has been viewed as a ...
How Much is Too Much?: Whole Session vs. First Query Behaviors in Task Type Prediction
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

One of the emerging and important problems in Interactive Information Retrieval research is predicting search tasks. Given a searcher's behavior during a search session, can the searcher's task be predicted? Which aspects of the task can be predicted, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2019

1512 pages

ISBN:9781450361729

DOI:10.1145/3331184

General Chairs:
Benjamin Piwowarski
CNRS - Sorbonne Universite, France
,
Max Chevalier
Universite de Toulouse, CNRS, France
,
Eric Gaussier
Universite Grenoble Alpes, CNRS, France
,
Program Chairs:
Yoelle Maarek
Amazon Research, Israel
,
Jian-Yun Nie
University of Montreal, Canada
,
Falk Scholer
RMIT University, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SIGIR '19

Sponsor:

SIGIR

SIGIR '19: The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 21 - 25, 2019

Paris, France

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
621
Total Downloads

Downloads (Last 12 months)91
Downloads (Last 6 weeks)16

Reflects downloads up to 22 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shah CWhite RThomas PMitra BSarkar SBelkin N(2023)Taking Search to TaskProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578288(1-13)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578288
Sarkar SAmirizaniani MShah C(2023)Representing Tasks with a Graph-Based Method for Supporting Users in Complex Search TasksProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578279(378-382)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578279
Shah CWhite R(2021)Task Intelligence for Search and RecommendationSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S01103ED1V01Y202105ICR07413:3(1-160)Online publication date: 9-Jun-2021
https://doi.org/10.2200/S01103ED1V01Y202105ICR074
Sen PGanguly DJones G(2021)I Know What You Need: Investigating Document Retrieval Effectiveness with Partial Session ContextsACM Transactions on Information Systems10.1145/348866740:3(1-30)Online publication date: 17-Nov-2021
https://dl.acm.org/doi/10.1145/3488667
Shah CWhite RScholer FThomas PElsweiler DJoho HKando NSmith C(2021)Bridging Task Expressions and Search QueriesProceedings of the 2021 Conference on Human Information Interaction and Retrieval10.1145/3406522.3446045(319-323)Online publication date: 14-Mar-2021
https://dl.acm.org/doi/10.1145/3406522.3446045
Sarkar SScholer FThomas PElsweiler DJoho HKando NSmith C(2021)A Context-independent Representation of TaskProceedings of the 2021 Conference on Human Information Interaction and Retrieval10.1145/3406522.3446008(359-362)Online publication date: 14-Mar-2021
https://dl.acm.org/doi/10.1145/3406522.3446008
Shah CWhite RHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Tutorial on Task-Based Search and AssistanceProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401422(2436-2439)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401422
Wu RDeng HTao JFan CLiu QChen Ld'Aquin MDietze SHauff CCurry ECudre Mauroux P(2020)Deep Behavior Tracing with Multi-level Temporality Preserved EmbeddingProceedings of the 29th ACM International Conference on Information & Knowledge Management10.1145/3340531.3412696(2813-2820)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3340531.3412696

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten