research-article

Examining the impact of domain and cognitive complexity on query formulation and reformulation

Authors:

Barbara M. Wildemuth,

Emma Boettcher,

Gergana DimitrovaAuthors Info & Claims

Volume 54, Issue 3

Pages 433 - 450

https://doi.org/10.1016/j.ipm.2018.01.009

Published: 01 May 2018 Publication History

Abstract

The purpose of this analysis was to evaluate an existing set of search tasks in terms of their effectiveness as part of a “shared infrastructure” for conducting interactive IR research. Twenty search tasks that varied in their cognitive complexity and domain were assigned to 47 study participants; the 3,101 moves used to complete those tasks were then analyzed in terms of frequency of each type of move and the sequential patterns they formed. The cognitive complexity of the tasks influenced the number of moves used to complete the tasks, with the most complex (i.e., Create) tasks requiring more moves than tasks at other levels of complexity. Across the four domains, the Commerce tasks elicited more search moves per search. When sequences of moves were analyzed, seven patterns were identified; some of these patterns were associated with particular task characteristics. The findings suggest that search tasks can be designed to elicit particular types of search behaviors and, thus, allow researchers to focus attention on particular aspects of IR interactions.

References

[1]

A. Agresti, Categorical Data Analysis, 2nd edition, John Wiley & Sons, Inc, Hoboken, NJ, 2002.

[2]

D. Albertson, C.III Meadows, Situated topic complexity in interactive video retrieval, Journal of the American Society for Information Science & Technology 62 (9) (2011) 1676–1695.

[3]

L.W. Anderson, D.A.A. Krathwohl, Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom's Taxonomy Of Educational Objectives, (2001) Longman, New York.

[4]

M.J. Bates, Information search tactics, Journal of the American Society for Information Science 30 (4) (1979) 205–214.

[5]

B.S. Bloom, Taxonomy of Educational Objectives, The classification of educational goals, Longmans, Green, New York, 1956.

[6]

P. Borlund, A study of the use of simulated work task situations in interactive information retrieval evaluations: A meta-evaluation, Journal of Documentation 72 (3) (2016) 394–413.

[7]

P. Bruza, S. Dennis, Query reformulation on the internet: Empirical data and the hyperindex search engine, in: Proceedings of Computer-Assisted Information Searching on Internet, RIAO '97, 1997, pp. 488–499.

[8]

K. Byström, P. Hansen, Conceptual framework for tasks in information studies, Journal of the American Society for Information Science & Technology 56 (10) (2005) 1050–1061.

[9]

K. Byström, K. Järvelin, Task complexity affects information seeking and use, Information Processing & Management 31 (2) (1995) 191–213.

[10]

D.J. Campbell, Task complexity: A review and analysis, Academy of Management Review 13 (1988) 40–52.

[11]

R. Fidel, Moves in online searching, Online Review 9 (1) (1985) 61–74.

[12]

R. Fidel, Searchers’ selection of search keys: III. Searching styles, Journal of the American Society for Information Science 42 (7) (1991) 515–527.

[13]

A. Hassan, R.W. White, S.T. Dumais, Wang Y.-M., Struggling or exploring? Disambiguating long search sessions. WSDM ’14, in: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, 2014, pp. 53–62.

[14]

K. Hughes-Morgan, M.L. Wilson, Information vs interaction – Examining difference interaction models over consistent metadata, in: Proceedings of the 4th Symposium on Information Interaction in Context (IIiX), 2012, pp. 72–81.

[15]

P. Ingwersen, K. Järvelin, Information retrieval in context – IriX, SIGIR Forum 39 (2) (2005) 31–39.

[16]

B.J. Jansen, D. Booth, B. Smith, Using the taxonomy of cognitive learning to model online searching, Information Processing & Management 45 (6) (2009) 643–663.

Digital Library

[17]

S. Karimi, F. Scholer, A. Clark, S. Kharazmi, Domain expert topic familiarity and search behavior, in: Proceedings of ACM International Conference on Research and Development in Information Retrieval, SIGIR’11, 2011, pp. 1135–1136.

[18]

D. Kelly, Methods for evaluating interactive information retrieval systems with users, Foundations and Trends in Information Retrieval 3 (1-2) (2009) 1–232.

Digital Library

[19]

D. Kelly, J. Arguello, A. Edwards, Wu W.-C., Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework, in: Proceedings of ACM International Conference on the Theory of Information Retrieval, ICTIR’15, 2015, pp. 101–110.

[20]

D. Kelly, C.R. Sugimoto, A systematic review of interactive information retrieval evaluation studies, 1967-2006, Journal of the American Society for Information Science & Technology 64 (4) (2013) 745–770.

[21]

Kim S., D. Soergel, Selecting and measuring task characteristics as independent variables, in: Proceedings of the ASIST Annual Meeting, 42, 2005, n.p. Retrieved 8/5/2016 from https://doi.org/10.1002/meet.14504201111.

[22]

K. Kinley, D. Tjondronegoro, H. Partridge, S. Edwards, Modeling users' web search behavior and their cognitive styles, Journal of the Association for Information Science & Technology 65 (6) (2014) 1107–1123.

[23]

Lau T., E. Horvitz, Patterns of search: Analyzing and modeling Web query refinement, in: Proceedings of the Seventh International Conference on User Modeling, UM99, 1999, pp. 119–128.

[24]

Li Y., Exploring the relationships between work task and search task in information search, Journal of the American Society for Information Science & Technology 60 (2) (2009) 275–291.

[25]

Li Y., N.J. Belkin, A faceted approach to conceptualizing tasks in information seeking, Information Processing & Management 44 (6) (2008) 1822–1837.

Digital Library

[26]

Liu J., M. Cole, Liu C., R. Bierig, J. Gwizdka, N. Belkin, et al., Search behaviors in different task types, in: Proceedings of the 10th Annual Joint Conference on Digital Libraries, JCDL ’10, 2010, pp. 69–78.

[27]

Liu J., Kim C.S., C. Creel, Exploring search task difficulty reasons in different task types and user knowledge groups, Information Processing & Management 51 (2015) 273–285.

[28]

S. Monchaux, F. Amadieu, A. Chevalier, C. Mariné, Query strategies during information searching: Effects of prior domain knowledge and complexity of the information problems to be solved, Information Processing & Management 51 (5) (2015) 557–569.

Digital Library

[29]

N. Pharo, A. Krahn, The effect of task type on preferred element types in an XML-based retrieval system, Journal of the American Society for Information Science & Technology 62 (9) (2011) 1717–1726.

[30]

Rieh S.Y., Xie H., Analysis of multiple query reformulations on the Web: The interactive information retrieval context, Information Processing and Management 42 (2006) 751–768.

[31]

D. Sharpe, Your chi-square test is statistically significant: Now what?, Practical Assessment, Research & Evaluation 20 (2015) 8. Available online: http://pareonline.net/getvn.asp?v=20&n=8.

[32]

Shaw D.J., R.F. Czaja, User interactions with the PDQ cancer information system, Bulletin of the Medical Library Association 80 (1) (1992) 29–35.

[33]

A.A. Shiri, C. Revie, The effects of topic complexity and familiarity on cognitive and physical moves in a thesaurus-enhanced search environment, Journal of Information Science 29 (6) (2003) 517–526.

[34]

S.J. Shute, P.J. Smith, Knowledge-based search tactics, Information Processing & Management 29 (1) (1993) 29–45.

[35]

R.S. Taylor, Information use environments, B. Dervin, M.J. Voigt (Eds.), Processing in communication Sciences, volume X, Ablex, Norwood, NJ, 1991, pp. 217–255.

[36]

A. Thatcher, Information seeking behaviorscognitive strategies in different search tasks on the WWW, International Journal of Industrial Ergonomics 36 (12) (2006) 1055–1068.

[37]

E. Toms, L. Freund, R. Kopak, J.C. Bartlett, The effect of task domain on search, in: Proceedings of Conference of the Centre for Advanced Studies on Collaborative Research, CASCON ’03, 2003, pp. 303–312. Retrieved December 1, 2017, from https://dl.acm.org/citation.cfm?id=961370.

[38]

N. Vibert, C. Ros, L. Le Bigot, M. Ramond, J. Gatefin, J.-F. Rouet, Effects of domain knowledge on reference search with the PubMed database: An experimental study, Journal of the American Society for Information Science & Technology 60 (7) (2009) 1423–1447.

[39]

N. Wacholder, Interactive query formulation, Annual Review of Information Science & Technology 45 (2011) 157–196.

[40]

B.M. Wildemuth, The effects of domain knowledge on search tactic formulation, Journal of the American Society for Information Science & Technology 55 (3) (2004) 246–258.

[41]

B.M. Wildemuth, L. Freund, E.G. Toms, Untangling search task complexity and difficulty in the context of interactive information retrieval studies, Journal of Documentation 70 (6) (2014) 1118–1140.

[42]

B.M. Wildemuth, M.E. Moore, End-user search behaviors and their relationship to search effectiveness, Bulletin of the Medical Library Association 83 (3) (1995) 294–304.

[43]

Xie I., Joo S., Transitions in search tactics during the Web-based search process, Journal of the American Society for Information Science & Technology 61 (11) (2010) 2188–2205.

[44]

Xie I., Joo S., Factors affecting the selection of search tactics: Tasks, knowledge, process, and systems, Information Processing & Management 48 (2) (2012) 254–270.

Digital Library

[45]

Zhang X., Liu J., M. Cole, N. Belkin, Predicting users' domain knowledge in information retrieval using multiple regression analysis of search behaviors, Journal of the Association for Information Science & Technology 66 (5) (2015) 980–1000.

Cited By

Liu YQin CMa XLi FWang Y(2024)Comparison of information search behavior for different exploratory tasksInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10379461:5Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103794
Li YLiu CHansen P(2023)Incubation and Verification Processes in Information Seeking: A Case Study in the Context of Autonomous LearningProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578289(153-160)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578289
O'Brien HCole AKampen ABrennan KElsweiler DKruschwitz ULudwig B(2022)The Effects of Domain and Search Expertise on Learning Outcomes in Digital Library UseProceedings of the 2022 Conference on Human Information Interaction and Retrieval10.1145/3498366.3505761(202-210)Online publication date: 14-Mar-2022
https://dl.acm.org/doi/10.1145/3498366.3505761
Show More Cited By

Index Terms

Examining the impact of domain and cognitive complexity on query formulation and reformulation
1. Human-centered computing
2. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Index terms have been assigned to the content through auto-classification.

Recommendations

Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework
ICTIR '15: Proceedings of the 2015 International Conference on The Theory of Information Retrieval

One of the most challenging aspects of designing interactive information retrieval (IIR) experiments with users is the development of search tasks. We describe an evaluation of 20 search tasks that were designed for use in IIR experiments and developed ...
Query Reformulation for Task-Oriented Web Searches
WI-IAT '11: Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Volume 03

Web searches are driven by information needs and intend the accomplishment of specific tasks. Information needs are determined by the topical subject of queries, i.e. what we search, while tasks are determined by the user motives that induce the ...
The exploration of objective task difficulty and domain knowledge effects on users' query formulation
ASIST '16: Proceedings of the 79th ASIS&T Annual Meeting: Creating Knowledge, Enhancing Lives through Information & Technology

In this paper, we explore the effects of objective task difficulty and domain knowledge on users' query formulation behaviors. The dataset from a user experiment was used in this research, which focused on the medical domain. The objective difficulty ...

Comments

Information & Contributors

Information

Published In

cover image Information Processing and Management: an International Journal

Information Processing and Management: an International Journal Volume 54, Issue 3

May 2018

117 pages

ISSN:0306-4573

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 May 2018

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu YQin CMa XLi FWang Y(2024)Comparison of information search behavior for different exploratory tasksInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10379461:5Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103794
Li YLiu CHansen P(2023)Incubation and Verification Processes in Information Seeking: A Case Study in the Context of Autonomous LearningProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578289(153-160)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578289
O'Brien HCole AKampen ABrennan KElsweiler DKruschwitz ULudwig B(2022)The Effects of Domain and Search Expertise on Learning Outcomes in Digital Library UseProceedings of the 2022 Conference on Human Information Interaction and Retrieval10.1145/3498366.3505761(202-210)Online publication date: 14-Mar-2022
https://dl.acm.org/doi/10.1145/3498366.3505761
Dosso CMoreno JChevalier ATamine LDemartini GZuccon GCulpepper JHuang ZTong H(2021)CoST: An annotated Data Collection for Complex SearchProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481998(4455-4464)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481998
Xie IBabu RLee HWang SLee T(2021)Orientation tactics and associated factors in the digital library environmentJournal of the Association for Information Science and Technology10.1002/asi.2446972:8(995-1010)Online publication date: 5-Jul-2021
https://dl.acm.org/doi/10.1002/asi.24469
Tamine LMelgarejo JPinel-Sauvagnat K(2020)What Can Task Teach Us About Query Reformulations?Advances in Information Retrieval10.1007/978-3-030-45439-5_42(636-650)Online publication date: 14-Apr-2020
https://dl.acm.org/doi/10.1007/978-3-030-45439-5_42
Chi YHe DJeng W(2020)Laypeople's source selection in online health information‐seeking processJournal of the Association for Information Science and Technology10.1002/asi.2434371:12(1484-1499)Online publication date: 10-Nov-2020
https://dl.acm.org/doi/10.1002/asi.24343
Choi BCapra RArguello JAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)The Effects of Working Memory during Search Tasks of Varying ComplexityProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298948(261-265)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298948
Capra RArguello JO'Brien HLi YChoi BCollins-Thompson KMei QDavison BLiu YYilmaz E(2018)The Effects of Manipulating Task Determinability on Search Behaviors and OutcomesThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval10.1145/3209978.3210047(445-454)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209978.3210047

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents