article

The effect of user characteristics on search effectiveness in information retrieval

Authors:

Azzah Al-Maskari,

Mark SandersonAuthors Info & Claims

Information Processing and Management: an International Journal, Volume 47, Issue 5

Pages 719 - 729

https://doi.org/10.1016/j.ipm.2011.03.002

Published: 01 September 2011 Publication History

Abstract

This paper investigates the influence of user characteristics (e.g. search experience and cognitive skills) on user effectiveness. A user study was conducted to investigate this effect, 56 participants completed searches for 56 topics using the TREC test collection. Results indicated that participants with search experience and high cognitive skills were more effective than those with less experience and slower perceptual abilities. However, all users rated themselves with the same level of satisfaction with the search results despite the fact they varied substantially in their effectiveness. Therefore, information retrieval evaluators should take these factors into consideration when investigating the impact of system effectiveness on user effectiveness.

References

[1]

Allan, J., Carterette, B., Lewis, J. (2005). When will information retrieval be ''good enough"? Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. New York, NY, USA: ACM. 433-440.

[2]

Allen, B. (1992). Cognitive differences in end user searching of a CD-ROM index. In Proceedings of the 15th annual international ACM SIGIR conference on research and development in information retrieval (pp. 298-309). Copenhagen, Denmark: ACM.

[3]

Al-Maskari, A., Sanderson, M., & Clough, P. (2008). The good and the bad system: Does the test collection predict users' effectiveness. In Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval (pp. 59-66). Singapore: ACM.

[4]

A review of factors influencing user satisfaction in information retrieval. Journal of the American Society for Information Science and Technology. v61 i5. 859-868.

[5]

Analyzing Social Science Data. Sage Publications Inc., London.

[6]

The effects and interaction of spatial visualization and domain expertise on information seeking. Computers in Human Behavior. v21 i2. 195-209.

[7]

Individual differences in human-computer interaction. In: Helander, M.G., Landauer, T.K., Prabhu, P.V. (Eds.), Handbook of human-computer interaction, Elsevier, Amsterdam, Holland.

[8]

Manual for kit of factor-referenced cognitive tests. Educational Testing Service, Princeton, NJ.

[9]

Evaluation issues in information retrieval. Information Processing and Management. v28 i4. 439-440.

[10]

Performances of 27 MEDLINE systems tested by searches with clinical questions. Journal of the American Medical Informatics Association. v1 i3. 285-295.

[11]

Factors associated with successful answering of clinical questions using an information retrieval system. Bulletin of the Medical Library Association. v88 i4. 323-331.

[12]

The turn: Integration of information seeking and retrieval in context. Springer.

[13]

Juvina, I., & van Oostendorp, H. (2004). Individual Differences and Behavioral Aspects Involved in Modeling Web Navigation. In Proceedings in the 8th ERCIM workshop "user interfaces for all" (pp. 77-95). Vienna, Austria: Springer.

[14]

Kelly, D., & Cool, C. (2002). The effects of topic familiarity on information search behavior. In Proceedings of the 2nd ACM/IEEE-CS joint conference on digital libraries (pp. 74-75). ACM.

[15]

Cognitive and task influences on Web searching behavior. Journal of the American Society for Information Science and Technology. v53 i2. 109-119.

[16]

Searching databases on CD-ROM: Comparison of the results of end-user searching with results from two modes of searching by skilled intermediaries. RQ. v33 i3. 370-386.

[17]

Differences between novice and experienced users in searching information on the World Wide Web. Journal of the American Society for Information Science. v51 i6. 576-581.

[18]

Information seeking in electronic environments. Cambridge University Press.

[19]

Navigation in hyperspace: An evaluation of the effects of navigational tools and subject matter expertise on browsing and information retrieval in hypertext. Interacting with Computers. v10 i2. 129-142.

[20]

Usability engineering. Morgan Kaufmann, San Francisco.

[21]

Cognitive style and on-line database search experience as predictors of Web search performance. Journal of the American Society for Information Science. v51 i6. 558-566.

[22]

Papaeconomou, C., Zijlema, A. F., & Ingwersen, P. (2008). Searchers' relevance judgments and criteria in evaluating web pages in a learning style perspective. In Proceedings of the second international symposium on information interaction in context (pp. 123-132). ACM.

[23]

Effectiveness of expert semantic knowledge as a navigational aid within hypertext. Behavior and Information Technology. v17 i6. 313-324.

[24]

Saito, H., & Miwa, K. (2001). A cognitive study of information seeking processes in the WWW: The effects of searcher's knowledge and experience. In Proceedings of the second international conference on web information systems engineering (WISE'01) (Vol. 1, pp. 321-327). Washington, DC, USA: IEEE Computer Society.

[25]

The state of retrieval system evaluation. Information Processing and Management. v28 i4. 441-449.

[26]

Smith, C. L., & Kantor, P. B. (2008). User adaptation: Good results from poor systems. In Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval (pp. 147-154). Singapore: ACM.

[27]

End-users, mediated searches, and front-end assistance programs on dialog: A comparison of learning, performance, and satisfaction. Journal of the American Society for Information Science. v41 i1. 27-42.

[28]

Why batch and user evaluations do not give the same results. Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, New York, NY, USA.

[29]

User performance versus precision measures for simple search tasks. Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, New York, NY, USA.

[30]

Voorhees, E. M. (2001). The philosophy of information retrieval evaluation. Evaluation of cross-language information retrieval systems, second workshop of the cross-language evaluation forum, CLEF, pp. 9-26.

Cited By

Azzopardi LMaxwell DHalvey MHauff C(2023)Driven to Distraction: Examining the Influence of Distractors on Search Behaviours, Performance and ExperienceProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578298(83-94)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578298
Roy NBalayn AMaxwell DHauff CChen HDuh WHuang HKato MMothe JPoblete B(2023)Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance AssessmentsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591694(718-728)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591694
Choi BArguello JCapra RWard A(2023)The Influences of a Knowledge Representation Tool on Searchers with Varying Cognitive AbilitiesACM Transactions on Information Systems10.1145/352766141:1(1-35)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3527661
Show More Cited By

Index Terms

The effect of user characteristics on search effectiveness in information retrieval
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Information systems
  1. Information retrieval

Index terms have been assigned to the content through auto-classification.

Recommendations

Chinese word segmentation and its effect on information retrieval

A set of IR experiments was carried out to study the impact of Chinese word segmentation and its effect on information retrieval (IR) at the Division of Information Studies, Nanyang Technological University, Singapore. A total of four automatic ...
On the potential search effectiveness of MeSH (medical subject headings) terms
IIiX '10: Proceedings of the third symposium on Information interaction in context

The creation of innovative techniques of document representation is critical to the development of effective information retrieval (IR) systems. In this paper, we report on the impact of state-of-the-art human indexing techniques, exemplified by MeSH (...
A domain-driven approach to improving search effectiveness in traditional online catalogs

Search performance can be greatly improved by using domain knowledge to assist users in developing a problem specification tailored to the information contained in the system. A methodology is presented for utilizing intelligent information retrieval ...

Comments

Information & Contributors

Information

Published In

cover image Information Processing and Management: an International Journal

Information Processing and Management: an International Journal Volume 47, Issue 5

September, 2011

173 pages

ISSN:0306-4573

Issue’s Table of Contents

Copyright © Elsevier Ltd © 2011.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 September 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
1
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Azzopardi LMaxwell DHalvey MHauff C(2023)Driven to Distraction: Examining the Influence of Distractors on Search Behaviours, Performance and ExperienceProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578298(83-94)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578298
Roy NBalayn AMaxwell DHauff CChen HDuh WHuang HKato MMothe JPoblete B(2023)Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance AssessmentsProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591694(718-728)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591694
Choi BArguello JCapra RWard A(2023)The Influences of a Knowledge Representation Tool on Searchers with Varying Cognitive AbilitiesACM Transactions on Information Systems10.1145/352766141:1(1-35)Online publication date: 25-Feb-2023
https://dl.acm.org/doi/10.1145/3527661
Ibrahim OYounis E(2022)Hybrid online–offline learning to rank using simulated annealing strategy based on dependent click modelKnowledge and Information Systems10.1007/s10115-022-01726-064:10(2833-2847)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10115-022-01726-0
Foulds OSuglia AAzzopardi LHalvey MHuang JChang YCheng XKamps JMurdock VWen JLiu Y(2020)Predicting Perceptual Speed from Search BehaviourProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401210(1989-1992)Online publication date: 25-Jul-2020
https://dl.acm.org/doi/10.1145/3397271.3401210
Foulds OAzzopardi LHalvey MO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)Reflecting upon Perceptual Speed Tests in Information RetrievalProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377982(234-242)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377982
Conati CLallé SRahman MToker D(2020)Comparing and Combining Interaction Data and Eye-tracking Data for the Real-time Prediction of User Cognitive Abilities in Visualization TasksACM Transactions on Interactive Intelligent Systems10.1145/330140010:2(1-41)Online publication date: 30-May-2020
https://dl.acm.org/doi/10.1145/3301400
Jin YTintarev NHtun NVerbert K(2020)Effects of personal characteristics in control-oriented user interfaces for music recommender systemsUser Modeling and User-Adapted Interaction10.1007/s11257-019-09247-230:2(199-249)Online publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1007/s11257-019-09247-2
Liu JLiu CBelkin N(2020)Personalization in text information retrievalJournal of the Association for Information Science and Technology10.1002/asi.2423471:3(349-369)Online publication date: 28-Jan-2020
https://dl.acm.org/doi/10.1002/asi.24234
Arguello JChoi B(2019)The Effects of Working Memory, Perceptual Speed, and Inhibition in Aggregated SearchACM Transactions on Information Systems10.1145/332212837:3(1-34)Online publication date: 16-May-2019
https://dl.acm.org/doi/10.1145/3322128
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents