Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3498366.3505769acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article
Open access

Search Interfaces for Biomedical Searching: How do Gaze, User Perception, Search Behaviour and Search Performance Relate?

Published: 14 March 2022 Publication History

Abstract

The objective of this controlled information retrieval (IR) user experiment is to gain an understanding of domain experts’ interactions with novel search interfaces within the context of biomedical information search, with a goal of better search interface design. In this paper, we examine the relationships among user perception, gaze and search behaviour and user search performance. An eye-tracking study of biomedical domain experts’ interactions with novel search interfaces was conducted. A total of thirty-two users participated and searched for documents answering eight complex exploratory search tasks, using four different search interfaces. The findings suggest that gaze behaviour in terms of fixation durations based measures of areas of interest (AOI), i.e., visual attention to the elements of title, author, abstract and MeSH (Medical Subject Headings) terms in document surrogates is correlated with search performance. Users are more likely to achieve better search performance by precision-based measures when 1) search tasks are perceived as difficult; 2) users attend to the element of abstract; and 3) users can recall using the per-query suggestions during the search processes. More importantly, our findings suggest that a user search interface design that displays contextual information between the suggested keywords and the document may better support users reformulating their queries for complex search tasks in the biomedical domain. We discuss implications for the design of search user interfaces for biomedical searching.

References

[1]
Oswald Barral and et al.2015. Exploring peripheral physiology as a predictor of perceived relevance in information retrieval. In Proceedings of the International Conference on Intelligent User Interfaces (IUI ’15). ACM, New York, 389–399. https://doi.org/10.1145/2678025.2701389
[2]
Nicholas J. Belkin. 2008. Some(what) grand challenges for information retrieval. SIGIR Forum 42, 1 (2008), 47–54. https://doi.org/10.1145/1394251.1394261
[3]
N. J. Belkin, P. G. Marchetti, and C. Cool. 1993. Braque: design of an interface to support user interaction in information retrieval. Information Processing & Management 29, 3 (1993), 325–344. https://doi.org/10.1016/0306-4573(93)90059-m
[4]
Nilavra Bhattacharya, Somnath Rakshit, Jacek Gwizdka, and Paul Kogut. 2020. Relevance prediction from eye-movements using semi-interpretable convolutional neural networks. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval (Vancouver BC, Canada) (CHIIR ’20). Association for Computing Machinery, New York, NY, USA, 223–233. https://doi.org/10.1145/3343413.3377960
[5]
Chris Buckley and Ellen M. Voorhees. 2004. Retrieval Evaluation with Incomplete Information. In Proceedings of the ACM SIGIR Conference (SIGIR ’04). Association for Computing Machinery, New York, NY, USA, 25–32. https://doi.org/10.1145/1008992.1009000
[6]
Ben Carterette. 2011. System Effectiveness, User Models, and User Utility: A Conceptual Framework for Investigation. In Proceedings of the ACM SIGIR Conference (SIGIR ’11). Association for Computing Machinery, New York, NY, USA, 903–912. https://doi.org/10.1145/2009916.2010037
[7]
Jacob Cohen. 1988. Statistical power analysis for the behavioral sciences. L. Erlbaum Associates, Hillsdale, NJ. https://doi.org/10.4324/9780203771587
[8]
Michael J. Cole, Jacek Gwizdka, Chang Liu, Nicholas J. Belkin, and Xiangmin Zhang. 2013. Inferring user knowledge level from eye movement patterns. Information Processing & Management 49, 5 (2013), 1075–1091. https://doi.org/10.1016/j.ipm.2012.08.004
[9]
Nick Craswell. 2009. Bpref. In Encyclopedia of Database Systems, Ling Liu and M. Tamer Özsu (Eds.). Springer, Boston, MA, 266–267. https://doi.org/10.1007/978-0-387-39940-9_489
[10]
Edward Cutrell and Zhiwei Guan. 2007. What are you looking for?: An eye-tracking study of information usage in web search. In Proceedings of the SIGCHI Conference (CHI ’07), Vol. 25. ACM, New York, 407–416. https://doi.org/10.1145/1240624.1240690
[11]
Masoud Davari, Daniel Hienert, Dagmar Kern, and Stefan Dietze. 2020. The role of word-eye-fixations for query term prediction. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval (Vancouver BC, Canada) (CHIIR ’20). Association for Computing Machinery, New York, NY, USA, 422–426. https://doi.org/10.1145/3343413.3378010
[12]
Heiner Deubel and Werner X. Schneider. 1996. Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research 36, 12 (1996), 1827–1837. https://doi.org/10.1016/0042-6989(95)00294-4
[13]
Susan T. Dumais, Georg Buscher, and Edward Cutrell. 2010. Individual Differences in Gaze Patterns for Web Search. In Proceedings of the Symposium on Information Interaction in Context (IIiX ’10), Vol. 3. Association for Computing Machinery, New York, NY, USA, 185–194. https://doi.org/10.1145/1840784.1840812
[14]
Carsten Eickhoff, Sebastian Dungs, and Vu Tran. 2015. An Eye-Tracking Study of Query Reformulation. In Proceedings of the ACM SIGIR Conference (SIGIR ’15). Association for Computing Machinery, New York, NY, USA, 13–22. https://doi.org/10.1145/2766462.2767703
[15]
Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39, 2 (2007), 175–191. https://doi.org/10.3758/bf03193146
[16]
Raya Fidel. 1991. Searchers’ selection of search keys: Ii. Controlled vocabulary or free-text searching. Journal of the American Society for Information Science 42, 7(1991), 501–514. https://doi.org/10.1002/(sici)1097-4571(199108)42:7<501::aid-asi5>3.0.co;2-v
[17]
Raya Fidel. 2012. Human Information Interaction. The MIT Press. https://doi.org/10.7551/mitpress/9780262017008.001.0001
[18]
Ronald Aylmer Fisher. 1935. The design of experiments(9th ed.). Hafner Press, New York. https://doi.org/10.2307/2343406
[19]
Joseph L. Fleiss, Bruce A. Levin, and Myunghee Cho Paik. 2003. Statistical methods for rates and proportions (3rd ed.). John Wiley, Hoboken, NJ. https://doi.org/10.1002/0471445428
[20]
Norbert Fuhr. 2017. Some common mistakes in IR evaluation, and how they can be avoided. SIGIR Forum 51, 3 (2017), 32–41. https://doi.org/10.1145/3190580.3190586
[21]
Joseph H Goldberg and Xerxes P Kotval. 1999. Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics 24, 6(1999), 631–645. https://doi.org/10.1016/s0169-8141(98)00068-7
[22]
Jacek Gwizdka. 2014. Characterizing Relevance with Eye-Tracking Measures. In Proceedings of the Information Interaction in Context Symposium (IIiX ’14), Vol. 5. Association for Computing Machinery, New York, NY, USA, 58–67. https://doi.org/10.1145/2637002.2637011
[23]
Marti Hearst. 2009. Search user interfaces. Cambridge University Press, New York. https://doi.org/10.1017/cbo9781139644082
[24]
William Hersh, Chris Buckley, T. J. Leone, and David Hickam. 1994. OHSUMED: An Interactive Retrieval Evaluation and New Large Test Collection for Research. In Proceedings of the ACM SIGIR Conference (SIGIR ’94), Vol. 17. Springer-Verlag, Berlin, Heidelberg, 192–201. https://doi.org/10.1007/978-1-4471-2099-5_20
[25]
Peter Ingwersen. 1996. Cognitive perspectives of information retrieval interaction: Elements of a cognitive IR theory. Journal of Documentation 52, 1 (1996), 3–50. https://doi.org/10.1108/eb026960
[26]
Kalervo Järvelin, Susan L. Price, Lois M. L. Delcambre, and Marianne Lykke Nielsen. 2008. Discounted Cumulated Gain Based Evaluation of Multiple-Query IR Sessions. In Proceedings of the ECIR ’08. Springer Berlin Heidelberg, Berlin, Heidelberg, 4–15.
[27]
Jiepu Jiang and James Allan. 2016. Correlation Between System and User Metrics in a Session. In Proceedings of the ACM CHIIR Conference (Carrboro, North Carolina, USA). Association for Computing Machinery, New York, NY, USA, 285–288. https://doi.org/10.1145/2854946.2855005
[28]
Yvonne Kammerer and Peter Gerjets. 2012. Effects of search interface and Internet-specific epistemic beliefs on source evaluations during Web search for medical information: An eye-tracking study. Behaviour & Information Technology 31, 1 (2012), 83–97. https://doi.org/10.1080/0144929x.2011.599040
[29]
Evangelos Kanoulas, Ben Carterette, Paul D. Clough, and Mark Sanderson. 2011. Evaluating Multi-Query Sessions. In Proceedings of the ACM SIGIR Conference (SIGIR ’11) (Beijing, China). Association for Computing Machinery, New York, NY, USA, 1053–1062. https://doi.org/10.1145/2009916.2010056
[30]
Max Kemman, Martijn Kleppe, and Jim Maarseveen. 2013. Eye Tracking the Use of a Collapsible Facets Panel in a Search Interface. Lect. Notes Comput. Sci. 8092 (2013), 405–408. https://doi.org/10.1007/978-3-642-40501-3_47
[31]
Suzanne Kieffer. 2017. ECOVAL: Ecological validity of cues and representative design in user experience evaluations. AIS Transactions on Human-Computer Interaction 9, 2(2017), 149–172. https://doi.org/10.17705/1thci.00093
[32]
Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, and Hwan-Jin Yoon. 2015. Eye-tracking analysis of user behavior and performance in web search on large and small screens. Journal of the Association for Information Science and Technology 66, 3(2015), 526–544.
[33]
Roger Kirk. 2012. Experimental Design: Procedures for the Behavioral Sciences (3rd ed.). SAGE. https://doi.org/10.4135/9781483384733
[34]
Bill Kules, Robert Capra, Matthew Banta, and Tito Sierra. 2009. What do exploratory searchers look at in a faceted search interface?. In Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL ’09). ACM, New York, 313–322. https://doi.org/10.1145/1555400.1555452
[35]
Chang Liu, Ying-Hsang Liu, Jingjing Liu, and Ralf Bierig. 2021. Search interface design and evaluation. Foundations and Trends in Information Retrieval 15, 3-4(2021), 243–416. https://doi.org/10.1561/1500000073
[36]
Ying-Hsang Liu and Ralf Bierig. 2014. A Review of Users’ Search Contexts for Lifelogging System Design. In Proceedings of the Information Interaction in Context Symposium (IIiX ’14), Vol. 5. Association for Computing Machinery, New York, NY, USA, 271–274. https://doi.org/10.1145/2637002.2637040
[37]
Ying-Hsang Liu, Paul Thomas, Marijana Bacic, Tom Gedeon, and Xindi Li. 2017. Natural search user interfaces for complex biomedical search: An eye tracking study. Journal of the Australian Library and Information Association 66, 4(2017), 364–381. https://doi.org/10.1080/24750158.2017.1357915
[38]
Ying-Hsang Liu and Nina Wacholder. 2017. Evaluating the impact of MeSH (Medical Subject Headings) terms on different types of searchers. Information Processing & Management 53, 4 (2017), 851–870. https://doi.org/10.1016/j.ipm.2017.03.004
[39]
Lori Lorigo, Maya Haridasan, Hrönn Brynjarsdóttir, Ling Xia, Thorsten Joachims, Geri Gay, Laura Granka, Fabio Pellacini, and Bing Pan. 2008. Eye tracking and online search: Lessons learned and challenges ahead. Journal of the American Society for Information Science & Technology 59, 7(2008), 1041–1052. https://doi.org/10.1002/asi.20794
[40]
Daniel Lüdecke. 2017. Esc: Effect size computation for meta analysis. R package version 0.4.0.
[41]
Marianne Lykke, Susan Price, and Lois Delcambre. 2012. How doctors search: A study of query behaviour and the impact on search results. Information Processing & Management 48, 6 (2012), 1151–1170. https://doi.org/10.1016/j.ipm.2012.02.006
[42]
Gary Marchionini. 2006. Exploratory search: From finding to understanding. Commun. ACM 49, 4 (2006), 41–46. https://doi.org/10.1145/1121949.1121979
[43]
Sandra P Marshall. 2000. Method and apparatus for eye tracking and monitoring pupil dilation to evaluate cognitive activity. US Patent 6,090,051.
[44]
Alistair Moffat and Justin Zobel. 2008. Rank-Biased Precision for Measurement of Retrieval Effectiveness. ACM Transactions on Information Systems 27, 1, Article 2(2008), 27 pages.
[45]
Xi Niu and Diane Kelly. 2014. The Use of Query Suggestions during Information Search. Information Processing & Management 50, 1 (2014), 218–234.
[46]
Miranda Lee Pao, Suzanne F. Grefsheim, Mel L. Barclay, James O. Woolliscroft, Mark McQuillan, and Barbara L. Shipman. 1993. Factors affecting students’ use of MEDLINE. Computers and Biomedical Research 26, 6 (1993), 541–555. https://doi.org/10.1006/cbmr.1993.1038
[47]
Elizabeth R Peterson, Ian J Deary, and Elizabeth J Austin. 2003. The reliability of Riding’s Cognitive Style Analysis test. Personality and Individual Differences 34 (2003), 881–891.
[48]
Tetsuya Sakai. 2007. Alternatives to Bpref. In Proceedings of the ACM SIGIR Conference (SIGIR ’07) (Amsterdam, The Netherlands). Association for Computing Machinery, New York, NY, USA, 71–78.
[49]
Joni Salminen, Bernard J. Jansen, Jisun An, Soon-Gyo Jung, Lene Nielsen, and Haewoon Kwak. 2018. Fixation and Confusion: Investigating Eye-Tracking Participants’ Exposure to Information in Personas. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (New Brunswick, NJ, USA) (CHIIR ’18). Association for Computing Machinery, New York, NY, USA, 110–119. https://doi.org/10.1145/3176349.3176391
[50]
Joni Salminen, Ying-Hsang Liu, Sercan Şengün, João M. Santos, Soon-gyo Jung, and Bernard J. Jansen. 2020. The effect of numerical and textual information on visual engagement and perceptions of AI-driven persona interfaces. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 357–368. https://doi.org/10.1145/3377325.3377492
[51]
Gerard Salton and Michael J. McGill. 1986. Introduction to Modern Information Retrieval. McGraw-Hill, New York.
[52]
Mark Sanderson, Monica Lestari Paramita, Paul Clough, and Evangelos Kanoulas. 2010. Do User Preferences and Evaluation Measures Line Up?. In Proceedings of the ACM SIGIR Conference (SIGIR ’10) (Geneva, Switzerland). Association for Computing Machinery, New York, NY, USA, 555–562.
[53]
Tefko Saracevic, Paul Kantor, Alice Y. Chamis, and Donna Trivison. 1988. A study of information seeking and retrieving. I. Background and methodology. Journal of the American Society for Information Science 39, 3(1988), 161–176.
[54]
Denis Savenkov, Pavel Braslavski, and Mikhail Lebedev. 2011. Search Snippet evaluation at Yandex: Lessons learned and future directions. Lect. Notes Comput. Sci. 6941 (2011), 14–25.
[55]
Ali Shiri and Crawford Revie. 2006. Query expansion behavior within a thesaurus-enhanced search environment: A user-centered evaluation. Journal of the American Society for Information Science & Technology 57, 4(2006), 462–478.
[56]
Moritz Spiller, Ying-Hsang Liu, Md Zakir Hossain, Tom Gedeon, Julia Geissler, and Andreas Nürnberger. 2021. Predicting visual search task success from eye gaze data as a basis for user-adaptive information visualization systems. ACM Transactions on Interactive Intelligent Systems 11, 2, Article 14 (2021), 25 pages. https://doi.org/10.1145/3446638
[57]
Muh-Chyun Tang, Ying-Hsang Liu, and Wan-Ching Wu. 2013. A study of the influence of task familiarity on user behaviors and performance with a MeSH term suggestion interface for bibliographic search. International Journal of Medical Informatics 82, 9(2013), 832–843. https://doi.org/10.1016/j.ijmedinf.2013.04.005
[58]
Nina Wacholder. 2011. Interactive query formulation. Annual Review of Information Science and Technology 45 (2011), 157–196.
[59]
Peter Wittek, Ying-Hsang Liu, Sándor Darányi, Tom Gedeon, and Ik Soo Lim. 2016. Risk and ambiguity in information seeking: Eye gaze patterns reveal contextual behavior in dealing with uncertainty. Frontiers in Psychology 7 (2016), 1790.
[60]
Theodore B Wright, David Ball, and William Hersh. 2017. Query expansion using MeSH terms for dataset retrieval: Ohsu at the bioCADDIE 2016 dataset retrieval challenge. Database 2017(2017).
[61]
Yan Zhang, Ramona Broussard, Weimao Ke, and Xuemei Gong. 2014. Evaluation of a scatter/gather interface for supporting distinct health information search tasks. Journal of the Association for Information Science & Technology 65, 5(2014), 1028–1041.

Cited By

View all
  • (2024)Exploratory search in information systems: a systematic reviewThe Electronic Library10.1108/EL-11-2023-026442:2(308-339)Online publication date: 14-Feb-2024
  • (2024)Comparison of information search behavior for different exploratory tasksInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10379461:5Online publication date: 1-Sep-2024
  • (2024)Understanding users' dynamic perceptions of search gain and cost in sessions: An expectation confirmation modelJournal of the Association for Information Science and Technology10.1002/asi.24935Online publication date: 17-Jun-2024
  • Show More Cited By

Index Terms

  1. Search Interfaces for Biomedical Searching: How do Gaze, User Perception, Search Behaviour and Search Performance Relate?
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Information & Contributors

            Information

            Published In

            cover image ACM Conferences
            CHIIR '22: Proceedings of the 2022 Conference on Human Information Interaction and Retrieval
            March 2022
            399 pages
            ISBN:9781450391863
            DOI:10.1145/3498366
            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Sponsors

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            Published: 14 March 2022

            Permissions

            Request permissions for this article.

            Check for updates

            Author Tags

            1. Exploratory search
            2. MeSH (Medical Subject Headings)
            3. Query suggestion
            4. Search interfaces
            5. Session-based evaluation

            Qualifiers

            • Research-article
            • Research
            • Refereed limited

            Funding Sources

            Conference

            CHIIR '22
            Sponsor:

            Acceptance Rates

            Overall Acceptance Rate 55 of 163 submissions, 34%

            Contributors

            Other Metrics

            Bibliometrics & Citations

            Bibliometrics

            Article Metrics

            • Downloads (Last 12 months)200
            • Downloads (Last 6 weeks)18
            Reflects downloads up to 01 Jan 2025

            Other Metrics

            Citations

            Cited By

            View all
            • (2024)Exploratory search in information systems: a systematic reviewThe Electronic Library10.1108/EL-11-2023-026442:2(308-339)Online publication date: 14-Feb-2024
            • (2024)Comparison of information search behavior for different exploratory tasksInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10379461:5Online publication date: 1-Sep-2024
            • (2024)Understanding users' dynamic perceptions of search gain and cost in sessions: An expectation confirmation modelJournal of the Association for Information Science and Technology10.1002/asi.24935Online publication date: 17-Jun-2024
            • (2023)Study of the Effect of Laser Radiation Parameters on the Efficiency of LithotripsyApplied Sciences10.3390/app1315856513:15(8565)Online publication date: 25-Jul-2023
            • (2023)How Data Scientists Review the Scholarly LiteratureProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578309(137-152)Online publication date: 19-Mar-2023
            • (2023)Online subject searching of humanities PhD students at a Swedish universityJournal of Documentation10.1108/JD-03-2023-004479:7(308-329)Online publication date: 30-Oct-2023
            • (2023)User Evaluation of Conversational Agents for Aerospace DomainInternational Journal of Human–Computer Interaction10.1080/10447318.2023.223954440:19(5549-5568)Online publication date: 2-Aug-2023
            • (2023)Development of a Visual Search Service Effectiveness Scale for Assessing Image Search Effectiveness: A Behavioral and Technological PerspectiveInternational Journal of Human–Computer Interaction10.1080/10447318.2023.219753540:14(3717-3731)Online publication date: 17-Apr-2023

            View Options

            View options

            PDF

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format.

            HTML Format

            Login options

            Media

            Figures

            Other

            Tables

            Share

            Share

            Share this Publication link

            Share on social media