Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3543507.3583343acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

A Passage-Level Reading Behavior Model for Mobile Search

Published: 30 April 2023 Publication History

Abstract

Reading is a vital and complex cognitive activity during users’ information-seeking process. Several studies have focused on understanding users’ reading behavior in desktop search. Their findings greatly contribute to the design of information retrieval models. However, little is known about how users read a result in mobile search, although search currently happens more frequently in mobile scenarios. In this paper, we conduct a lab-based user study to investigate users’ fine-grained reading behavior patterns in mobile search. We find that users’ reading attention allocation is strongly affected by several behavior biases, such as position and selection biases. Inspired by these findings, we propose a probabilistic generative model, the Passage-level Reading behavior Model (PRM), to model users’ reading behavior in mobile search. The PRM utilizes observable passage-level exposure and viewport duration events to infer users’ unobserved skimming event, reading event, and satisfaction perception during the reading process. Besides fitting the passage-level reading behavior, we utilize the fitted parameters of PRM to estimate the passage-level and document-level relevance. Experimental results show that PRM outperforms existing unsupervised relevance estimation models. PRM has strong interpretability and provides valuable insights into the understanding of how users seek and perceive useful information in mobile search.

References

[1]
Klinton Bicknell and Roger Levy. 2010. A rational model of eye movement control in reading. In Proceedings of the 48th annual meeting of the Association for Computational Linguistics. 1168–1178.
[2]
Andrei Broder. 2002. A taxonomy of web search. In ACM Sigir forum, Vol. 36. ACM New York, NY, USA, 3–10.
[3]
George Buchanan and Fernando Loizides. 2007. Investigating document triage on paper and electronic media. In International Conference on Theory and Practice of Digital Libraries. Springer, 416–427.
[4]
Orkut Buyukkokten. 2002. Wireless Web access on handheld devices. Stanford University.
[5]
Andrew Dillon. 1992. Reading from paper versus screens: A critical review of the empirical literature. Ergonomics 35, 10 (1992), 1297–1326.
[6]
Robert L Duchnicky and Paul A Kolers. 1983. Readability of text scrolled on visual display terminals as a function of window size. Human factors 25, 6 (1983), 683–692.
[7]
Ralf Engbert, André Longtin, and Reinhold Kliegl. 2002. A dynamical model of saccade generation in reading based on spatially distributed lexical processing. Vision research 42, 5 (2002), 621–636.
[8]
Ralf Engbert, Antje Nuthmann, Eike M Richter, and Reinhold Kliegl. 2005. SWIFT: a dynamical model of saccade generation during reading.Psychological review 112, 4 (2005), 777.
[9]
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychological bulletin 76, 5 (1971), 378.
[10]
Max Grusky, Jeiran Jahani, Josh Schwartz, Dan Valente, Yoav Artzi, and Mor Naaman. 2017. Modeling sub-document attention using viewport time. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6475–6480.
[11]
Qi Guo and Eugene Agichtein. 2012. Beyond dwell time: estimating document relevance from cursor movements and other post-click searcher behavior. In Proceedings of the 21st international conference on World Wide Web. 569–578.
[12]
Qi Guo, Haojian Jin, Dmitry Lagun, Shuai Yuan, and Eugene Agichtein. 2013. Mining touch interaction data on mobile devices to predict web search result relevance. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. 153–162.
[13]
Michael Hahn and Frank Keller. 2016. Modeling human reading with neural attention. arXiv preprint arXiv:1608.05604 (2016).
[14]
Jeff Huang, Ryen W White, Georg Buscher, and Kuansan Wang. 2012. Improving searcher models using mouse cursor activity. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval. 195–204.
[15]
Kalervo Järvelin and Jaana Kekäläinen. 2000. IR Evaluation Methods for Retrieving Highly Relevant Documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Athens, Greece) (SIGIR ’00). ACM, New York, NY, USA, 41–48. https://doi.org/10.1145/345508.345545
[16]
Matt Jones, George Buchanan, and Harold Thimbleby. 2003. Improving web search on small screen devices. Interacting with computers 15, 4 (2003), 479–495.
[17]
Diane Kelly. 2005. Implicit feedback: Using behavior to infer relevance. In New directions in cognitive information retrieval. Springer, 169–186.
[18]
Diane Kelly and Nicholas J Belkin. 2001. Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval. 408–409.
[19]
Jaewon Kim, Paul Thomas, Ramesh Sankaranarayana, Tom Gedeon, and Hwan-Jin Yoon. 2015. Eye-tracking analysis of user behavior and performance in web search on large and small screens. Journal of the Association for Information Science and Technology 66, 3 (2015), 526–544.
[20]
Dmitry Lagun and Eugene Agichtein. 2011. Viewser: Enabling large-scale remote user studies of web search examination and interaction. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. 365–374.
[21]
Dmitry Lagun, Chih-Hung Hsieh, Dale Webster, and Vidhya Navalpakkam. 2014. Towards better measurement of attention and satisfaction in mobile search. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 113–122.
[22]
J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.
[23]
Xiangsheng Li, Yiqun Liu, Jiaxin Mao, Zexue He, Min Zhang, and Shaoping Ma. 2018. Understanding reading attention distribution during relevance judgement. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 733–742.
[24]
Xiangsheng Li, Jiaxin Mao, Chao Wang, Yiqun Liu, Min Zhang, and Shaoping Ma. 2019. Teach machine how to read: reading behavior inspired relevance estimation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 795–804.
[25]
Yiqun Liu, Chao Wang, Ke Zhou, Jianyun Nie, Min Zhang, and Shaoping Ma. 2014. From skimming to reading: A two-stage examination model for web search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. 849–858.
[26]
Jiaxin Mao, Yiqun Liu, Noriko Kando, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. Investigating result usefulness in mobile search. In European Conference on Information Retrieval. Springer, 223–236.
[27]
Jiaxin Mao, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. Constructing click models for mobile search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 775–784.
[28]
Catherine C Marshall and Frank M Shipman III. 1997. Spatial hypertext and the practice of information triage. In Proceedings of the eighth ACM conference on Hypertext. 124–133.
[29]
Kate Moran. 2016. Reading Content on Mobile Devices. Nielsen Norman Group, https://www.nngroup.com/articles/mobile-content/ (2016).
[30]
Jakob Nielsen. 2011. Mobile content is twice as difficult. Nielsen Norman Group, https://www.nngroup.com/articles/mobile-content-is-twice-as-difficult-2011/ (2011).
[31]
Keith Rayner, Alexander Pollatsek, Jane Ashby, and Charles Clifton Jr. 2012. Psychology of reading. (2012).
[32]
Erik D Reichle, Keith Rayner, and Alexander Pollatsek. 1999. Eye movement control in reading: Accounting for initial fixation locations and refixations within the EZ Reader model. Vision research 39, 26 (1999), 4403–4411.
[33]
Erik D Reichle, Keith Rayner, and Alexander Pollatsek. 2003. The EZ Reader model of eye-movement control in reading: Comparisons to other models. Behavioral and brain sciences 26, 4 (2003), 445–476.
[34]
Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of the 17th International ACM SIGIR Conference on Research and Development in Information Retrieval(SIGIR’94). Springer, 232–241.
[35]
Milad Shokouhi, Rosie Jones, Umut Ozertem, Karthik Raghunathan, and Fernando Diaz. 2014. Mobile query reformulations. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 1011–1014.
[36]
Maximilian Speicher, Andreas Both, and Martin Gaedke. 2013. TellMyRelevance! predicting the relevance of web search results from cursor interactions. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 1281–1290.
[37]
Jan Theeuwes. 1993. Visual selective attention: A theoretical analysis. Acta psychologica 83, 2 (1993), 93–154.
[38]
Adam Trischler, Zheng Ye, Xingdi Yuan, and Kaheer Suleman. 2016. Natural language comprehension with the epireader. arXiv preprint arXiv:1606.02270 (2016).
[39]
Manisha Verma and Emine Yilmaz. 2016. Characterizing relevance on mobile and desktop. In European Conference on Information Retrieval. Springer, 212–223.
[40]
Yizhong Wang, Kai Liu, Jing Liu, Wei He, Yajuan Lyu, Hua Wu, Sujian Li, and Haifeng Wang. 2018. Multi-passage machine reading comprehension with cross-passage answer verification. arXiv preprint arXiv:1805.02220 (2018).
[41]
Ryen W White and Diane Kelly. 2006. A study on the effects of personalization and task information on implicit feedback performance. In Proceedings of the 15th ACM international conference on Information and knowledge management. 297–306.
[42]
Zhijing Wu, Jiaxin Mao, Yiqun Liu, Jingtao Zhan, Yukun Zheng, Min Zhang, and Shaoping Ma. 2020. Leveraging passage-level cumulative gain for document ranking. In Proceedings of The Web Conference 2020. 2421–2431.
[43]
Zhijing Wu, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. Investigating Reading Behavior in Fine-grained Relevance Judgment. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1889–1892.
[44]
Zhi-Jing Wu, Yi-Qun Liu, Jia-Xin Mao, Min Zhang, and Shao-Ping Ma. 2022. Leveraging Document-Level and Query-Level Passage Cumulative Gain for Document Ranking. Journal of Computer Science and Technology 37, 4 (2022), 814–838.
[45]
Chengxiang Zhai and John Lafferty. 2017. A study of smoothing methods for language models applied to ad hoc information retrieval. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 268–276.
[46]
Zhuosheng Zhang, Junjie Yang, and Hai Zhao. 2020. Retrospective reader for machine reading comprehension. arXiv preprint arXiv:2001.09694 (2020).
[47]
Yukun Zheng, Jiaxin Mao, Yiqun Liu, Cheng Luo, Min Zhang, and Shaoping Ma. 2019. Constructing click model for mobile search with viewport time. ACM Transactions on Information Systems (TOIS) 37, 4 (2019), 1–34.
[48]
Yukun Zheng, Jiaxin Mao, Yiqun Liu, Mark Sanderson, Min Zhang, and Shaoping Ma. 2020. Investigating examination behavior in mobile search. In Proceedings of the 13th International Conference on Web Search and Data Mining. 771–779.
[49]
Yukun Zheng, Jiaxin Mao, Yiqun Liu, Zixin Ye, Min Zhang, and Shaoping Ma. 2019. Human behavior inspired machine reading comprehension. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 425–434.

Cited By

View all
  • (2024)Evaluating Generative Ad Hoc Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657849(1916-1929)Online publication date: 10-Jul-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WWW '23: Proceedings of the ACM Web Conference 2023
April 2023
4293 pages
ISBN:9781450394161
DOI:10.1145/3543507
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. document relevance estimation
  2. mobile search
  3. passage ranking
  4. probabilistic generative model
  5. reading behavior

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • the China Postdoctoral Science Foundation

Conference

WWW '23
Sponsor:
WWW '23: The ACM Web Conference 2023
April 30 - May 4, 2023
TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)97
  • Downloads (Last 6 weeks)5
Reflects downloads up to 09 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Evaluating Generative Ad Hoc Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657849(1916-1929)Online publication date: 10-Jul-2024

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media