research-article

Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification

Authors:

Jaime Arguello,

Fernando DiazAuthors Info & Claims

CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval

Pages 5 - 14

https://doi.org/10.1145/3406522.3446021

Published: 14 March 2021 Publication History

Abstract

While current information retrieval systems are effective for known-item retrieval where the searcher provides a precise name or identifier for the item being sought, systems tend to be much less effective for cases where the searcher is unable to express a precise name or identifier. We refer to this as tip of the tongue (TOT) known-item retrieval, named after the cognitive state of not being able to retrieve an item from memory. Using movie search as a case study, we explore the characteristics of questions posed by searchers in TOT states in a community question answering website. We analyze how searchers express their information needs during TOT states in the movie domain. Specifically, what information do searchers remember about the item being sought and how do they convey this information? Our results suggest that searchers use a combination of information about: (1) the content of the item sought, (2) the context in which they previously engaged with the item, and (3) previous attempts to find the item using other resources (e.g., search engines). Additionally, searchers convey information by sometimes expressing uncertainty (i.e., hedging), opinions, emotions, and by performing relative (vs. absolute) comparisons with attributes of the item. As a result of our analysis, we believe that searchers in TOT states may require specialized query understanding methods or document representations. Finally, our preliminary retrieval experiments show the impact of each information type presented in information requests on retrieval performance.

References

[1]

Jaime Arguello, Bogeum Choi, and Robert Capra. 2018. Factors Influencing Users? Information Requests: Medium, Target, and Extra-Topical Dimension. ACM Transactions of Information Systems, Vol. 36, 4 (2018), 37.

Digital Library

[2]

G. H. Bower, S. Thompson-Schill, and E. Tulving. 1994. Reducing Retroactive Interference: An Interference Analysis. Journal of Experimental Psychology: Learning, Memory, and Cognition, Vol. 1, 20 (1994), 51--66.

[3]

Timothy F. Brady, Talia Konkle, George A. Alvarez, and Aude Oliva. 2008. Visual Long-term Memory has a Massive Storage Capacity for Object Details. Proceedings of the National Academy of Sciences, Vol. 105, 38 (2008), 14325--14329.

[4]

Andrei Broder. 2002. A Taxonomy of Web Search. SIGIR Forum, Vol. 36, 2 (2002), 3--10.

Digital Library

[5]

A S Brown. 1991. A Review of the Tip-of-the-tongue Experience. Psychol Bull, Vol. 109, 2 (Mar 1991), 204--223.

[6]

Roger Brown and David McNeill. 1966. The “Tip of the Tongue” Phenomenon. Journal of Verbal Learning and Verbal Behavior, Vol. 5, 4 (1966), 325 -- 337.

[7]

Michael A. Cohen, Todd S. Horowitz, and Jeremy M. Wolfe. 2009. Auditory Recognition Memory is Inferior to Visual Recognition Memory. Proceedings of the National Academy of Sciences, Vol. 106, 14 (2009), 6008--6010.

[8]

Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A Computational Approach to Politeness with Application to Social Factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). ACL, 250--259.

[9]

Doug Downey, Susan Dumais, Dan Liebling, and Eric Horvitz. 2008. Understanding the Relationship between Searchers' Queries and Information Goals. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM '08). ACM, 449--458.

Digital Library

[10]

Susan Dumais, Edward Cutrell, JJ Cadiz, Gavin Jancke, Raman Sarin, and Daniel C. Robbins. 2003. Stuff I've Seen: A System for Personal Information Retrieval and Re-Use. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval (SIGIR '03). ACM, 72--79.

[11]

David Elsweiler, Ian Ruthven, and Christopher Jones. 2005. Dealing with Fragmented Recollection of Context in Information Management. In Proceedings of the 2005 Context-Based Information Retrieval Workshop in Conjunction with the Fifth International and Interdisciplinary Conference on Modeling and Using Context (CIR @ CONTEXT '05), Vol. 151.

[12]

David Elsweiler, Ian Ruthven, and Christopher Jones. 2007. Towards Memory Supporting Personal Information Management Tools. Journal of the American Society for Information Science and Technology, Vol. 58, 7 (2007), 924--946.

Digital Library

[13]

Marguerite Fuller, Liadh Kelly, and Gareth Jones. 2008. Applying Contextual Memory Cues for Retrieval from Personal Information Archives. In Proceedings of the 2008 Personal Information Management Workshop (PIM '08).

[14]

Orit Furman, Nimrod Dorfman, Uri Hasson, Lila Davachi, and Yadin Dudai. 2007. They Saw a Movie: Long-term Memory for an Extended Audiovisual Narrative. Learning & memory, Vol. 14, 6 (2007), 457--467.

[15]

Matthias Hagen, Daniel W"agner, and Benno Stein. 2015. A Corpus of Realistic Known-Item Topics with Associated Web Pages in the ClueWeb09. In Advances in Information Retrieval, Allan Hanbury, Gabriella Kazai, Andreas Rauber, and Norbert Fuhr (Eds.). Springer International Publishing, 513--525.

[16]

Christine Hosey, Lara Vujoviç, Brian St. Thomas, Jean Garcia-Gathright, and Jennifer Thom. 2019. Just Give Me What I Want: How People Use and Evaluate Music Search. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, 1--12.

Digital Library

[17]

Hyeon Kyeong Hwang, Eelco Herder, and Marco Ronchetti. 2017. A Link Back to MemoryLane: The Role of Context in Bookmarking and Refinding. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter . ACM, 8:1--8:10.

Digital Library

[18]

Ida Kathrine Hammeleff Jørgensen and Toine Bogers. 2020. "Kinda like The Sims... But with Ghosts"?: A Qualitative Analysis of Video Game Re-Finding Requests on Reddit. In International Conference on the Foundations of Digital Games. ACM.

Digital Library

[19]

Marcel Just and Hiram Brownell. 1974. Retrieval of Concrete and Abstract Prose Descriptions from Memory. Canadian Journal of Psychology/Revue canadienne de psychologie, Vol. 28 (1974), 339--350.

[20]

Makoto P. Kato, Takehiro Yamamoto, Hiroaki Ohshima, and Katsumi Tanaka. 2014a. Cognitive Search Intents Hidden behind Queries: A User Study on Query Formulations. In Proceedings of the 23rd International Conference on World Wide Web (WWW '14). ACM, 313--314.

Digital Library

[21]

Makoto P. Kato, Takehiro Yamamoto, Hiroaki Ohshima, and Katsumi Tanaka. 2014b. Investigating Users? Query Formulations for Cognitive Search Intents. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR '14). ACM, 577--586.

Digital Library

[22]

Liadh Kelly, Yi Chen, Marguerite Fuller, and Gareth J. F. Jones. 2008. A Study of Remembered Context for Information Access from Personal Digital Archives. In Proceedings of the Second International Symposium on Information Interaction in Context. ACM, 44--50.

[23]

Robert Krovetz. 1993. Viewing morphology as an inference process. In Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '93). ACM, 191--202.

Digital Library

[24]

Sudarshan Lamkhede and Sudeep Das. 2019. Challenges in Search on Streaming Services: Netflix Case Study. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '19). ACM, 1371--1374.

Digital Library

[25]

J. R. Landis and G. G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics, Vol. 33, 1 (1977), 159--174.

[26]

Jin Ha Lee, Allen Renear, and Linda C. Smith. 2006. Known-Item Search: Variations on a Concept. Proceedings of the American Society for Information Science and Technology, Vol. 43, 1 (2006), 1--17.

[27]

Jingjing Liu, Scott Cyphers, Panupong Pasupat, Ian McGraw, and Jim Glass. 2012b. A Conversational Movie Search System Based on Conditional Random Fields. In Proceedings of the 13th Annual Conference of the International Speech Communication Association (INTERSPEECH '12).

[28]

Qiaoling Liu, Eugene Agichtein, Gideon Dror, Yoelle Maarek, and Idan Szpektor. 2012a. When Web Search Fails, Searchers become Askers: Understanding the Transition. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '12). 801--810.

Digital Library

[29]

Janet Metcalfe, Bennett L. Schwartz, and Paul A. Bloom. 2017. The Tip-of-the-tongue State and Curiosity. Cognitive Research: Principles and Implications, Vol. 2, 1 (2017), 31.

[30]

Bhaskar Mitra and Nick Craswell. 2018. An introduction to neural information retrieval. Foundations and Trends® in Information Retrieval, Vol. 13, 1 (2018), 1--126.

[31]

Heather Pon-Barry, Karl Schultz, Elizabeth Owen Bratt, Brady Clark, and Stanley Peters. 2006. Responding to Student Uncertainty in Spoken Tutorial Dialogue Systems. International Journal of Artificial Intelligence in Education, Vol. 16, 2 (2006), 171--194.

Digital Library

[32]

Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends® in Information Retrieval, Vol. 3, 4 (2009), 333--389.

Digital Library

[33]

Stephen Robertson, Hugo Zaragoza, and Michael Taylor. 2004. Simple BM25 extension to multiple weighted fields. Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management (CIKM '04). 42--49.

Digital Library

[34]

David C. Rubin. 1977. Very Long-term Memory for Prose and Verse. Journal of Verbal Learning and Verbal Behavior, Vol. 16, 5 (1977), 611 -- 621.

[35]

David C. Rubin and A. E. Wenzel. 1996. One Hundred Years of Forgetting : A Quantitative Description of Retention. Psychological Review, Vol. 3, 103 (1996), 734--760.

[36]

Michael D. Rugg and Edward L. Wilding. 2000. Retrieval Processing and Episodic Memory. Trends in Cognitive Sciences, Vol. 4 (2000), 108--115.

[37]

Bennett L. Schwartz and Janet Metcalfe. 2011. Tip-of-the-tongue (TOT) States: Retrieval, Behavior, and Experience. Memory & Cognition, Vol. 39, 5 (Jul 2011), 737--749.

[38]

Trevor Strohman, Donald Metzler, Howard Turtle, and W Bruce Croft. 2005. Indri: A Language Model-based Search Engine for complex Queries. In Proceedings of the International Conference on Intelligent Analysis, Vol. 2. Citeseer, 2--6.

[39]

Endel Tulving. 1993. What Is Episodic Memory? Current Directions in Psychological Science, Vol. 2, 3 (1993), 67--70.

[40]

Tom Wilson. 1999. Models in Information Behaviour Research. Journal of Documentation, Vol. 55 (07 1999).

[41]

A. Daniel Yarmey. 1973. I Recognize your Face but I can't Remember your Name: Further Evidence on the Tip-of-the-tongue Phenomenon. Memory & Cognition, Vol. 1, 3 (1973), 287--290.

[42]

Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, and Saurabh Tiwary. 2018. Neural ranking models with multiple document fields. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM '18). 700--708.

Digital Library

Cited By

Yang EJänich TMayfield JLawrie DHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Language Fairness in Multilingual Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657943(2487-2491)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657943
Borges LJha RCallan JMartins BHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Generalizable Tip-of-the-Tongue Retrieval with LLM Re-rankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657917(2437-2441)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657917
Kreutz CSchaer PSchenkel RKlein MBen-David AJäschke RKelly M(2024)Capturing Stability of Information Needs in Digital LibrariesProceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL57899.2023.00011(276-278)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1109/JCDL57899.2023.00011
Show More Cited By

Index Terms

Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
1. Information systems
  1. Information retrieval
    1. Users and interactive retrieval
      1. Search interfaces

Recommendations

When the Music Stops: Tip-of-the-Tongue Retrieval for Music
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

We present a study of Tip-of-the-tongue (ToT) retrieval for music, where a searcher is trying to find an existing music entity, but is unable to succeed as they cannot accurately recall important identifying information. ToT information needs are ...
How Many Neighbours for Known-Item Search?
Similarity Search and Applications
Abstract
In the ongoing multimedia age, search needs become more variable and challenging to aid. In the area of content-based similarity search, asking search engines for one or just a few nearest neighbours to a query does not have to be sufficient to ...
Known-item retrieval on broadcast TV

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHIIR '21: Proceedings of the 2021 Conference on Human Information Interaction and Retrieval

March 2021

384 pages

ISBN:9781450380553

DOI:10.1145/3406522

General Chairs:
Falk Scholer
RMIT University, Australia
,
Paul Thomas
Paul Thomas (Microsoft, Australia
,
Program Chairs:
David Elsweiler
University of Regensburg, Germany
,
Hideo Joho
University of Tsukuba, Japan
,
Noriko Kando
National Institute of Informatics, Japan
,
Catherine Smith
Kent State University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHIIR '21

Sponsor:

CHIIR '21: ACM SIGIR Conference on Human Information Interaction and Retrieval

March 14 - 19, 2021

Canberra ACT, Australia

Acceptance Rates

Overall Acceptance Rate 55 of 163 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
556
Total Downloads

Downloads (Last 12 months)159
Downloads (Last 6 weeks)14

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang EJänich TMayfield JLawrie DHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Language Fairness in Multilingual Information RetrievalProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657943(2487-2491)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657943
Borges LJha RCallan JMartins BHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Generalizable Tip-of-the-Tongue Retrieval with LLM Re-rankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657917(2437-2441)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657917
Kreutz CSchaer PSchenkel RKlein MBen-David AJäschke RKelly M(2024)Capturing Stability of Information Needs in Digital LibrariesProceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL57899.2023.00011(276-278)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1109/JCDL57899.2023.00011
Bogers TGäde MKoolen MPetras VSkov M(2024)Understanding complex casual leisure information needs: an analysis of search requests for books, games, movies and musicJournal of Documentation10.1108/JD-03-2024-007081:1(168-194)Online publication date: 18-Oct-2024
https://doi.org/10.1108/JD-03-2024-0070
Michalkova DRodriguez MMoshfeghi Y(2023)Understanding Feeling-of-Knowing in Information Search: An EEG StudyACM Transactions on Information Systems10.1145/361138442:3(1-30)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3611384
Mysore SMccallum AZamani H(2023)Large Language Model Augmented Narrative Driven RecommendationsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608829(777-783)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608829
Ha JLee DOh C(2023)You Know What I’m Saying: Designing Conversational Strategies of AI Agent for Tip of the Tongue PhenomenonExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585670(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585670
Bhargav SSchuth AHauff CChen HDuh WHuang HKato MMothe JPoblete B(2023)When the Music Stops: Tip-of-the-Tongue Retrieval for MusicProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592086(2506-2510)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592086
Zhe Herng KKhaw YLim SSan T(2023)Application of Convolutional Neural Network (CNN) and Word Embeddings on Plot Summaries for Movie Prediction2023 4th International Conference on Artificial Intelligence and Data Sciences (AiDAS)10.1109/AiDAS60501.2023.10284629(175-179)Online publication date: 6-Sep-2023
https://doi.org/10.1109/AiDAS60501.2023.10284629
Michalkova DRodriguez MMoshfeghi Y(2023)Confidence as Part of Searcher’s Cognitive ContextMachine Learning, Optimization, and Data Science10.1007/978-3-031-25891-6_39(510-524)Online publication date: 10-Mar-2023
https://doi.org/10.1007/978-3-031-25891-6_39
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten