Report on the CHIIR 2021 third workshop on evaluation of personalisation in information retrieval (WEPIR 2021)
Article No.: 7, Pages 1 - 11
Abstract
The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction & Retrieval (CHIIR 2021) in Canberra, Australia, as a virtual event. WEPIR 2021 followed on from the first and second WEPIRs held at CHIIR 2018 and 2019. The purpose of the workshop was again to bring together researchers from different backgrounds, interested in advancing the evaluation of personalisation in information retrieval. The workshop focused on further development of a common understanding of the challenges, requirements and practical limitations of personalisation in information retrieval and its evaluation.
References
[1]
Maristella Agosti, Norbert Fuhr, Elaine Toms, and Perti Vakkari. Evaluation methodologies in information retrieval. Dagstuhl Seminar 13441, Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Schloss Dagstuhl, Germany, 2014.
[2]
Nicholas J. Belkin, Susan Dumais, Noriko Kando, and Mark Sanderson. Whole-session evaluation of interactive information retrieval systems. NII Shonan Meeting Report 2012-7, National Institute of Informatics, Japan, Tokyo, Japan, 2016.
[3]
Ben Carterette, Evangelos Kanoulos, Mark Hall, and Peter Clough. Overview of the trec 2014 session track. In Ellen M. Voorheen, editor, Proceedings of the 23rd Text REtrieval Conference, TREC 2014, Gaithersburg, MD, 2015. National Institute of Standards and Technology. URL http://trec.nist.gov/pubs/trec23/papers/overview-session.pdf.
[4]
Owen Conlan, Liadh Kelly, Kevin Koidl, Seamus Lawless, Killian Levacher, and Athanasios Staikopoulos, editors. EvalUMAP2016: Towards Comparative Evaluation in the User Modelling, Adaptation and Personalization, Halifax, Canada, 2016.
[5]
Susan Dumais and Nicholas J. Belkin. The TREC interactive tracks: Putting the user into search. In Ellen M. Voorhees and Donna K. Harman, editors, TREC. Experiment and evaluation in information retrieval, pages 123 -- 152. MIT Press, Cambridge, MA, 2005.
[6]
John H Falk and Lynn D Dierking. Learning from museums. Rowman & Littlefield, 2018.
[7]
Yasin Ghafourian, Petr Knoth, and Allan Hanbury. Information retrieval evaluation in knowledge acquisition tasks. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 88 -- 95, Canberra, Australia, Virtual event, 2021.
[8]
Gareth J. F. Jones, Nicholas J. Belkin, Seamus Lawless, and Gabriella Pasi. Report on the CHIIR 2018 workshop on evaluation of personalisation in information retrieval (WEPIR 2018). SIGIR Forum, 52(1):129--134, 2018a.
[9]
Gareth J. F. Jones, Nicholas J. Belkin, Seamus Lawless, and Gabriella Pasi. Report on the CHIIR 2018 workshop on evaluation of personalisation in information retrieval (WEPIR 2019). SIGIR Forum, 53(1):29--37, 2018b.
[10]
Evangelos Kanoulas, Ben Carterette, Paul D. Clough, and Mark Sanderson. Evaluating multi-query sessions. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, pages 1053--1062, Beijing, China, 2011. ACM.
[11]
Gabriella Pasi, Gareth J. F. Jones, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Prochzta Sen. Overview of the CLEF 2017 personalised information retrieval pilot lab (PIRCLEF 2017). In Proceedings of CLEF 2017, Dublin, Ireland, 2017. Springer.
[12]
Gabriella Pasi, Gareth J. F. Jones, Keith Curtis, Stefania Marrara, Camilla Sanvitto, Debasis Ganguly, and Prochzta Sen. Overview of the CLEF 2018 personalised information retrieval pilot lab (PIR-CLEF 2018). In Proceedings of CLEF 2018, Avignon, France, 2018. Springer.
[13]
Gabriella Pasi, Gareth J. F. Jones, Lorraine Goeuriot, Liadh Kelly, Stefania Marrara, and Camilla Sanvitto. Overview of the CLEF 2019 personalised information retrieval lab (2019). In Proceedings of CLEF 2019, pages 417--424, Lugano, Switzerland, 2019. Springer.
[14]
Camilla Sanvitto, Debasis Ganguly, Gareth J. F. Jones, and Gabriella Pasi. A laboratory-based method for the evaluation of personalised search. In Emine Yilmaz and Charles L. A. Clarke, editors, Proceedings of The Seventh International Workshop on Evaluating Information Access - A Satellite Workshop of NTCIR-12, pages 13 -- 16, Tokyo, Japan, 2016.
[15]
Yoshiyuki Shoji, Kenro Aihara, Martin J. Dürst, Noriko Kando, Takuya Nakaya, Hiroaki Ohshima, Takehiro Yamamoto, and Yusuke Yamamoto. Location-based reminder for memorizing what visitors learn at a museum. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 79 -- 87, Canberra, Australia, Virtual event, 2021.
[16]
Masaki Ueta, Tomoya Hashiguchi, Hun-Long Pham, Yoshiyuki Shoji, Noriko Kando, Yusuke Yamamoto, Takehiro Yamamoto, and Hiroaki Ohshima. Quiz generation on the electronic guide application for improving learning experience in the museum. In Proceedings of the Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021), pages 96 -- 104, Canberra, Australia, Virtual event, 2021.
- Report on the CHIIR 2021 third workshop on evaluation of personalisation in information retrieval (WEPIR 2021)
Recommendations
Report on the future conversations workshop at CHIIR 2021
The Future Conversations workshop at CHIIR'21 looked to the future of search, recommendation, and information interaction to ask: where are the opportunities for conversational interactions? What do we need to do to get there? Furthermore, who stands to ...
Report on the CHIIR 2019 Second Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2019)
The Second Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2019) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction & Retrieval (CHIIR 2019) in Glasgow, Scotland. WEPIR 2019 followed on from ...
Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2020): In Memoriam Seamus Lawless
CHIIR '20: Proceedings of the 2020 Conference on Human Information Interaction and RetrievalThe Third WEPIR 2020 workshop builds on the success of the first two WEPIR meetings held at CHIIR 2018 and CHIIR 2019. WEPIR 2020 again brings together researchers from different backgrounds interested in continuing to explore and advance the evaluation ...
Comments
Information & Contributors
Information
Published In
Copyright © 2021 Copyright is held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 July 2021
Published in SIGIR Volume 55, Issue 1
Check for updates
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 73Total Downloads
- Downloads (Last 12 months)7
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in