demonstration

Fairness-aware Recommendation with librec-auto

Authors:

Masoud MansouryAuthors Info & Claims

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

Pages 594 - 596

https://doi.org/10.1145/3383313.3411525

Published: 22 September 2020 Publication History

Abstract

Comparative experimentation is important for studying reproducibility in recommender systems. This is particularly true in areas without well-established methodologies, such as fairness-aware recommendation. In this paper, we describe fairness-aware enhancements to our recommender systems experimentation tool librec-auto. These enhancements include metrics for various classes of fairness definitions, extension of the experimental model to support result re-ranking and a library of associated re-ranking algorithms, and additional support for experiment automation and reporting. The associated demo will help attendees move quickly to configuring and running their own experiments with librec-auto.

References

[1]

Himan Abdollahpouri, Gediminas Adomavicius, Robin Burke, Ido Guy, Dietmar Jannach, Toshihiro Kamishima, Jan Krasnodebski, and Luiz Pizzato. 2020. Multistakeholder recommendation: Survey and research directions. User Modeling and User-Adapted Interaction 30, 1 (2020), 127–158.

[2]

Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev. 104(2016), 671.

[3]

Joeran Beel, Corinna Breitinger, Stefan Langer, Andreas Lommatzsch, and Bela Gipp. 2016. Towards reproducibility in recommender-systems research. User modeling and user-adapted interaction 26, 1 (2016), 69–101.

[4]

Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H Chi, 2019. Fairness in recommendation ranking through pairwise comparisons. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2212–2220.

Digital Library

[5]

Dan Biddle. 2006. Adverse impact and test validation: A practitioner’s guide to valid and defensible employment testing. Gower Publishing, Ltd.

[6]

Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In The 41st international acm sigir conference on research & development in information retrieval. 405–414.

[7]

Robin Burke. 2017. Multisided Fairness for Recommendation. In Workshop on Fairness, Accountability and Transparency in Machine Learning (FATML). Halifax, Nova Scotia.

[8]

Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. 335–336.

Digital Library

[9]

Carlos Castillo. 2019. Fairness and transparency in ranking. In ACM SIGIR Forum, Vol. 52. ACM New York, NY, USA, 64–71.

[10]

Michael D Ekstrand. 2018. The LKPY package for recommender systems experiments: Next-generation tools and lessons learned from the LensKit project. arXiv preprint arXiv:1809.03125(2018).

[11]

Guibing Guo, Jie Zhang, Zhu Sun, and Neil Yorke-Smith. 2015. LibRec: A Java Library for Recommender Systems. In UMAP Workshops, Vol. 4.

[12]

Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. 2011. Fairness-aware learning through regularization approach. In 2011 IEEE 11th International Conference on Data Mining Workshops. IEEE, 643–650.

Digital Library

[13]

Caitlin Kuhlman, MaryAnn VanValkenburg, and Elke Rundensteiner. 2019. FARE: Diagnostics for Fair Ranking using Pairwise Error Metrics. In The World Wide Web Conference. 2936–2942.

Digital Library

[14]

Weiwen Liu, Jun Guo, Nasim Sonboli, Robin Burke, and Shengyu Zhang. 2019. Personalized Fairness-Aware Re-Ranking for Microlending. In Proceedings of the 13th ACM Conference on Recommender Systems (Copenhagen, Denmark) (RecSys ’19). Association for Computing Machinery, New York, NY, USA, 467–471. https://doi.org/10.1145/3298689.3347016

Digital Library

[15]

Masoud Mansoury, Robin Burke, Aldo Ordonez-Gauger, and Xavier Sepulveda. 2018. Automating recommender systems experimentation with librec-auto. In Proceedings of the 12th ACM Conference on Recommender Systems. 500–501.

Digital Library

[16]

Steffen Rendle. 2012. Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology (TIST) 3, 3(2012), 1–22.

Digital Library

[17]

Ya’acov Ritov, Yuekai Sun, and Ruofei Zhao. 2017. On conditional parity as a notion of non-discrimination in machine learning. arXiv preprint arXiv:1706.08519(2017).

[18]

Rodrygo LT Santos, Jie Peng, Craig Macdonald, and Iadh Ounis. 2010. Explicit search result diversification through sub-queries. In European conference on information retrieval. Springer, 87–99.

Digital Library

[19]

Ashudeep Singh and Thorsten Joachims. 2018. Fairness of exposure in rankings. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2219–2228.

Digital Library

[20]

Nasim Sonboli, Farzad Eskandanian, Robin Burke, Weiwen Liu, and Bamshad Mobasher. 2020. Opportunistic Multi-aspect Fairness through Personalized Re-ranking. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. to appear.

Digital Library

[21]

Harald Steck. 2018. Calibrated recommendations. In Proceedings of the 12th ACM conference on recommender systems. 154–162.

Digital Library

[22]

Virginia Tsintzou, Evaggelia Pitoura, and Panayiotis Tsaparas. 2018. Bias disparity in recommendation systems. arXiv preprint arXiv:1811.01461(2018).

[23]

Ke Yang and Julia Stoyanovich. 2017. Measuring fairness in ranked outputs. In Proceedings of the 29th International Conference on Scientific and Statistical Database Management. 1–6.

Digital Library

[24]

Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collaborative filtering. In Advances in Neural Information Processing Systems. 2921–2930.

[25]

Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017. Fa* ir: A fair top-k ranking algorithm. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1569–1578.

Digital Library

[26]

Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In International Conference on Machine Learning. 325–333.

[27]

Shuai Zhang, Yi Tay, Lina Yao, Bin Wu, and Aixin Sun. 2019. Deeprec: An open-source toolkit for deep learning based recommendation. arXiv preprint arXiv:1905.10536(2019).

[28]

Cai-Nicolas Ziegler, Sean M McNee, Joseph A Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In Proceedings of the 14th international conference on World Wide Web. 22–32.

Digital Library

Cited By

Awal GTehlan U(2024)Mapping the research landscape of recommender systems for digital librariesRecord and Library Journal10.20473/rlj.V10-I1.2024.180-19410:1(180-194)Online publication date: 22-Jun-2024
https://doi.org/10.20473/rlj.V10-I1.2024.180-194
Cachel KRundensteiner ESerra ESpezzano F(2024)FairRankTune: A Python Toolkit for Fair Ranking TasksProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679238(5195-5199)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679238
Bauer CChung LCornelissen Avan Driessel Ivan der Hoorn Dde Jong YLe LNajiyan Tabriz SSpaans RThijsen CVerbeeten RWesseling VWieland F(2023)FairRecKit: A Web-based Analysis Software for Recommender EvaluationsProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578274(438-443)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578274
Show More Cited By

Recommendations

User-oriented Fairness in Recommendation
WWW '21: Proceedings of the Web Conference 2021

As a highly data-driven application, recommender systems could be affected by data bias, resulting in unfair results for different data groups, which could be a reason that affects the system performance. Therefore, it is important to identify and ...
Experimentation with fairness-aware recommendation using librec-auto: hands-on tutorial
FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency

The field of machine learning fairness has developed metrics, methodologies, and data sets for experimenting with classification algorithms. However, equivalent research is lacking in the area of personalized recommender systems. This 180-minute hands-...
Automating recommender systems experimentation with librec-auto
RecSys '18: Proceedings of the 12th ACM Conference on Recommender Systems

Recommender systems research often requires the creation and execution of large numbers of algorithmic experiments to determine the sensitivity of results to the values of various hyperparameters. Existing recommender systems platforms fail to provide a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

September 2020

796 pages

ISBN:9781450375832

DOI:10.1145/3383313

Copyright © 2020 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 September 2020

Check for updates

Author Tags

Qualifiers

Demonstration
Research
Refereed limited

Conference

RecSys '20

Sponsor:

RecSys '20: Fourteenth ACM Conference on Recommender Systems

September 22 - 26, 2020

Virtual Event, Brazil

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)3

Reflects downloads up to 20 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Awal GTehlan U(2024)Mapping the research landscape of recommender systems for digital librariesRecord and Library Journal10.20473/rlj.V10-I1.2024.180-19410:1(180-194)Online publication date: 22-Jun-2024
https://doi.org/10.20473/rlj.V10-I1.2024.180-194
Cachel KRundensteiner ESerra ESpezzano F(2024)FairRankTune: A Python Toolkit for Fair Ranking TasksProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679238(5195-5199)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679238
Bauer CChung LCornelissen Avan Driessel Ivan der Hoorn Dde Jong YLe LNajiyan Tabriz SSpaans RThijsen CVerbeeten RWesseling VWieland F(2023)FairRecKit: A Web-based Analysis Software for Recommender EvaluationsProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578274(438-443)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578274
Smith JBeattie LCramer H(2023)Scoping Fairness Objectives and Identifying Fairness Metrics for Recommender Systems: The Practitioners’ PerspectiveProceedings of the ACM Web Conference 202310.1145/3543507.3583204(3648-3659)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583204
Sonboli NMansoury MGuo ZKadekodi SLiu WLiu ZSchwartz ABurke RDemartini GZuccon GCulpepper JHuang ZTong H(2021)librec-auto: A Tool for Recommender Systems ExperimentationProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482006(4584-4593)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482006
Schelenz L(2021)Diversity-aware Recommendations for Social Justice? Exploring User Diversity and Fairness in Recommender SystemsAdjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3450614.3463293(404-410)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3450614.3463293
Bellogín ASaid A(2021)Improving accountability in recommender systems research through reproducibilityUser Modeling and User-Adapted Interaction10.1007/s11257-021-09302-x31:5(941-977)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1007/s11257-021-09302-x
Ekstrand MDas ABurke RDiaz F(2012)Fairness in Recommender SystemsRecommender Systems Handbook10.1007/978-1-0716-2197-4_18(679-707)Online publication date: 24-Feb-2012
https://doi.org/10.1007/978-1-0716-2197-4_18

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten