Policy Fairness and Unknown Bias Dynamics in Sequential Allocations
Article No.: 38, Pages 1 - 10
Abstract
This work considers a dynamic decision making framework for allocating opportunities over time to advantaged and disadvantaged individuals, focusing on the example of college admissions. Here, individuals in the disadvantaged group are assumed to experience a societal bias that limits their success probability. Bias dynamics dictate how the societal bias changes based on the current allocation of opportunities. We model this environment as a Markov Decision Process (MDP) and empirically examine the purely utility maximising policy in terms of fairness. We demonstrate the influence of the bias dynamics on long-term fairness of allocations, and analyse the interplay between utility and policy-fairness for different dynamics under different optimisation parameters. We consider the cases of known and unknown bias dynamics. For known dynamics, we show that a short horizon view presents fairness as a trade-off for utility, but a long horizon view reveals that the two are aligned. Moreover, we suggest that when the dynamics are unknown, the approach towards epistemic uncertainty may also affect fairness, and should be considered when designing fair decision making models.
Supplemental Material
PDF File
Policy Fairness and Unknown Bias Dynamics in Sequential Allocations - Appendix.pdf - a technical appendix of the MDP parameters and additional experiments. eaamo23-41_code.zip - Python code for generating the experimental results.
- Download
- 520.62 KB
ZIP File
Policy Fairness and Unknown Bias Dynamics in Sequential Allocations - Appendix.pdf - a technical appendix of the MDP parameters and additional experiments. eaamo23-41_code.zip - Python code for generating the experimental results.
- Download
- 7.24 KB
References
[1]
Krishna Acharya, Eshwar Ram Arunachaleswaran, Sampath Kannan, Aaron Roth, and Juba Ziani. 2023. Wealth dynamics over generations: Analysis and interventions. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 42–57.
[2]
Nil-Jana Akpinar, Cyrus DiCiccio, Preetam Nandy, and Kinjal Basu. 2022. Long-term Dynamics of Fairness Intervention in Connection Recommender Systems. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 22–35.
[3]
Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2019. Fairness and Machine Learning. fairmlbook. org.
[4]
Yahav Bechavod, Katrina Ligett, Aaron Roth, Bo Waggoner, and Steven Z Wu. 2019. Equal opportunity in online classification with partial feedback. Advances in Neural Information Processing Systems 32 (2019).
[5]
Marc G Bellemare, Will Dabney, and Rémi Munos. 2017. A distributional perspective on reinforcement learning. In International Conference on Machine Learning. PMLR, 449–458.
[6]
Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153–163.
[7]
P Clavier, S Allassonnière, and E Le Pennec. 2022. Robust Reinforcement Learning with Distributional Risk-averse formulation. ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments, Baltimore, Maryland, USA (2022).
[8]
Elliot Creager, David Madras, Toniann Pitassi, and Richard Zemel. 2020. Causal modeling for fairness in dynamical systems. In International Conference on Machine Learning. PMLR, 2185–2195.
[9]
Alexander D’Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, David Sculley, and Yoni Halpern. 2020. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 525–534.
[10]
Xolani Dastile, Turgay Celik, and Moshe Potsane. 2020. Statistical and machine learning models in credit scoring: A systematic literature survey. Applied Soft Computing 91 (2020), 106263.
[11]
Christos Dimitrakakis. 2011. Robust bayesian reinforcement learning through tight lower bounds. In European Workshop on Reinforcement Learning. Springer, 177–188.
[12]
Christos Dimitrakakis, Yang Liu, David C Parkes, and Goran Radanovic. 2019. Bayesian fairness. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 509–516.
[13]
Hadi Elzayn, Shahin Jabbari, Christopher Jung, Michael Kearns, Seth Neel, Aaron Roth, and Zachary Schutzman. 2019. Fair algorithms for learning in allocation problems. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 170–179.
[14]
Danielle Ensign, Sorelle A Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway feedback loops in predictive policing. In Conference on Fairness, Accountability and Transparency. PMLR, 160–171.
[15]
Danielle Ensign, Frielder Sorelle, Neville Scott, Scheidegger Carlos, and Venkatasubramanian Suresh. 2018. Decision making with limited feedback. In Algorithmic Learning Theory. PMLR, 359–367.
[16]
Hannes Eriksson and Christos Dimitrakakis. 2020. Epistemic risk-sensitive reinforcement learning. ESANN 2020 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Online event, 2-4 October 2020 (2020).
[17]
European Union. 2021. Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206. Accessed: 2022-08-08.
[18]
Hanming Fang and Andrea Moro. 2011. Theories of statistical discrimination and affirmative action: A survey. Handbook of social economics 1 (2011), 133–200.
[19]
Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, 2021. Towards long-term fairness in recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 445–453.
[20]
Ganesh Ghalme, Vineet Nair, Vishakha Patil, and Yilun Zhou. 2021. State-Visitation Fairness in Average-Reward MDPs. arXiv preprint arXiv:2102.07120 (2021).
[21]
Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, Aviv Tamar, 2015. Bayesian reinforcement learning: A survey. Foundations and Trends® in Machine Learning 8, 5-6 (2015), 359–483.
[22]
Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016).
[23]
Hoda Heidari, Claudio Ferrari, Krishna Gummadi, and Andreas Krause. 2018. Fairness behind a veil of ignorance: A welfare analysis for automated decision making. Advances in Neural Information Processing Systems 31 (2018).
[24]
Hoda Heidari and Jon Kleinberg. 2021. Allocating Opportunities in a Dynamic Model of Intergenerational Mobility. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. 15–25.
[25]
Madeline E Heilman, Caryn J Block, and Jonathan A Lucas. 1992. Presumed incompetent? Stigmatization and affirmative action efforts.Journal of Applied Psychology 77, 4 (1992), 536.
[26]
Sarah D Herrmann, Robert Mark Adelman, Jessica E Bodford, Oliver Graudejus, Morris A Okun, and Virginia SY Kwan. 2016. The effects of a female role model on academic performance and persistence of women in STEM courses. Basic and Applied Social Psychology 38, 5 (2016), 258–268.
[27]
Yaowei Hu and Lu Zhang. 2022. Achieving long-term fairness in sequential decision making. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 9549–9557.
[28]
Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. 2017. Fairness in reinforcement learning. In International conference on machine learning. PMLR, 1617–1626.
[29]
Emilio Jorge, Hannes Eriksson, Christos Dimitrakakis, Debabrota Basu, and Divya Grover. 2020. Inferential induction: A novel framework for bayesian reinforcement learning. (2020).
[30]
Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. 2016. Fairness in learning: Classic and contextual bandits. Advances in neural information processing systems 29 (2016).
[31]
Nikola H Konstantinov and Christoph Lampert. 2022. Fairness-aware pac learning from corrupted data. Journal of Machine Learning Research 23 (2022).
[32]
Benjamin Laufer. 2021. Beyond Validity: Current Auditing Methods for Criminal Risk Assessments Do Not Consider Sequential Feedback Effects. (2021).
[33]
David Lindner, Hoda Heidari, and Andreas Krause. 2021. Addressing the Long-term Impact of ML Decisions via Policy Regret. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21. International Joint Conference on Artificial Intelligence, Inc, 537–544.
[34]
Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. 2018. Delayed impact of fair machine learning. In International Conference on Machine Learning. PMLR, 3150–3158.
[35]
Lydia T Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer Chayes. 2020. The disparate equilibria of algorithmic decision making when individuals invest rationally. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 381–391.
[36]
Kristian Lum and William Isaac. 2016. To predict and serve?Significance 13, 5 (2016), 14–19.
[37]
Claire Cain Miller. 2015. Can an algorithm hire better than a human. The New York Times 25 (2015).
[38]
Thomas Minka. 2000. Bayesian linear regression. Technical Report. MIT Media Lab. http://research.microsoft.com/en-us/um/people/minka/papers/linear.html
[39]
Alan Mishler and Niccolò Dalmasso. 2022. Fair When Trained, Unfair When Deployed: Observable Fairness Measures are Unstable in Performative Prediction Settings. arXiv preprint arXiv:2202.05049 (2022).
[40]
Tetsuro Morimura, Masashi Sugiyama, Hisashi Kashima, Hirotaka Hachiya, and Toshiyuki Tanaka. 2010. Nonparametric return distribution approximation for reinforcement learning. In Proceedings of the 27th International Conference on International Conference on Machine Learning. 799–806.
[41]
Tetsuro Morimura, Masashi Sugiyama, Hisashi Kashima, Hirotaka Hachiya, and Toshiyuki Tanaka. 2010. Parametric return density estimation for reinforcement learning. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence. 368–375.
[42]
Hussein Mouzannar, Mesrob I Ohannessian, and Nathan Srebro. 2019. From fair decision making to social equality. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 359–368.
[43]
Pegah Nokhiz, Aravinda Kanchana Ruwanpathirana, Neal Patwari, and Suresh Venkatasubramanian. 2021. Precarity: Modeling the Long Term Effects of Compounded Decisions on Individual Instability. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 199–208.
[44]
Benjamin Paaßen, Astrid Bunge, Carolin Hainke, Leon Sindelar, and Matthias Vogelsang. 2019. Dynamic fairness-breaking vicious cycles in automatic decision making. ESANN 2019 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), 24-26 April 2019 (2019).
[45]
Liam Peet-Pare, Nidhi Hegde, and Alona Fyshe. 2022. Long Term Fairness for Minority Groups via Performative Distributionally Robust Optimization. ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments, Baltimore, Maryland, USA (2022).
[46]
Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. 2020. Performative prediction. In International Conference on Machine Learning. PMLR, 7599–7609.
[47]
José Pombal, Pedro Saleiro, Mário AT Figueiredo, and Pedro Bizarro. 2022. Prisoners of Their Own Devices: How Models Induce Data Bias in Performative Prediction. ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments, Baltimore, Maryland, USA (2022).
[48]
Bhagyashree Puranik, Upamanyu Madhow, and Ramtin Pedarsani. 2022. A Dynamic Decision-Making Framework Promoting Long-Term Fairness. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 547–556.
[49]
Martin L Puterman. 2014. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons.
[50]
Lydia Reader, Pegah Nokhiz, Cathleen Power, Neal Patwari, Suresh Venkatasubramanian, and Sorelle Friedler. 2022. Models for understanding and quantifying feedback in societal systems. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 1765–1775.
[51]
Marc Rigter, Bruno Lacerda, and Nick Hawes. 2021. Risk-averse bayes-adaptive reinforcement learning. Advances in Neural Information Processing Systems 34 (2021), 1142–1154.
[52]
Samordnaopptak. 2021. The Norwegian Universities and Colleges Admission Service: Gender points. https://www.samordnaopptak.no/info/opptak/opptak-uhg/poengberegning/legge-til-poeng/kjonnspoeng/index.html. Accessed: 2022-08-08.
[53]
Sebastian Scher, Simone Kopeinik, Andreas Trügler, and Dominik Kowald. 2022. Long-term dynamics of fairness: understanding the impact of data-driven targeted help on job seekers. arXiv preprint arXiv:2208.08881 (2022).
[54]
Pola Schwöbel and Peter Remmers. 2022. The Long Arc of Fairness: Formalisations and Ethical Discourse. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022), 2179–2188.
[55]
Tareq Si Salem, Georgios Iosifidis, and Giovanni Neglia. 2022. Enabling Long-term Fairness in Dynamic Resource Allocation. Proceedings of the ACM on Measurement and Analysis of Computing Systems 6, 3 (2022), 1–36.
[56]
Sean R Sinclair, Siddhartha Banerjee, and Christina Lee Yu. 2022. Sequential fair allocation: Achieving the optimal envy-efficiency tradeoff curve. ACM SIGMETRICS Performance Evaluation Review 50, 1 (2022), 95–96.
[57]
Sean R Sinclair, Gauri Jain, Siddhartha Banerjee, and Christina Lee Yu. 2020. Sequential fair allocation of limited resources under stochastic demands. arXiv preprint arXiv:2011.14382 (2020).
[58]
Yi Sun, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2022. The Backfire Effects of Fairness Constraints. ICML 2022 Workshop on Responsible Decision Making in Dynamic Environments, Baltimore, Maryland, USA (2022).
[59]
Behzad Tabibian, Vicenç Gómez, Abir De, Bernhard Schölkopf, and Manuel Gomez Rodriguez. 2019. Consequential ranking algorithms and long-term welfare. arXiv preprint arXiv:1905.05305 (2019).
[60]
Aviv Tamar, Dotan Di Castro, and Shie Mannor. 2016. Learning the variance of the reward-to-go. The Journal of Machine Learning Research 17, 1 (2016), 361–396.
[61]
Ruibo Tu, Xueru Zhang, Yang Liu, Hedvig Kjellström, Mingyan Liu, Kun Zhang, and Cheng Zhang. 2020. How Do Fair Decisions Fare in Long-term Qualification?. In Thirty-fourth Conference on Neural Information Processing Systems.
[62]
Kenneth M Tyler, Falynn A Thompson, Donna E Gay, Jennifer Burris, Howard Lloyd, and Sycarah Fisher. 2016. Internalized stereotypes and academic self-handicapping among Black American male high school students. Negro Educational Review 67, 1-4 (2016), 5.
[63]
Aline Weber, Blossom Metevier, Yuriy Brun, Philip S Thomas, and Bruno Castro da Silva. 2022. Enforcing Delayed-Impact Fairness Guarantees. arXiv preprint arXiv:2208.11744 (2022).
[64]
Min Wen, Osbert Bastani, and Ufuk Topcu. 2021. Algorithms for fairness in sequential decision making. In International Conference on Artificial Intelligence and Statistics. PMLR, 1144–1152.
[65]
DJ White. 1988. Mean, variance, and probabilistic criteria in finite Markov decision processes: A review. Journal of Optimization Theory and Applications 56, 1 (1988), 1–29.
[66]
Joshua Williams and J Zico Kolter. 2019. Dynamic modeling and equilibria in fair decision making. arXiv preprint arXiv:1911.06837 (2019).
[67]
Tongxin Yin, Reilly Raab, Mingyan Liu, and Yang Liu. [n. d.]. Long-Term Fairness with Unknown Dynamics. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models.
[68]
Xueru Zhang and Mingyan Liu. 2021. Fairness in learning-based sequential decision algorithms: A survey. In Handbook of Reinforcement Learning and Control. Springer, 525–555.
Index Terms
- Policy Fairness and Unknown Bias Dynamics in Sequential Allocations
Recommendations
Analysis of short term fairness and its impact on packet level performance
In this paper, we consider the one-bit feedback scheduler with a threshold in a wireless network. Due to its feedback reduction structure, the one-bit feedback scheduler has been studied as one of the good candidates implementable in a practical ...
Chaotic dynamics of sequential CSTRs
For two CSTRs (constant flow stirred tank reactors) operating in series (sequentially) with constant input and a first order reaction, chaotic behavior arising from a large amplitude periodic perturbation of a quasi-periodic orbit has been observed. ...
Airtime Fairness for IEEE 802.11 Multirate Networks
Under a multi rate network scenario, the IEEE 802.11 DCF MAC fails to provide air-time fairness for all competing stations since the protocol is designed for ensuring max-min throughput fairness and the maximum achievable throughput by any station gets ...
Comments
Information & Contributors
Information
Published In
October 2023
498 pages
ISBN:9798400703812
DOI:10.1145/3617694
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Norges Forskningsråd
Conference
EAAMO '23: Equity and Access in Algorithms, Mechanisms, and Optimization
October 30 - November 1, 2023
MA, Boston, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 388Total Downloads
- Downloads (Last 12 months)301
- Downloads (Last 6 weeks)22
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in