research-article

Open access

Unbiased Graph Embedding with Biased Graph Observations

Authors:

Hongning WangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1423 - 1433

https://doi.org/10.1145/3485447.3512189

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Graph embedding techniques are pivotal in real-world machine learning tasks that operate on graph-structured data, such as social recommendation and protein structure modeling. Embeddings are mostly performed on the node level for learning representations of each node. Since the formation of a graph is inevitably affected by certain sensitive node attributes, the node embeddings can inherit such sensitive information and introduce undesirable biases in downstream tasks. Most existing works impose ad-hoc constraints on the node embeddings to restrict their distributions for unbiasedness/fairness, which however compromise the utility of the resulting embeddings. In this paper, we propose a principled new way for unbiased graph embedding by learning node embeddings from an underlying bias-free graph, which is not influenced by sensitive node attributes. Motivated by this new perspective, we propose two complementary methods for uncovering such an underlying graph, with the goal of introducing minimum impact on the utility of the embeddings. Both our theoretical justification and extensive experimental comparisons against state-of-the-art solutions demonstrate the effectiveness of our proposed methods.

References

[1]

Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitnik. 2021. Towards a Unified Framework for Fair and Stable Graph Representation Learning. In Proceedings of Conference on Uncertainty in Artificial Intelligence, UAI.

[2]

Edoardo Maria Airoldi, David M Blei, Stephen E Fienberg, and Eric P Xing. 2008. Mixed membership stochastic blockmodels. Journal of machine learning research(2008).

[3]

Mikhail Belkin and Partha Niyogi. 2001. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Nips, Vol. 14. 585–591.

[4]

Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2021. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research 50, 1 (2021), 3–44.

[5]

Avishek Joey Bose and William L. Hamilton. 2019. Compositional Fairness Constraints for Graph Embeddings. arxiv:1905.10674 [cs.LG]

[6]

Maarten Buyl and Tijl De Bie. 2020. DeBayes: a Bayesian Method for Debiasing Network Embeddings. In International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 119). 1220–1229.

[7]

Flavio P Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. 2017. Optimized pre-processing for discrimination prevention. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995–4004.

[8]

Alexandra Chouldechova. 2016. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arxiv:1610.07524 [stat.AP]

[9]

Fan Chung and Linyuan Lu. 2002. The average distances in random graphs with given expected degrees. Proceedings of the National Academy of Sciences (2002).

[10]

Enyan Dai and Suhang Wang. 2020. Learning Fair Graph Neural Networks with Limited and Private Sensitive Attribute Information. arXiv preprint arXiv:2009.01454(2020).

[11]

Enyan Dai and Suhang Wang. 2021. Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 680–688.

Digital Library

[12]

Palash Goyal and Emilio Ferrara. 2018. Graph embedding techniques, applications, and performance: A survey. Knowledge-Based Systems 151 (Jul 2018), 78–94.

[13]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855–864.

Digital Library

[14]

Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016), 3315–3323.

[15]

F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4(2015), 1–19.

[16]

John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, 2021. Highly accurate protein structure prediction with AlphaFold. Nature 596, 7873 (2021), 583–589.

[17]

Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. Fairness-Aware Classifier with Prejudice Remover Regularizer. In Machine Learning and Knowledge Discovery in Databases, Peter A. Flach, Tijl De Bie, and Nello Cristianini (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 35–50.

[18]

Bo Kang, Jefrey Lijffijt, and Tijl De Bie. 2018. Conditional Network Embeddings. arxiv:1805.07544 [stat.ML]

[19]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016).

[20]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. arxiv:1609.02907 [cs.LG]

[21]

T. Kloek and H. K. van Dijk. 1978. Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo. Econometrica 46, 1 (1978), 1–19.

[22]

Carol T Kulik, L Robert Jr, 2000. Demographics in service encounters: effects of racial and gender congruence on perceived fairness. Social Justice Research 13, 4 (2000), 375–402.

[23]

Timothy La Fond and Jennifer Neville. 2010. Randomization Tests for Distinguishing Social Influence and Homophily Effects. In Proceedings of the 19th International Conference on World Wide Web (Raleigh, North Carolina, USA) (WWW ’10). Association for Computing Machinery, New York, NY, USA, 601–610.

Digital Library

[24]

Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, and Zoubin Ghahramani. 2010. Kronecker Graphs: An Approach to Modeling Networks. J. Mach. Learn. Res. 11 (March 2010), 985–1042.

Digital Library

[25]

Peng Liu, Lemei Zhang, and Jon Atle Gulla. 2019. Real-time social recommendation based on graph embedding and temporal context. International Journal of Human-Computer Studies 121 (2019), 58–72.

[26]

David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018. Learning Adversarially Fair and Transferable Representations. arxiv:1802.06309 [cs.LG]

[27]

Miller McPherson, Lynn Smith-Lovin, and James M. Cook. 2001. Birds of a Feather: Homophily in Social Networks. Review of Sociology 27(2001), 415–444.

[28]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781(2013).

[29]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Vol. 26. Curran Associates, Inc.

[30]

Andriy Mnih and Russ R Salakhutdinov. 2008. Probabilistic Matrix Factorization. In Advances in Neural Information Processing Systems, J. Platt, D. Koller, Y. Singer, and S. Roweis (Eds.). Vol. 20. Curran Associates, Inc.

[31]

Andrew Y Ng, Michael I Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In Advances in neural information processing systems. 849–856.

[32]

Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric Transitivity Preserving Graph Embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16). Association for Computing Machinery, New York, NY, USA, 1105–1114.

Digital Library

[33]

John Palowitch and Bryan Perozzi. 2020. MONET: Debiasing Graph Embeddings via the Metadata-Orthogonal Training Unit. arxiv:1909.11793 [cs.LG]

[34]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.

Digital Library

[35]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online Learning of Social Representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, New York, USA) (KDD ’14). Association for Computing Machinery, New York, NY, USA, 701–710.

Digital Library

[36]

Joseph J. Pfeiffer, Sebastian Moreno, Timothy La Fond, Jennifer Neville, and Brian Gallagher. 2014. Attributed Graph Models: Modeling Network Structure with Correlated Attributes. In Proceedings of the 23rd International Conference on World Wide Web (Seoul, Korea) (WWW ’14). Association for Computing Machinery, New York, NY, USA, 831–842.

Digital Library

[37]

Tahleen Rahman, Bartlomiej Surma, Michael Backes, and Yang Zhang. 2019. Fairwalk: Towards Fair Graph Embedding. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 3289–3295.

[38]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian Personalized Ranking from Implicit Feedback. arxiv:1205.2618 [cs.IR]

[39]

Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S Yu, Lifang He, and Bo Li. 2018. Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528(2018).

[40]

Lubos Takac and Michal Zabovsky. 2012. Data analysis in public social networks. In International scientific conference and international workshop present day trends of innovations.

[41]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-Scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW ’15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1067–1077.

[42]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903(2017).

[43]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. arxiv:1710.10903 [stat.ML]

[44]

Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural Deep Network Embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16). Association for Computing Machinery, New York, NY, USA, 1225–1234.

Digital Library

[45]

Blake Woodworth, Suriya Gunasekar, Mesrob I. Ohannessian, and Nathan Srebro. 2017. Learning Non-Discriminatory Predictors. In Proceedings of the 2017 Conference on Learning Theory(Proceedings of Machine Learning Research, Vol. 65), Satyen Kaleand Ohad Shamir (Eds.). PMLR, 1920–1953.

[46]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International conference on machine learning. PMLR, 6861–6871.

[47]

Min Xie, Hongzhi Yin, Hao Wang, Fanjiang Xu, Weitong Chen, and Sen Wang. 2016. Learning graph-based poi embedding for location-based recommendation. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 15–24.

Digital Library

[48]

Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning Fair Representations. In Proceedings of the 30th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 28), Sanjoy Dasgupta and David McAllester (Eds.). PMLR, Atlanta, Georgia, USA, 325–333.

Cited By

Medda GFabbri FMarras MBoratto LFenu G(2024)GNNUERS: Fairness Explanation in GNNs for Recommendation via Counterfactual ReasoningACM Transactions on Intelligent Systems and Technology10.1145/3655631Online publication date: 3-Apr-2024
https://dl.acm.org/doi/10.1145/3655631
Zheng CJiang GYan XYin PZhou QCheng J(2024)GE2: A General and Efficient Knowledge Graph Embedding Learning SystemProceedings of the ACM on Management of Data10.1145/36549862:3(1-27)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654986
Chen ARossi RPark NTrivedi PWang YYu TKim SDernoncourt FAhmed N(2024)Fairness-Aware Graph Neural Networks: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/364914218:6(1-23)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3649142
Show More Cited By

Index Terms

Unbiased Graph Embedding with Biased Graph Observations

Index terms have been assigned to the content through auto-classification.

Recommendations

Embedding parameterized graph classes into normed spaces
Graph embedding via multi-scale graph representations
Abstract
Graph embedding provides an effective way to encode graph nodes as continuous vector representations in a low-dimensional space. The high-order proximities based graph embedding methods can preserve global structures, but the high-...
Unsupervised Graph Embedding via Adaptive Graph Learning
Graph autoencoders (GAEs) are powerful tools in representation learning for graph embedding. However, the performance of GAEs is very dependent on the quality of the graph structure, i.e., of the adjacency matrix. In other words, GAEs would perform poorly ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
1,021
Total Downloads

Downloads (Last 12 months)305
Downloads (Last 6 weeks)27

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Medda GFabbri FMarras MBoratto LFenu G(2024)GNNUERS: Fairness Explanation in GNNs for Recommendation via Counterfactual ReasoningACM Transactions on Intelligent Systems and Technology10.1145/3655631Online publication date: 3-Apr-2024
https://dl.acm.org/doi/10.1145/3655631
Zheng CJiang GYan XYin PZhou QCheng J(2024)GE2: A General and Efficient Knowledge Graph Embedding Learning SystemProceedings of the ACM on Management of Data10.1145/36549862:3(1-27)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654986
Chen ARossi RPark NTrivedi PWang YYu TKim SDernoncourt FAhmed N(2024)Fairness-Aware Graph Neural Networks: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/364914218:6(1-23)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1145/3649142
Chen WWu YZhang ZZhuang FHe ZXie RXia F(2024)FairGap: Fairness-Aware Recommendation via Generating Counterfactual GraphACM Transactions on Information Systems10.1145/363835242:4(1-25)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3638352
Dong YLei ZZheng ZWang SMa JHuang AChen CLi JChua TNgo CKumar RLauw HKa-Wei Lee R(2024)PyGDebias: A Python Library for Debiasing in Graph LearningCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3651239(1019-1022)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589335.3651239
Wo ZShao MWang WGuo XLin LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Graph Contrastive Learning via Interventional View GenerationProceedings of the ACM on Web Conference 202410.1145/3589334.3645687(1024-1034)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645687
Nian YChang YJin WLin LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Globally Interpretable Graph Learning via Distribution MatchingProceedings of the ACM on Web Conference 202410.1145/3589334.3645674(992-1002)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645674
Gourru ALaclau CChoudhary MLargeron C(2024)Variational Perspective on Fair Edge PredictionAdvances in Intelligent Data Analysis XXII10.1007/978-3-031-58547-0_8(93-104)Online publication date: 16-Apr-2024
https://doi.org/10.1007/978-3-031-58547-0_8
Luo ZHuang HLian JSong XXie XJin HOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Cross-links matter for link predictionProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669607(79594-79612)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669607
Dong YMa JWang SChen CLi J(2023)Fairness in Graph Mining: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.326559835:10(10583-10602)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TKDE.2023.3265598
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents