Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3543507.3583207acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article
Public Access

Fair Graph Representation Learning via Diverse Mixture-of-Experts

Published: 30 April 2023 Publication History

Abstract

Graph Neural Networks (GNNs) have demonstrated a great representation learning capability on graph data and have been utilized in various downstream applications. However, real-world data in web-based applications (e.g., recommendation and advertising) always contains bias, preventing GNNs from learning fair representations. Although many works were proposed to address the fairness issue, they suffer from the significant problem of insufficient learnable knowledge with limited attributes after debiasing. To address this problem, we develop Graph-Fairness Mixture of Experts (G-Fame), a novel plug-and-play method to assist any GNNs to learn distinguishable representations with unbiased attributes. Furthermore, based on G-Fame, we propose G-Fame++, which introduces three novel strategies to improve the representation fairness from node representations, model layer, and parameter redundancy perspectives. In particular, we first present the embedding diversified method to learn distinguishable node representations. Second, we design the layer diversified strategy to maximize the output difference of distinct model layers. Third, we introduce the expert diversified method to minimize expert parameter similarities to learn diverse and complementary representations. Extensive experiments demonstrate the superiority of G-Fame and G-Fame++ in both accuracy and fairness, compared to state-of-the-art methods across multiple graph datasets.

References

[1]
Yahav Bechavod, Christopher Jung, and Steven Z Wu. 2020. Metric-free individual fairness in online learning. In NeurIPS.
[2]
Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. 2017. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075 (2017).
[3]
Reuben Binns. 2020. On the apparent conflict between individual and group fairness. In FAccT.
[4]
Robin Burke. 2017. Multisided fairness for recommendation. arXiv preprint arXiv:1707.00093 (2017).
[5]
Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020. Simple and deep graph convolutional networks. In ICML.
[6]
Sean Current, Yuntian He, Saket Gurukar, and Srinivasan Parthasarathy. 2022. FairMod: Fair Link Prediction and Recommendation via Graph Modification. arXiv preprint arXiv:2201.11596 (2022).
[7]
Enyan Dai and Suhang Wang. 2021. Say no to the discrimination: Learning fair graph neural networks with limited sensitive attribute information. In WSDM.
[8]
Yushun Dong, Song Wang, Yu Wang, Tyler Derr, and Jundong Li. 2022. On Structural Explanation of Bias in Graph Neural Networks. In KDD.
[9]
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In ITCS.
[10]
Harrison Edwards and Amos Storkey. 2015. Censoring representations with an adversary. arXiv preprint arXiv:1511.05897 (2015).
[11]
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In WWW.
[12]
Yujie Fan, Mingxuan Ju, Chuxu Zhang, and Yanfang Ye. 2022. Heterogeneous temporal graph neural network. In SDM.
[13]
Will Fleisher. 2021. What’s Fair about Individual Fairness¿. In AAAI.
[14]
Arpita Ghosh and Robert Kleinberg. 2016. Inferential privacy guarantees for differentially private mechanisms. arXiv preprint arXiv:1603.01508 (2016).
[15]
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In ICML.
[16]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM (2020).
[17]
Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD.
[18]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In NeurIPS.
[19]
Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. In NeurIPS.
[20]
Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Ali Mostafavi, and Xia Hu. 2022. Generalized Demographic Parity for Group Fairness. In ICLR.
[21]
Jian Kang and Hanghang Tong. 2021. Fair graph mining. In ICDM.
[22]
Jian Kang, Tiankai Xie, Xintao Wu, Ross Maciejewski, and Hanghang Tong. 2021. MultiFair: Multi-Group Fairness in Machine Learning. arXiv preprint arXiv:2105.11069 (2021).
[23]
Shima Khoshraftar and Aijun An. 2022. A Survey on Graph Representation Learning Methods. arXiv preprint arXiv:2204.01855 (2022).
[24]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
[25]
Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 (2016).
[26]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[27]
Guohao Li, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2019. Deepgcns: Can gcns go as deep as cnns¿. In ICCV.
[28]
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. 2018. Visualizing the loss landscape of neural nets. In NeurIPS.
[29]
Peizhao Li, Yifei Wang, Han Zhao, Pengyu Hong, and Hongfu Liu. 2021. On dyadic fairness: Exploring and mitigating bias in graph connections. In ICLR.
[30]
Peiyuan Liao, Han Zhao, Keyulu Xu, Tommi Jaakkola, Geoffrey J Gordon, Stefanie Jegelka, and Ruslan Salakhutdinov. 2021. Information obfuscation of graph neural networks. In ICML.
[31]
Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Schölkopf, and Adrian Weller. 2021. Learning with hyperspherical uniformity. In AISTATS.
[32]
Sepideh Mahabadi and Ali Vakilian. 2020. Individual fairness for k-clustering. In ICML.
[33]
Farzan Masrour, Tyler Wilson, Heng Yan, Pang-Ning Tan, and Abdol Esfahanian. 2020. Bursting the filter bubble: Fairness-aware network link prediction. In AAAI.
[34]
Dana Pessach and Erez Shmueli. 2022. A Review on Fairness in Machine Learning. Comput. Surveys (2022).
[35]
Tahleen Rahman, Bartlomiej Surma, Michael Backes, and Yang Zhang. 2019. Fairwalk: Towards fair graph embedding. In IJCAI.
[36]
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In ICLR.
[37]
Noam Shazeer, *Azalia Mirhoseini, *Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In ICLR.
[38]
Indro Spinelli, Simone Scardapane, Amir Hussain, and Aurelio Uncini. 2021. Fairdrop: Biased edge dropout for enhancing fairness in graph representation learning. IEEE Transactions on Artificial Intelligence (2021).
[39]
Yijun Tian, Kaiwen Dong, Chunhui Zhang, Chuxu Zhang, and Nitesh V Chawla. 2023. Heterogeneous Graph Masked Autoencoders. In AAAI.
[40]
Yijun Tian, Chuxu Zhang, Zhichun Guo, Xiangliang Zhang, and Nitesh Chawla. 2023. Learning MLPs on Graphs: A Unified View of Effectiveness, Robustness, and Efficiency. In ICLR.
[41]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of Machine Learning Research (2008).
[42]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
[43]
Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2019. Deep Graph Infomax. In ICLR.
[44]
Le Wu, Lei Chen, Pengyang Shao, Richang Hong, Xiting Wang, and Meng Wang. 2021. Learning fair representations for recommendation: A graph-based perspective. In WWW.
[45]
Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. 2020. Graph neural networks in recommender systems: a survey. Comput. Surveys (2020).
[46]
Lianghao Xia, Chao Huang, and Chuxu Zhang. 2022. Self-supervised hypergraph transformer for recommender systems. In KDD.
[47]
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks¿. In ICLR.
[48]
Moyi Yang, Junjie Sheng, Xiangfeng Wang, Wenyan Liu, Bo Jin, Jun Wang, and Hongyuan Zha. 2022. Obtaining Dyadic Fairness by Optimal Transport. arXiv preprint arXiv:2202.04520 (2022).
[49]
Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collaborative filtering. In NeurIPS.
[50]
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In WWW.
[51]
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P Gummadi. 2019. Fairness constraints: A flexible approach for fair classification. Journal of Machine Learning Research (2019).
[52]
Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In KDD.
[53]
Chunhui Zhang, Yijun Tian, Mingxuan Ju, Zheyuan Liu, Yanfang Ye, Nitesh Chawla, and Chuxu Zhang. 2023. Chasing All-Round Graph Representation Robustness: Model, Training, and Optimization. In ICLR.
[54]
Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V Chawla. 2020. Few-shot knowledge graph completion. In AAAI.
[55]
Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. In NeurIPS.

Cited By

View all
  • (2024)PGODEProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693424(33305-33328)Online publication date: 21-Jul-2024
  • (2024)Fairness-Aware Graph Neural Networks: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/364914218:6(1-23)Online publication date: 12-Apr-2024
  • (2024)Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing ApproachProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671848(1701-1712)Online publication date: 25-Aug-2024
  • Show More Cited By

Index Terms

  1. Fair Graph Representation Learning via Diverse Mixture-of-Experts
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          WWW '23: Proceedings of the ACM Web Conference 2023
          April 2023
          4293 pages
          ISBN:9781450394161
          DOI:10.1145/3543507
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 30 April 2023

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Fairness
          2. Graph Representation Learning
          3. Mixture-of-Experts

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          Conference

          WWW '23
          Sponsor:
          WWW '23: The ACM Web Conference 2023
          April 30 - May 4, 2023
          TX, Austin, USA

          Acceptance Rates

          Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)640
          • Downloads (Last 6 weeks)85
          Reflects downloads up to 22 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)PGODEProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693424(33305-33328)Online publication date: 21-Jul-2024
          • (2024)Fairness-Aware Graph Neural Networks: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/364914218:6(1-23)Online publication date: 12-Apr-2024
          • (2024)Toward Structure Fairness in Dynamic Graph Embedding: A Trend-aware Dual Debiasing ApproachProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671848(1701-1712)Online publication date: 25-Aug-2024
          • (2024)FaDE: A Face Segment Driven Identity Anonymization Framework For Fair Face RecognitionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679737(1121-1131)Online publication date: 21-Oct-2024
          • (2024)FATE: Learning Effective Binary Descriptors With Group FairnessIEEE Transactions on Image Processing10.1109/TIP.2024.340613433(3648-3661)Online publication date: 2024
          • (2024)BP-MoE: Behavior Pattern-aware Mixture-of-Experts for Temporal Graph Representation LearningKnowledge-Based Systems10.1016/j.knosys.2024.112056299(112056)Online publication date: Sep-2024
          • (2024)Advancing crowd forecasting with graphs across microscopic trajectory to macroscopic dynamicsInformation Fusion10.1016/j.inffus.2024.102275106:COnline publication date: 25-Jun-2024
          • (2024)Symbolic Prompt Tuning Completes the App Promotion GraphMachine Learning and Knowledge Discovery in Databases. Applied Data Science Track10.1007/978-3-031-70381-2_12(183-198)Online publication date: 22-Aug-2024
          • (2023)Heterogeneous Temporal Graph Neural Network ExplainerProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614909(1298-1307)Online publication date: 21-Oct-2023

          View Options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Login options

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media