research-article

Few-Shot Graph Learning for Molecular Property Prediction

Authors:

Nitesh V. ChawlaAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

Pages 2559 - 2567

https://doi.org/10.1145/3442381.3450112

Published: 03 June 2021 Publication History

Abstract

The recent success of graph neural networks has significantly boosted molecular property prediction, advancing activities such as drug discovery. The existing deep neural network methods usually require large training dataset for each property, impairing their performance in cases (especially for new molecular properties) with a limited amount of experimental data, which are common in real situations. To this end, we propose Meta-MGNN, a novel model for few-shot molecular property prediction. Meta-MGNN applies molecular graph neural network to learn molecular representations and builds a meta-learning framework for model optimization. To exploit unlabeled molecular information and address task heterogeneity of different molecular properties, Meta-MGNN further incorporates molecular structures, attribute based self-supervised modules and self-attentive task weights into the former framework, strengthening the whole learning model. Extensive experiments on two public multi-property datasets demonstrate that Meta-MGNN outperforms a variety of state-of-the-art methods.

References

[1]

Han Altae-Tran, Bharath Ramsundar, Aneesh S Pappu, and Vijay Pande. 2017. Low data drug discovery with one-shot learning. In ACS Central Science.

[2]

Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. 2018. Meta-learning with differentiable closed-form solvers. In International Conference for Learning Representation (ICLR).

[3]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).

[4]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In The Web Conference.

Digital Library

[5]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (ICML).

[6]

Victor Garcia and Joan Bruna. 2018. Few-shot learning with graph neural networks. In International Conference for Learning Representation (ICLR).

[7]

Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. 2017. Neural message passing for quantum chemistry. In International Conference on Machine Learning (ICML).

[8]

Zhichun Guo, Wenhao Yu, Chuxu Zhang, Meng Jiang, and Nitesh Chawla. 2020. GraSeq: graph and sequence fusion learning for molecular property prediction. In International Conference on Information and Knowledge Management (CIKM).

Digital Library

[9]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems.

[10]

Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. 2020. Strategies for pre-training graph neural networks. In International Conference for Learning Representation (ICLR).

[11]

Jongmin Kim, Taesup Kim, Sungwoong Kim, and Chang D Yoo. 2019. Edge-labeling graph neural network for few-shot learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]

Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference for Learning Representation.

[13]

Michael Kuhn, Ivica Letunic, Lars Juhl Jensen, and Peer Bork. 2016. The SIDER database of drugs and side effects. In Nucleic Acids Research.

[14]

Greg Landrum. 2013. Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling.

[15]

Yoonho Lee and Seungjin Choi. 2018. Gradient-based meta-learning with learned layerwise metric and subspace. In International Conference on Machine Learning.

[16]

Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In International Conference for Learning Representation (ICLR).

[17]

Ke Liu, Xiangyan Sun, Lei Jia, Jun Ma, Haoming Xing, Junqiu Wu, Hua Gao, Yax Sun, Florian Boulnois, and Jie Fan. 2019. Chemi-Net: a molecular graph convolutional network for accurate drug property prediction. In International Journal of Molecular Sciences.

[18]

Chengqiang Lu, Qi Liu, Chao Wang, Zhenya Huang, Peize Lin, and Lixin He. 2019. Molecular property prediction: A multilevel quantum interactions modeling perspective. In AAAI Conference on Artificial Intelligence (AAAI).

Digital Library

[19]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. In Journal of Machine Learning Research.

[20]

Elman Mansimov, Omar Mahmood, Seokho Kang, and Kyunghyun Cho. 2019. Molecular geometry prediction using a deep generative graph neural network. In Scientific Reports.

[21]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NeurIPS).

[22]

Sereina Riniker and Gregory A Landrum. 2013. Similarity maps-a visualization strategy for molecular fingerprints and machine-learning methods. In Journal of Cheminformatics.

[23]

Gregory Sliwoski, Sandeepkumar Kothiwale, Jens Meiler, and Edward W Lowe. 2014. Computational methods in drug discovery. In Pharmacological Reviews.

[24]

Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. 2019. Session-based social recommendation via dynamic graph attention networks. In The ACM International Conference on Web Search and Data Mining.

Digital Library

[25]

Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to compare: relation network for few-shot learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]

Jessica Vamathevan, Dominic Clark, Paul Czodrowski, Ian Dunham, Edgardo Ferran, George Lee, Bin Li, Anant Madabhushi, Parantu Shah, Michaela Spitzer, 2019. Applications of machine learning in drug discovery and development. In Nature Reviews Drug Discovery.

[27]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph attention networks. In International Conference for Learning Representation (ICLR).

[28]

Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, 2016. Matching networks for one shot learning. In Advances in Neural Information Processing Systems (NeurIPS).

[29]

Sheng Wang, Yuzhi Guo, Yuhong Wang, Hongmao Sun, and Junzhou Huang. 2019. SMILES-BERT: Large scale unsupervised pre-training for molecular property prediction. In ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB).

Digital Library

[30]

Michael J Waring, John Arrowsmith, Andrew R Leach, Paul D Leeson, Sam Mandrell, Robert M Owen, Garry Pairaudeau, William D Pennie, Stephen D Pickett, Jibo Wang, 2015. An analysis of the attrition of drug candidates from four major pharmaceutical companies. In Nature Reviews Drug Discovery.

[31]

David Weininger, Arthur Weininger, and Joseph L Weininger. 1989. SMILES. 2. Algorithm for generation of unique SMILES notation. In Journal of Chemical Information and Computer Sciences.

[32]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. In IEEE Transactions on Neural Networks and Learning Systems.

[33]

Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. 2018. MoleculeNet: a benchmark for molecular machine learning. In Chemical Science.

[34]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks?. In International Conference for Learning Representation (ICLR).

[35]

Zheng Xu, Sheng Wang, Feiyun Zhu, and Junzhou Huang. 2017. Seq2seq fingerprint: An unsupervised deep molecular embedding for drug discovery. In ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB).

Digital Library

[36]

Huaxiu Yao, Yiding Liu, Ying Wei, Xianfeng Tang, and Zhenhui Li. 2019. Learning from multiple cities: A meta-learning approach for spatial-temporal prediction. In The Web Conference (WWW).

Digital Library

[37]

Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh Chawla, and Zhenhui Li. 2020. Graph few-shot learning via knowledge transfer. In AAAI Conference on Artificial Intelligence (AAAI).

[38]

Wenhao Yu, Mengxia Yu, Tong Zhao, and Meng Jiang. 2020. Identifying referential intention with heterogeneous contexts. In The Web Conference (WWW).

Digital Library

[39]

Chuxu Zhang, Dongjin Song, Chao Huang, Ananthram Swami, and Nitesh V Chawla. 2019. Heterogeneous graph neural network. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD).

Digital Library

[40]

Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V Chawla. 2020. Few-shot knowledge graph completion. In AAAI Conference on Artificial Intelligence (AAAI).

[41]

Chuxu Zhang, Lu Yu, Mandana Saebi, Meng Jiang, and Nitesh Chawla. 2020. Few-shot multi-hop relation reasoning over knowledge bases. In Conference on Empirical Methods in Natural Language Processing: Findings (EMNLP: Findings).

[42]

Ruixiang Zhang, Tong Che, Zoubin Ghahramani, Yoshua Bengio, and Yangqiu Song. 2018. Metagan: An adversarial approach to few-shot learning. In Advances in Neural Information Processing Systems (NeurIPS).

[43]

Xiaoyu Zhang, Sheng Wang, Feiyun Zhu, Zheng Xu, Yuhong Wang, and Junzhou Huang. 2018. Seq3seq fingerprint: towards end-to-end semi-supervised deep drug discovery. In ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB).

Digital Library

[44]

Tong Zhao, Bo Ni, Wenhao Yu, and Meng Jiang. 2020. Early anomaly detection by learning and forecasting behavior. In arXiv preprint arXiv:2010.10016.

[45]

Shuangjia Zheng, Xin Yan, Yuedong Yang, and Jun Xu. 2019. Identifying structure–property relationships through SMILES syntax analysis with self-attention mechanism. In Journal of Chemical Information and Modeling.

[46]

Fan Zhou, Chengtai Cao, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Ji Geng. 2019. Meta-GNN: On few-shot node classification in graph meta-learning. In International Conference on Information and Knowledge Management (CIKM).

Digital Library

Cited By

Du MXie XLuo JLi J(2024)Meta-learning-based Inductive logistic matrix completion for prediction of kinase inhibitorsJournal of Cheminformatics10.1186/s13321-024-00838-916:1Online publication date: 16-Apr-2024
https://doi.org/10.1186/s13321-024-00838-9
Fang YZhao XXiao Wde Rijke M(2024)Few-shot Learning for Heterogeneous Information NetworksACM Transactions on Information Systems10.1145/364931142:4(1-24)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3649311
Wang SDong YHuang XChen CLi J(2024)Learning Hierarchical Task Structures for Few-shot Graph ClassificationACM Transactions on Knowledge Discovery from Data10.1145/363547318:3(1-20)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3635473
Show More Cited By

Few-Shot Graph Learning for Molecular Property Prediction
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Prediction
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Molecular property prediction (e.g., energy) is an essential problem in chemistry and biology. Unfortunately, many supervised learning methods usually suffer from the problem of scarce labeled molecules in the chemical space, where such property labels ...
Few-shot learning with transformers via graph embeddings for molecular property prediction
Abstract
Molecular property prediction is an essential task in drug discovery. Recently, deep neural networks have accelerated the discovery of compounds with improved molecular profiles for effective drug development. In particular, graph neural networks ...
Few-shot learning via graph embeddings with convolutional networks for low-data molecular property prediction
Abstract
Graph neural networks and convolutional architectures have proven to be pivotal in improving the prediction of molecular properties in drug discovery. However, this is fundamentally a low data problem that is incompatible with regular deep ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
1,468
Total Downloads

Downloads (Last 12 months)541
Downloads (Last 6 weeks)29

Reflects downloads up to

Other Metrics

View Author Metrics

Citations

Cited By

Du MXie XLuo JLi J(2024)Meta-learning-based Inductive logistic matrix completion for prediction of kinase inhibitorsJournal of Cheminformatics10.1186/s13321-024-00838-916:1Online publication date: 16-Apr-2024
https://doi.org/10.1186/s13321-024-00838-9
Fang YZhao XXiao Wde Rijke M(2024)Few-shot Learning for Heterogeneous Information NetworksACM Transactions on Information Systems10.1145/364931142:4(1-24)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3649311
Wang SDong YHuang XChen CLi J(2024)Learning Hierarchical Task Structures for Few-shot Graph ClassificationACM Transactions on Knowledge Discovery from Data10.1145/363547318:3(1-20)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3635473
Luo PZhu XXu TZheng YChen E(2024)Semantic Interaction Matching Network for Few-Shot Knowledge Graph CompletionACM Transactions on the Web10.1145/358955718:2(1-19)Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1145/3589557
Yue LLiu QLiu YGao WYao FLi WChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Cooperative Classification and Rationalization for Graph GeneralizationProceedings of the ACM on Web Conference 202410.1145/3589334.3645332(344-352)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645332
Yao QShen ZWang YDou D(2024)Property-Aware Relation Networks for Few-Shot Molecular Property PredictionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336809046:8(5413-5429)Online publication date: Aug-2024
https://doi.org/10.1109/TPAMI.2024.3368090
Liang LXu ZSong ZKing IQi YYe J(2024)Tackling Long-Tailed Distribution Issue in Graph Neural Networks via NormalizationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.331528436:5(2213-2223)Online publication date: May-2024
https://doi.org/10.1109/TKDE.2023.3315284
Wu KYang XWang ZLi NZhang JLiu L(2024)Data-balanced transformer for accelerated ionizable lipid nanoparticles screening in mRNA deliveryBriefings in Bioinformatics10.1093/bib/bbae18625:3Online publication date: 25-Apr-2024
https://doi.org/10.1093/bib/bbae186
Qian XJu BShen PYang KLi LLiu Q(2024)Meta Learning with Attention Based FP-GNNs for Few-Shot Molecular Property PredictionACS Omega10.1021/acsomega.4c021479:22(23940-23948)Online publication date: 23-May-2024
https://doi.org/10.1021/acsomega.4c02147
Svensson EHoedt PHochreiter SKlambauer G(2024)HyperPCM: Robust Task-Conditioned Modeling of Drug–Target InteractionsJournal of Chemical Information and Modeling10.1021/acs.jcim.3c0141764:7(2539-2553)Online publication date: 8-Jan-2024
https://doi.org/10.1021/acs.jcim.3c01417
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents