De novo Drug Design against SARS-CoV-2 Protein Targets using SMILES-based Deep Reinforcement Learning
Authors: 
Xiuyuan Hu, Yanghepu Li, Guoqing Liu, Yang Zhao, Hao Zhang, Liang ZhaoAuthors Info & Claims
Xiuyuan Hu, Yanghepu Li, Guoqing Liu, Yang Zhao, Hao Zhang, Liang ZhaoAuthors Info & ClaimsPages 161 - 166
Abstract
De novo drug design is an important task within the field of computer-aided drug design, and in recent years, numerous machine learning algorithms have been proposed for this purpose. The SARS-CoV-2 virus has posed a severe crisis to humanity over the past few years, making drug design targeting its protein targets a critical challenge. In this paper, we introduce a SMILES-based deep reinforcement learning algorithm to design small molecule inhibitors that bind well with SARS-CoV-2 targets. Experimental results demonstrate that our algorithm is capable of generating satisfactory drug candidates against SARS-CoV-2 protein targets and has the potential to be extended to other targets.
References
[1]
Sungsoo Ahn, Junsu Kim, Hankook Lee, and Jinwoo Shin. 2020. Guiding Deep Molecular Optimization with Genetic Exploration. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 12008–12021.
[2]
Viraj Bagal, Rishal Aggarwal, P. K. Vinod, and U. Deva Priyakumar. 2022. MolGPT: Molecular Generation Using a Transformer-Decoder Model. Journal of Chemical Information and Modeling 62, 9 (2022), 2064–2076.
[3]
Esben Jannik Bjerrum. 2017. SMILES enumeration as data augmentation for neural network modeling of molecules. arXiv preprint arXiv:1703.07076 (2017).
[4]
Thomas Blaschke, Josep Arús-Pous, Hongming Chen, Christian Margreitter, Christian Tyrchan, Ola Engkvist, Kostas Papadopoulos, and Atanas Patronov. 2020. REINVENT 2.0: an AI tool for de novo drug design. Journal of chemical information and modeling 60, 12 (2020), 5918–5922.
[5]
Thomas Blaschke, Josep Arús‐Pous, Hongming Chen, Christian Margreitter, Christian Tyrchan, Ola Engkvist, Kostas Papadopoulos, and Atanas Patronov. 2020. REINVENT 2.0: An AI Tool for De Novo Drug Design. Journal of Chemical Information and Modeling (2020).
[6]
Thomas Blaschke, Ola Engkvist, Jürgen Bajorath, and Hongming Chen. 2020. Memory-assisted reinforcement learning for diverse molecular de novo design. Journal of Chemical Information and Modeling (2020).
[7]
Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. 2019. GuacaMol: benchmarking models for de novo molecular design. Journal of chemical information and modeling 59, 3 (2019), 1096–1108.
[8]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. https://doi.org/10.18653/v1/N19-1423
[10]
Alice Douangamath, Daren Fearon, Paul Gehrtz, Tobias Krojer, Petra Lukacik, C David Owen, Efrat Resnick, Claire Strain-Damerell, Anthony Aimon, Péter Ábrányi-Balogh, 2020. Crystallographic and electrophilic fragment screening of the SARS-CoV-2 main protease. Nature communications 11, 1 (2020), 5047.
[11]
Yuanqi Du, Tianfan Fu, Jimeng Sun, and Shengchao Liu. 2022. Molgensurvey: A systematic survey in machine learning models for molecule design. arXiv preprint arXiv:2203.14500 (2022).
[12]
Peter Eckmann, Kunyang Sun, Bo Zhao, Mudong Feng, Michael K Gilson, and Rose Yu. 2022. LIMO: Latent Inceptionism for Targeted Molecule Generation. In International Conference on Machine Learning. PMLR.
[13]
Tianfan Fu, Cao Xiao, Lucas M Glass, and Jimeng Sun. 2021. MOLER: incorporate molecule-level reward to enhance deep generative model for molecule optimization. IEEE transactions on knowledge and data engineering 34, 11 (2021), 5459–5471.
[14]
Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. 2022. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in Neural Information Processing Systems 35 (2022), 21342–21357.
[15]
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. 2018. Automatic chemical design using a data-driven continuous representation of molecules. ACS central science 4, 2 (2018), 268–276.
[16]
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik. 2017. Objective-reinforced generative adversarial networks (ORGAN) for sequence generation models. arXiv preprint arXiv:1705.10843 (2017).
[17]
Nafisa M Hassan, Amr A Alhossary, Yuguang Mu, and Chee-Keong Kwoh. 2017. Protein-ligand blind docking using QuickVina-W with inter-process spatio-temporal integration. Scientific reports 7, 1 (2017), 15451.
[18]
Hauke S Hillen, Goran Kokic, Lucas Farnung, Christian Dienemann, Dimitry Tegunov, and Patrick Cramer. 2020. Structure of replicating SARS-CoV-2 polymerase. Nature 584, 7819 (2020), 154–156.
[19]
Xiuyuan Hu, Guoqing Liu, Yang Zhao, and Hao Zhang. 2023. De novo Drug Design using Reinforcement Learning with Multiple GPT Agents. In Thirty-seventh Conference on Neural Information Processing Systems.
[20]
Ross Irwin, Spyridon Dimitriadis, Jiazhen He, and Esben Jannik Bjerrum. 2022. Chemformer: a pre-trained transformer for computational chemistry. Machine Learning: Science and Technology 3, 1 (2022), 015022.
[21]
Wengong Jin, Regina Barzilay, and T. Jaakkola. 2020. Multi-Objective Molecule Generation using Interpretable Substructures. In International Conference on Machine Learning (ICML). PMLR, 4849–4859.
[22]
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research 4 (1996), 237–285.
[23]
B Ravi Kiran, Ibrahim Sobh, Victor Talpaert, Patrick Mannion, Ahmad A Al Sallab, Senthil Yogamani, and Patrick Pérez. 2021. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems 23, 6 (2021), 4909–4926.
[24]
Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. 2016. End-to-end training of deep visuomotor policies. The Journal of Machine Learning Research 17, 1 (2016), 1334–1373.
[25]
Junjie Li, Sotetsu Koyamada, Qiwei Ye, Guoqing Liu, Chao Wang, Ruihan Yang, Li Zhao, Tao Qin, Tie-Yan Liu, and Hsiao-Wuen Hon. 2020. Suphx: Mastering mahjong with deep reinforcement learning. arXiv preprint arXiv:2003.13590 (2020).
[26]
Guoqing Liu, Mengzhang Cai, Li Zhao, Tao Qin, Adrian Brown, Jimmy Bischoff, and Tie-Yan Liu. 2022. Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation. In 2022 IEEE Conference on Games (CoG). IEEE, 237–244.
[27]
Guoqing Liu, Di Xue, Shufang Xie, Yingce Xia, Austin Tripp, Krzysztof Maziarz, Marwin Segler, Tao Qin, Zongzhang Zhang, and Tie-Yan Liu. 2023. Retrosynthetic Planning with Dual Value Networks. arXiv preprint arXiv:2301.13755 (2023).
[28]
Guoqing Liu, Li Zhao, Pushi Zhang, Jiang Bian, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2021. Demonstration actor critic. Neurocomputing 434 (2021), 194–202.
[29]
David Mendez, Anna Gaulton, A Patrícia Bento, Jon Chambers, Marleen De Veij, Eloy Félix, María Paula Magariños, Juan F Mosquera, Prudence Mutowo, Michał Nowotka, 2019. ChEMBL: towards direct deposition of bioassay data. Nucleic acids research 47, D1 (2019), D930–D940.
[30]
Henry Moss, David Leslie, Daniel Beck, Javier Gonzalez, and Paul Rayson. 2020. Boss: Bayesian optimization over string spaces. Advances in neural information processing systems 33 (2020), 15476–15486.
[31]
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. 2017. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics 9 (2017).
[32]
Jerzy Osipiuk, Saara-Anne Azizi, Steve Dvorkin, Michael Endres, Robert Jedrzejczak, Krysten A Jones, Soowon Kang, Rahul S Kathayat, Youngchang Kim, Vladislav G Lisnyak, 2021. Structure of papain-like protease from SARS-CoV-2 and its complexes with non-covalent inhibitors. Nature communications 12, 1 (2021), 743.
[33]
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
[34]
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, 2018. Improving language understanding by generative pre-training. (2018).
[35]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
[36]
David M Rogers, Rupesh Agarwal, Josh V Vermaas, Micholas Dean Smith, Rajitha T Rajeshwar, Connor Cooper, Ada Sedova, Swen Boehm, Matthew Baker, Jens Glaser, 2023. SARS-CoV2 billion-compound docking. Scientific Data 10, 1 (2023), 173.
[37]
Victor T Sabe, Thandokuhle Ntombela, Lindiwe A Jhamba, Glenn EM Maguire, Thavendran Govender, Tricia Naicker, and Hendrik G Kruger. 2021. Current trends in computer aided drug design and a highlight of drugs discovered via computational techniques: A review. European Journal of Medicinal Chemistry 224 (2021), 113705.
[38]
Diogo Santos-Martins, Leonardo Solis-Vasquez, Andreas F Tillack, Michel F Sanner, Andreas Koch, and Stefano Forli. 2021. Accelerating AutoDock4 with GPUs and gradient-based local search. Journal of chemical theory and computation 17, 2 (2021), 1060–1073.
[39]
John S Schreck, Connor W Coley, and Kyle JM Bishop. 2019. Learning retrosynthetic planning through simulated experience. ACS central science 5, 6 (2019), 970–981.
[40]
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. 2018. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science 4, 1 (2018), 120–131.
[41]
Marwin HS Segler, Thierry Kogej, Christian Tyrchan, and Mark P Waller. 2018. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS central science 4, 1 (2018), 120–131.
[42]
Marwin HS Segler, Mike Preuss, and Mark P Waller. 2018. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555, 7698 (2018), 604–610.
[43]
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, 2016. Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2016), 484–489.
[44]
Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
[45]
Hampus Gummesson Svensson, Christian Tyrchan, Ola Engkvist, and Morteza Haghir Chehreghani. 2023. Utilizing Reinforcement Learning for de novo Drug Design. arXiv preprint arXiv:2303.17615 (2023).
[46]
Austin Tripp, Krzysztof Maziarz, Sarah Lewis, Guoqing Liu, and Marwin Segler. 2022. Re-evaluating chemical synthesis planning algorithms. In NeurIPS 2022 AI for Science: Progress and Promises.
[47]
Oleg Trott and Arthur J Olson. 2010. AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry 31, 2 (2010), 455–461.
[48]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[49]
Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, 2023. Scientific discovery in the age of artificial intelligence. Nature 620, 7972 (2023), 47–60.
[50]
Jike Wang, Chang-Yu Hsieh, Mingyang Wang, Xiaorui Wang, Zhenxing Wu, Dejun Jiang, Benben Liao, Xujun Zhang, Bo Yang, Qiaojun He, 2021. Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning. Nature Machine Intelligence 3, 10 (2021), 914–922.
[51]
David Weininger. 1988. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. Journal of chemical information and computer sciences 28, 1 (1988), 31–36.
[52]
Soojung Yang, Doyeong Hwang, Seul Lee, Seongok Ryu, and Sung Ju Hwang. 2021. Hit and lead discovery with explorative rl and fragment-based molecule generation. Advances in Neural Information Processing Systems 34 (2021), 7924–7936.
[53]
Naruki Yoshikawa, Kei Terayama, Masato Sumita, Teruki Homma, Kenta Oono, and Koji Tsuda. 2018. Population-based de novo molecule generation, using grammatical evolution. Chemistry Letters 47, 11 (2018), 1431–1434.
[54]
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. 2018. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems 31 (2018).
[55]
Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, 2023. Artificial Intelligence for Science in Quantum, Atomistic, and Continuum Systems. arXiv preprint arXiv:2307.08423 (2023).
[56]
Zhenpeng Zhou, Steven M. Kearnes, Li Li, Richard N. Zare, and Patrick F. Riley. 2019. Optimization of Molecules via Deep Reinforcement Learning. Scientific Reports 9 (2019).
Index Terms
- De novo Drug Design against SARS-CoV-2 Protein Targets using SMILES-based Deep Reinforcement Learning
Recommendations
The interaction of the bioflavonoids with five SARS-CoV-2 proteins targets: An in silico study
AbstractFlavonoids have been shown to have antioxidant, anti-inflammatory, anti-proliferative, antibacterial and antiviral efficacy. Therefore, in this study, we choose 85 flavonoid compounds and screened them to determine their in-silico ...
Highlights- Investigate the in-silico interaction of flavonoid compounds with protein targets crucial for SARS-CoV-2 infection.
Identification of Potential SARS-CoV-2 Main Protease Inhibitors Using Drug Repurposing and Molecular Modeling
Bioinformatics Research and ApplicationsAbstractStructure-based virtual screening of a molecular library of bioactive compounds was carried out to identify potential inhibitors against SARS-CoV-2 main protease (Mpro), an enzyme critically important for mediating viral replication and ...
Computational screening of 645 antiviral peptides against the receptor-binding domain of the spike protein in SARS-CoV-2
AbstractThe receptor-binding domain (RBD) of SARS-CoV-2 spike (S) protein plays a vital role in binding and internalization through the alpha-helix (AH) of human angiotensin-converting enzyme 2 (hACE2). Thus, it is a potential target for designing and ...
Graphical abstractDisplay Omitted
Highlights- Antiviral peptides can be a promising therapeutic strategy to inhibit SARS-CoV-2.
- 645 antiviral peptides were screened against RBD of the spike protein of SARS-CoV-2.
- 150 ns molecular dynamics simulation has been performed for ...
Comments
Information & Contributors
Information
Published In
December 2023
266 pages
ISBN:9798400709043
DOI:10.1145/3638985
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 March 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIT 2023
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 34Total Downloads
- Downloads (Last 12 months)34
- Downloads (Last 6 weeks)5
Reflects downloads up to 01 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format