Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3340531.3412179acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article
Open access

A Methodology Based on Deep Q-Learning/Genetic Algorithms for Optimizing COVID-19 Pandemic Government Actions

Published: 19 October 2020 Publication History

Abstract

Whenever countries are threatened by a pandemic, as is the case with the COVID-19 virus, governments need help to take the right actions to safeguard public health as well as to mitigate the negative effects on the economy. A restrictive approach can seriously damage the economy. Conversely, a relaxed one may put at risk a high percentage of the population. Other investigations in this area are focused on modelling the spread of the virus or estimating the impact of the different measures on its propagation. However, in this paper, we propose a new methodology for helping governments in planning the phases to combat the pandemic based on their priorities. To this end, we implement the SEIR epidemiological model to represent the evolution of the COVID-19 virus on the population.
To optimize the best sequences of actions governments can take, we propose a methodology with two approaches, one based on Deep Q-Learning and another one based on Genetic Algorithms. The sequences of actions (confinement, self-isolation, two-meter distance or not taking restrictions) are evaluated according to a reward system focused on meeting two objectives: firstly, getting few people infected so that hospitals are not overwhelmed, and secondly, avoiding taking drastic measures which could cause serious damage to the economy. The conducted experiments evaluate our methodology based on the accumulated rewards during the established period. The experiments also prove that it is a valid tool for governments to reduce the negative effects of a pandemic by optimizing the planning of the phases. According to our results, the approach based on Deep Q-Learning outperforms the one based on Genetic Algorithms.

Supplementary Material

MP4 File (3340531.3412179.mp4)
Whenever countries are threatened by a pandemic, as is the case with the COVID-19 virus, governments need help to take the right actions to safeguard public health as well as to mitigate the negative effects on the economy. In this piece of investigation, we propose a new methodology for helping governments in planning the phases to combat the pandemic based on their priorities. To optimize the best sequences of actions governments can take, we propose a methodology with two approaches, one based on Deep Q-Learning and another one based on Genetic Algorithms. The experiments prove that ours is a valid tool for governments to reduce the negative effects of a pandemic by optimizing the planning of the phases.

References

[1]
Mohammed AA Al-qaness, Ahmed A Ewees, Hong Fan, and Mohamed Abd El Aziz. 2020. Optimization method for forecasting confirmed cases of COVID-19 in China. Journal of Clinical Medicine 9, 3 (2020), 674.
[2]
Ahmad Alimadadi, Sachin Aryal, Ishan Manandhar, Patricia B Munroe, Bina Joe, and Xi Cheng. 2020. Artificial intelligence and machine learning to fight Covid-19.
[3]
Sina F Ardabili, Amir Mosavi, Pedram Ghamisi, Filip Ferdinand, Annamaria R Varkonyi-Koczy, Uwe Reuter, Timon Rabczuk, and Peter M Atkinson. 2020. COVID-19 Outbreak Prediction with Machine Learning. Available at SSRN 3580188 (2020).
[4]
Thomas Back. 1994. Selective pressure in evolutionary algorithms: A characterization of selection mechanisms. In Proceedings of the first IEEE conference on evolutionary computation. IEEE World Congress on Computational Intelligence. IEEE, 57--62.
[5]
Robert J Barro, José F Ursúa, and Joanna Weng. 2020. The coronavirus and the great influenza pandemic: Lessons from the ?spanish flu? for the coronavirus's potential effects on mortality and economic activity. Technical Report. National Bureau of Economic Research.
[6]
Mucahid Barstugan, Umut Ozkaya, and Saban Ozturk. 2020. Coronavirus (covid- classification using ct images by machine learning methods. arXiv preprint arXiv:2003.09424 (2020).
[7]
Tian-Mu Chen, Jia Rui, Qiu-Peng Wang, Ze-Yu Zhao, Jing-An Cui, and Ling Yin. 2020. A mathematical model for simulating the phase-based transmissibility of a novel coronavirus. Infectious Diseases of Poverty 9, 1 (2020), 1--8.
[8]
Alexander D'Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, D Sculley, and Yoni Halpern. 2020. Fairness is not static: deeper understanding of long term fairness via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 525--534.
[9]
Ludovic Denoyer and Patrick Gallinari. 2014. Deep Sequential Neural Network. arXiv:cs.LG/1410.0510
[10]
Alberto d'Onofrio. 2002. Stability properties of pulse vaccination strategy in SEIR epidemic model. Mathematical biosciences 179, 1 (2002), 57--72.
[11]
Nuno Fernandes. 2020. Economic effects of coronavirus outbreak (COVID-19) on the world economy. Available at SSRN 3557504 (2020).
[12]
Yiyue Ge, Tingzhong Tian, Sulin Huang, Fangping Wan, Jingxin Li, Shuya Li, Hui Yang, Lixiang Hong, Nian Wu, Enming Yuan, et al. 2020. A data-driven drug repositioning framework discovered a potential therapeutic agent targeting COVID-19. bioRxiv (2020).
[13]
Ophir Gozes, Maayan Frid-Adar, Hayit Greenspan, Patrick D Browning, Huangqi Zhang, Wenbin Ji, Adam Bernheim, and Eliot Siegel. 2020. Rapid ai development cycle for the coronavirus (covid-19) pandemic: Initial results for automated detection & patient monitoring using deep learning ct image analysis. arXiv preprint arXiv:2003.05037 (2020).
[14]
Matthew Hausknecht and Peter Stone. 2015. Deep recurrent q-learning for partially observable mdps. In 2015 AAAI Fall Symposium Series.
[15]
Feng He, Yu Deng, and Weina Li. 2020. Coronavirus disease 2019: What we know? Journal of medical virology (2020).
[16]
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, et al. 2018. Deep q-learning from demonstrations. In Thirty-Second AAAI Conference on Artificial Intelligence.
[17]
Zixin Hu, Qiyang Ge, Li Jin, and Momiao Xiong. 2020. Artificial intelligence forecasting of covid-19 in china. arXiv preprint arXiv:2002.07112 (2020).
[18]
Natsuko Imai, Anne Cori, Ilaria Dorigatti, Marc Baguelin, Christl A Donnelly, Steven Riley, and Neil M Ferguson. 2020. Report 3: transmissibility of 2019-nCoV. In Imperial College London.
[19]
Kia Jahanbin and Vahid Rahmanian. 2020. Using Twitter and web news mining to predict COVID-19 outbreak. Asian Pacific Journal of Tropical Medicine 13 (2020).
[20]
William Ogilvy Kermack and Anderson G McKendrick. 1927. A contribution to the mathematical theory of epidemics. Proceedings of the royal society of london. Series A, Containing papers of a mathematical and physical character 115, 772 (1927), 700--721.
[21]
Harshad Khadilkar, Tanuja Ganu, and Deva P Seetharam. 2020. Optimising Lockdown Policies for Epidemic Control using Reinforcement Learning. arXiv preprint arXiv:2003.14093 (2020).
[22]
Chih-Cheng Lai, Tzu-Ping Shih, Wen-Chien Ko, Hung-Jen Tang, and Po-Ren Hsueh. 2020. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and corona virus disease-2019 (COVID-19): the epidemic and the challenges. International journal of antimicrobial agents (2020), 105924.
[23]
Lin Li, Lixin Qin, Zeguo Xu, Youbing Yin, Xin Wang, Bin Kong, Junjie Bai, Yi Lu, Zhenghan Fang, Qi Song, et al. 2020. Artificial intelligence distinguishes COVID-19 from community acquired pneumonia on chest CT. Radiology (2020), 200905.
[24]
Michael Y Li and James S Muldowney. 1995. Global stability for the SEIR model in epidemiology. Mathematical biosciences 125, 2 (1995), 155--164.
[25]
Pieter Libin, Arno Moonens, Timothy Verstraeten, Fabian Perez-Sanjines, Niel Hens, Philippe Lemey, and Ann Nowé. 2020. Deep reinforcement learning for large-scale epidemic control. arXiv preprint arXiv:2003.13676 (2020).
[26]
Changliu Liu. 2020. A Microscopic Epidemic Model and Pandemic Prediction Using Multi-Agent Reinforcement Learning. arXiv preprint arXiv:2004.12959 (2020).
[27]
Anice C Lowen, Samira Mubareka, John Steel, and Peter Palese. 2007. Influenza virus transmission is dependent on relative humidity and temperature. PLoS Pathog 3, 10 (2007), e151.
[28]
Hayden C Metsky, Catherine A Freije, Tinna-Solveig F Kosoko-Thoroddsen, Pardis C Sabeti, and Cameron Myhrvold. 2020. CRISPR-based COVID-19 surveillance using a genomically-comprehensive machine learning approach. bioRxiv (2020).
[29]
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
[30]
Wim Naudé. 2020. Artificial Intelligence against COVID-19: An early review. (2020).
[31]
World Health Organization. [n.d.]. Coronavirus disease (COVID-19) Pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019
[32]
World Health Organization et al. 2020. Rational use of personal protective equipment for coronavirus disease (COVID-19): interim guidance, 27 February 2020. Technical Report. World Health Organization.
[33]
Martin L Puterman. 2014. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons.
[34]
Gurjit S Randhawa, Maximillian PM Soltysiak, Hadi El Roz, Camila PE de Souza, Kathleen A Hill, and Lila Kari. 2020. Machine learning using intrinsic genomic signatures for rapid classification of novel pathogens: COVID-19 case study. Plos one 15, 4 (2020), e0232391.
[35]
Arni SR Srinivasa Rao and Jose A Vazquez. 2020. Identification of COVID-19 can be quicker through artificial intelligence framework using a mobile phone--based survey when cities and towns are under quarantine. Infection Control & Hospital Epidemiology (2020), 1--5.
[36]
Fei-Yue Wang, Jun Jason Zhang, Xinhu Zheng, Xiao Wang, Yong Yuan, Xiaoxiao Dai, Jie Zhang, and Liuqing Yang. 2016. Where does AlphaGo go: From churchturing thesis to AlphaGo thesis and beyond. IEEE/CAA Journal of Automatica Sinica 3, 2 (2016), 113--120.
[37]
Linda Wang and Alexander Wong. 2020. COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images. arXiv preprint arXiv:2003.09871 (2020).
[38]
Christopher JCH Watkins and Peter Dayan. 1992. Q-learning. Machine learning 8, 3--4 (1992), 279--292.
[39]
Helen J Wearing, Pejman Rohani, and Matt J Keeling. 2005. Appropriate models for the management of infectious diseases. PLoS medicine 2, 7 (2005).
[40]
Darrell Whitley. 1994. A genetic algorithm tutorial. Statistics and computing 4, 2 (1994), 65--85.
[41]
Li Yan, Hai-Tao Zhang, Yang Xiao, Maolin Wang, Chuan Sun, Jing Liang, Shusheng Li, Mingyang Zhang, Yuqi Guo, Ying Xiao, et al. 2020. Prediction of survival for severe Covid-19 patients with three clinical features: development of a machine learning-based prognostic model with clinical data in Wuhan. medRxiv (2020).
[42]
Zifeng Yang, Zhiqi Zeng, Ke Wang, Sook-San Wong, Wenhua Liang, Mark Zanin, Peng Liu, Xudong Cao, Zhongqiang Gao, Zhitong Mai, et al. 2020. Modified SEIR and AI prediction of the epidemics trend of COVID-19 in China under public health interventions. Journal of Thoracic Disease 12, 3 (2020), 165.

Cited By

View all
  • (2024)Context-aware composition of agent policies by Markov decision process entity embeddings and agent ensemblesSemantic Web10.3233/SW-23353115:4(1443-1471)Online publication date: 4-Oct-2024
  • (2024)Intelligent Blocking and Prevention of SARS-CoV-2 Based on Evolutionary Reinforcement Learning2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10611941(1-8)Online publication date: 30-Jun-2024
  • (2024)A review of big data analytics models for assessing non-pharmaceutical interventions for COVID-19 pandemic managementJournal of Management Analytics10.1080/23270012.2024.237263211:3(358-388)Online publication date: 30-Jul-2024
  • Show More Cited By

Index Terms

  1. A Methodology Based on Deep Q-Learning/Genetic Algorithms for Optimizing COVID-19 Pandemic Government Actions

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
      October 2020
      3619 pages
      ISBN:9781450368599
      DOI:10.1145/3340531
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 October 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. COVID-19
      2. SEIR model
      3. coronavirus
      4. deep q-learning
      5. genetic algorithms
      6. reinforcement learning
      7. sequential decision making

      Qualifiers

      • Research-article

      Conference

      CIKM '20
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

      Upcoming Conference

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)149
      • Downloads (Last 6 weeks)21
      Reflects downloads up to 04 Oct 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Context-aware composition of agent policies by Markov decision process entity embeddings and agent ensemblesSemantic Web10.3233/SW-23353115:4(1443-1471)Online publication date: 4-Oct-2024
      • (2024)Intelligent Blocking and Prevention of SARS-CoV-2 Based on Evolutionary Reinforcement Learning2024 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC60901.2024.10611941(1-8)Online publication date: 30-Jun-2024
      • (2024)A review of big data analytics models for assessing non-pharmaceutical interventions for COVID-19 pandemic managementJournal of Management Analytics10.1080/23270012.2024.237263211:3(358-388)Online publication date: 30-Jul-2024
      • (2024)Modeling and control of COVID-19 disease using deep reinforcement learning methodMedical & Biological Engineering & Computing10.1007/s11517-024-03153-5Online publication date: 28-Jun-2024
      • (2023)Explaining Deep Q-Learning Experience Replay with SHapley Additive exPlanationsMachine Learning and Knowledge Extraction10.3390/make50400725:4(1433-1455)Online publication date: 9-Oct-2023
      • (2023)Optimal Risk-aware POI Recommendations during EpidemicsProceedings of the 4th ACM SIGSPATIAL International Workshop on Spatial Computing for Epidemiology10.1145/3615898.3628256(1-12)Online publication date: 13-Nov-2023
      • (2023)Comparing Countries On COVID-19 Government MeasuresProceedings of the 2023 7th International Conference on Medical and Health Informatics10.1145/3608298.3608359(337-342)Online publication date: 12-May-2023
      • (2023) Forecasting COVID ‐19 cases using dynamic time warping and incremental machine learning methods Expert Systems10.1111/exsy.1323740:6Online publication date: 24-Jan-2023
      • (2023)Complex Network Analysis with Deep Q- Learning for Predicting Patient Flow in Hospital Systems2023 International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI)10.1109/ICAIIHI57871.2023.10489237(1-6)Online publication date: 29-Dec-2023
      • (2023)Multivariate time series short term forecasting using cumulative data of coronavirusEvolving Systems10.1007/s12530-023-09509-w15:3(811-828)Online publication date: 4-Jun-2023
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Get Access

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media