Scott 

et al. BMC Public Health (2023) 23:988 BMC Public Health

https://doi.org/10.1186/s12889-023-15936-w

RESEARCH Open Access

COVID‑19 epidemic modelling for policy

decision support in Victoria, Australia 2020–
2021
Nick Scott1,2*†, Romesh G Abeysuriya1,2†, Dominic Delport1, Rachel Sacks‑Davis1,2, Jonathan Nolan3,
Daniel West3, Brett Sutton3, Euan M Wallace3 and Margaret Hellard1,2,4,5,6 

Abstract 
Background  Policy responses to COVID-19 in Victoria, Australia over 2020–2021 have been supported by evidence
generated through mathematical modelling. This study describes the design, key findings, and process for policy
translation of a series of modelling studies conducted for the Victorian Department of Health COVID-19 response
team during this period.
Methods  An agent-based model, Covasim, was used to simulate the impact of policy interventions on COVID-19
outbreaks and epidemic waves. The model was continually adapted to enable scenario analysis of settings or policies
being considered at the time (e.g. elimination of community transmission versus disease control). Model scenarios
were co-designed with government, to fill evidence gaps prior to key decisions.
Results  Understanding outbreak risk following incursions was critical to eliminating community COVID-19 transmis‑
sion. Analyses showed risk depended on whether the first detected case was the index case, a primary contact of
the index case, or a ‘mystery case’. There were benefits of early lockdown on first case detection and gradual easing
of restrictions to minimise resurgence risk from undetected cases. As vaccination coverage increased and the focus
shifted to controlling rather than eliminating community transmission, understanding health system demand was
critical. Analyses showed that vaccines alone could not protect health systems and need to be complemented with
other public health measures.
Conclusions  Model evidence offered the greatest value when decisions needed to be made pre-emptively, or for
questions that could not be answered with empiric data and data analysis alone. Co-designing scenarios with policy-
makers ensured relevance and increased policy translation.
Keywords  COVID-19, Mathematical model, Outbreak analysis, Disease control

†
Nick Scott and Romesh G. Abeysuriya are equal first authors.
*Correspondence:
Nick Scott
Nick.Scott@burnet.edu.au
Full list of author information is available at the end of the article

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecom‑
mons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Scott et al. BMC Public Health (2023) 23:988 Page 2 of 12

Introduction that continued to grow despite extensive lockdowns

Australia’s COVID-19 strategy has changed markedly and public health measures in place. In parallel, vaccine
over 2020–2022. Starting from an extended period where availability was increasing, prompting a transition in
tight border controls, contact tracing and lockdowns COVID-19 strategy from an aim of eliminating commu-
were used to eliminate community COVID-19 trans- nity transmission to an aim of slowing the spread until
mission, in mid-2021 and as vaccines became available, high population-level vaccine protection was established
public health restrictions were instead used to main- [3]. The context of a simultaneous growing epidemic,
tain health system capacity by controlling disease rather increasing vaccine coverage and change in COVID-19
than eliminating community transmission. In the state of strategy required the development of a roadmap for eas-
Victoria, Australia, the government used epidemiologi- ing restrictions. The key questions during this phase of
cal modelling extensively over this period to assess the the outbreak centred around the interaction between
health-related outcomes of potential policies. Model out- restrictions and vaccine coverage, and ensuring that
comes were used to complement information from other health system demand did not exceed capacity.
sources including analysis of epidemiological data, eco- By December 2021, over 90% two-dose vaccine cov-
nomic forecasting, international experience and expert erage among people over 16 years had been reached in
opinion to inform decision-making on COVID-19 policy Victoria and transmission of the Delta variant appeared
responses. to be stable in the community with minimal restrictions
Australia’s first wave of COVID-19 was mainly driven in place [4]. Subsequently, the emergence of the Omicron
by imported infections from international travel [1]. All variant triggered a large epidemic wave, the management
Australian States and Territories implemented lock- of which is still ongoing.
downs, which stopped local transmission and led to local Throughout these different epidemic stages, our mod-
elimination of COVID-19 (‘COVID-zero’) by June 2020 elling team worked closely with the Victorian govern-
in seven of eight jurisdictions. This initial success shaped ment to conduct analyses to estimate COVID-19-related
Australia’s COVID-19 pre-vaccination phase policies. outcomes from different policy options. In this paper we
Victoria was the only exception to this initial achieve- present a selection of analyses that were conducted at
ment of COVID-zero, experiencing an outbreak of the different stages of the pandemic and describe how the
wild-type variant from June-November 2020. This out- modelling was used to inform decision-making at critical
break comprised distinct periods of epidemic growth times. This is useful to understand the circumstances and
(June-August) where a sequence of public health restric- ways in which modelling can be most valuable for policy
tions were introduced to control it, and epidemic decline making.
(August-November) where restrictions were incre-
mentally eased as case numbers decreased. During this
period, key policy questions centred around how to relax Methods
restrictions without epidemic resurgence. Following this Model overview
outbreak, community COVID-19 transmission was elim- We used an established agent-based microsimulation
inated from Victoria [2]. model, Covasim [5, 6], developed by the Institute for
Between November 2020 and July 2021, Victoria con- Disease Modelling (USA) and collaborators including
tinued to eliminate community COVID-19 transmis- the Burnet Institute, to model epidemics in Melbourne
sion through international travel restrictions, hotel [6–8]. The model is open source and available online [9].
quarantine of international arrivals, and reactive city In brief, agents in the model are assigned an age (which
or state-wide lockdowns to assist intense contact trac- affects their susceptibility to infection and disease prog-
ing following incursions into the community. Relatively nosis), a household, a school (for people aged 5–17) or
brief lockdowns were implemented on three occasions a workplace (for people over 18, up to 65), and they par-
following such incursions. Decisions around the scale of ticipate in a number of community activities that may
the response for each outbreak had to be made follow- include attending restaurants, pubs, places of worship,
ing only a small number of detected cases and based on community sport, and small social gatherings. Details of
an incomplete understanding of how much transmis- included contact types, network structures, transmission
sion had already occurred. Key questions during this dynamics and disease outcomes in the latest model ver-
time focused on how quickly restrictions needed to be sion are provided in the supplement, though the model
imposed, the duration of restrictions, and level of restric- was continually adapted as analysis questions changed to
tions needed to contain outbreaks. enable scenario analysis of settings or policies relevant
Between July 2021 and November 2021, Victoria expe- for critical decisions, and so not all features were used for
rienced an outbreak of the more infectious Delta variant all analyses presented in this paper.
 Scott et al. BMC Public Health (2023) 23:988 Page 3 of 12

Interventions Scenario type 1: prospective outbreak analyses

The model includes vaccination (including individual (elimination strategy context)
dosing schedules, vaccine types and waning immu- The model was initialized with no cases, public health
nity), testing (PCR or rapid antigen tests), contact settings were varied, and one or more incursions were
tracing (with probability of tracing contacts depending simulated. The model was stochastic so onward transmis-
on the setting the contact occurs, the capacity of the sion did not occur in every simulation, but where trans-
system and the tracing policy at the time), quarantine mission did occur, it could lead to case detection through
of close contacts, isolation of confirmed cases, masks, symptomatic testing or surveillance testing interventions.
and a variety of policy restrictions to prevent or reduce After detection, policy options could be triggered to con-
transmission in different settings (e.g. closing schools tain the outbreak. The main objective was to assess how
or workplaces, density limits in hospitality and retail different public health settings balanced the intensity and
settings, restrictions on social gathering sizes). Further duration of restrictions against the risk of the outbreak
details are provided in the supplement. growing out of control.
The principal output measure was the percentage of
simulations where the epidemic reached different sizes
Calibration and SARS‑CoV‑2 variants over a fixed period (e.g., 90 days). Simulations were often
Model parameters for transmission and testing were categorized as no cases detected; some cases detected
continually calibrated and adjusted to fit data on daily but the virus eventually eliminated; or the 7-day average
new detected cases, hospital demand and ICU demand daily detected cases reaching different threshold levels by
for each new analysis and variant. For the wild-type the end of the model simulation.
variant, this was based on calibration to data over the
June-November 2020 epidemic wave. For the Alpha Scenario type 2: reactive outbreak analyses (elimination
and Kappa variants, this was based on international strategy context)
literature and data on the relative transmissibility and Outbreak analyses were calibrated to replicate outbreak
severity compared with the wild-type variant (since characteristics at the analysis date. Simulations were
there were no major outbreaks in Australia). For the sampled and rejected if they were not within +/- 10% of
Delta variant this was based on calibration to data over the cumulative detected cases. Similarly, model simula-
the Aug-Sep 2021 epidemic wave. tions could be filtered to include only those where par-
ticular numbers of ‘mystery cases’ (diagnosed cases for
which modelled contract tracing could not find a trans-
Scenario types mission source) were detected. This filtered set of model
As the epidemic progressed, the types of scenarios simulations were conditioned on the observed state of
examined changed along with the policies under con- outbreak so far, and were used to generate projections for
sideration and key questions. Four representative the impact of prospective interventions. This meant that
scenario types were retrospectively identified for pres- when an outbreak had begun with a series of ‘unlucky’
entation in this paper (Table 1). events at the start (such as chain of superspreading
events), that could be captured in the model.

Table 1  Representative scenario types presented in this paper

Scenario type Outcomes Context Calibration

Prospective outbreak • Probability of outbreak growing under No community cases; elimination No parameter fitting, sensitivity analyses
different theoretical circumstances strategy for key parameters
Reactive outbreak • Probability of existing outbreak reaching Small number of diagnosed cases fol‑ No parameter fitting, filter outbreaks to
different levels or being eliminated lowing period of no community cases; match observed
• Time required to contain outbreak elimination strategy
Easing restrictions • Timing and magnitude of epidemic Large number of cases; elimination Fit model to cases/hospital demand/
peak strategy deaths
• Probability of resurgence following eas‑
ing of restrictions
Managing health • Timing and magnitude of epidemic Large number of cases; epidemic control
system utilization peak(s) strategy
• Peak hospital/ICU demand
• Number of deaths
Scott et al. BMC Public Health (2023) 23:988 Page 4 of 12

As with the prospective outbreak analyses, the main impact of dynamically introducing or easing restrictions
output measure was the percentage of simulations reach- in the context of higher case numbers. The key outcome
ing different categories of 7-day average daily detected measures were hospital and ICU demand rather than the
cases and key questions focused on the duration of number of cases.
restrictions required to contain the outbreak.
Results
Scenario type 3: easing restrictions (elimination strategy An overview of overarching strategy and when principal
context) scenario types were used is shown in Fig. 1, overlaid on
For situations where a large outbreak had occurred and case numbers and vaccine coverage for epidemic context.
restrictions had already been imposed, policy questions
arose about when to ease restrictions and whether to do Easing restrictions example: wild‑type epidemic wave,
it incrementally or collectively. Model scenarios were leading to elimination (September 2020)
calibrated analogously to the constrained outbreak analy- Following a wild-type epidemic wave in Victoria (June-
ses, where the model was initialized with a small num- Nov 2020) that was contained by imposing restrictions,
ber of cases and simulations were only retained if they an analysis was conducted on 14 September to assess
were consistent with the actual outbreak, accounting for the impact of the timing of easing restrictions on resur-
restrictions imposed to date. Using only the retained sim- gence risk (Fig. 2) [8]. Model calibration involved running
ulations, scenarios were run comparing outcomes with simulations starting from zero cases, sampling over ini-
restrictions in place for different periods. tial seed infections (i.e. who in the model is the first case)
The main outcome measure was the probability of and uncertainty ranges of calibrated transmission param-
reaching > N diagnoses per day following the easing of eters, and retaining simulations within a threshold of the
restrictions (i.e., “resurgence risk”), and how this varied case data (Fig.  2, grey lines). For a set of 1000 retained
according to the timing and extent that restrictions were simulations, scenarios were applied to a short-term pro-
eased. jection to compare the impact of easing restrictions
on different dates. The red and blue lines in Fig. 2 show
Scenario type 4: health system utilization (control strategy individual simulations for each restriction scenario, high-
context) lighting the broad range of outcomes possible. For each
As the broader COVID-19 strategy transitioned from scenario, a proportion of simulations trend to zero while
elimination to control, health system utilization became a proportion result in a resurgence of infections, defined
increasingly relevant. Scenarios were calibrated similarly as reaching a threshold of daily diagnoses. This analysis
to the easing restrictions analyses, and compared the showed that an additional two weeks of restrictions could

Fig. 1  Victorian COVID-19 timeline. New daily cases and vaccine coverage are shown over time, contextualising the overall strategic phase
(elimination or suppression) and the dominant scenario type being run at different stages of the pandemic (vertical shaded regions)
 Scott et al. BMC Public Health (2023) 23:988 Page 5 of 12

Fig. 2  Wild-type epidemic wave in Victoria in 2020, in the context of an elimination COVID-19 strategy, with scenarios considering resurgence risk
if restrictions were eased on 14 Sep (red) or 28 Sep (blue). Model simulations were started with random infected seed cases and randomly sampled
transmission parameters, and were retained if they were within sufficient bounds of the observed data. Throughout the simulations ‘Stage 3’, masks
and ‘Stage 4’ restrictions were imposed, with their impact derived through model calibration (see [8])

more than halve the risk of a resurgence in cases. Factor- days of the start of the simulation (e.g. >30 in Fig. 3). All
ing this and other evidence, restrictions were maintained simulations included contact tracing (including second-
and cases continued to decline, before being gradually ring tracing for some contacts).
eased from 28 September (e.g. small gatherings of five Several factors were identified that influenced out-
people outdoors). break risk including the infectiousness of the variant,
the number of generations of transmissions before the
Prospective outbreak analysis example (October 2020) first detected case, and the policy response to the ini-
After the incidence of community acquired cases tial case detection. The number of infection generations
returned to zero from 29 to 2020, outbreak risks from prior to the first diagnosis being recorded was particu-
potential incursions were assessed. Simulations started larly predictive of the outcome. In simulations where the
with one undiagnosed case. The outbreak risk was index case was the first identified, light restriction con-
defined as the percentage of simulations that reached a tained the outbreak 94% of the time (Fig. 3). However, if
particular 7-day average diagnosis threshold within 90 the index case was not the first case detected, this made

Fig. 3  Outbreak analysis in Victoria in 2021, in the context of pursuing an elimination strategy. Simulations were run starting from zero cases and a
randomly infected seed case. Table shows the proportion of simulations where an outbreak occurred (defined as reaching a 7-day average of > 30
diagnoses/day within 90 days), according to which infection generation was detected, the infectiousness of the virus (baseline = wild type; 50%
more infectious ~ Alpha variant) and what responses were implemented on detection of the first case (light = masks and limits on social gatherings;
moderate = light + density limits; heavy = lockdown)
Scott et al. BMC Public Health (2023) 23:988 Page 6 of 12

containment much more difficult. At the time of analysis, the first few diagnoses. In May 2021 an outbreak of the
the Alpha variant was estimated to be 50% more infec- Kappa variant occurred, with the first diagnosis on 24
tious than the wild type variant [10, 11], and in simula- May, and 45 and 58 cases were diagnosed cumulatively
tions where an Alpha variant infection was identified in within the next 7 and 10 days, respectively. Mandatory
the community and the index case was not the first case masks were imposed on the first day, and a lockdown was
diagnosed, then even a moderate response (mask man- imposed on the third day.
dates, limits on social gatherings, and density limits) Modelling was used to estimate the duration of the
would be unlikely to contain it (Fig.  3). These analyses lockdown needed to contain the outbreak. Simulations
highlighted the importance of interventions to increase started with a single case and were retained if transmis-
case detection (e.g., asymptomatic screening of quaran- sion occurred (i.e., the incursion did not fizzle out), the
tine hotel workers). index case was never diagnosed, and there were between
The impact of the timing of the response to an outbreak 43 and 65 diagnoses after 7 days and 53–80 diagnoses
was also assessed. For early variants (wild-type, Alpha), after 12 days. Using 1000 simulations that met these
heavy restrictions were always able to contain outbreaks, criteria (sampling over initial seeds, contact network
but a delay in introducing restrictions increased the dura- structures and transmission parameters), scenarios were
tion of restrictions to contain the outbreak (Fig.  4). The run for policy changes under consideration, where after
analysis showed that if some restrictions were already two weeks of lockdown (June 11), either the lockdown
active at the time of the incursion, the duration of restric- was maintained, or restrictions were eased by: opening
tions would also be shorter. schools only; opening schools and venues with density
This supported a “go hard go early” approach to restric- limit; opening schools, venues with density limits and
tions in the elimination context, which was applied when allowing small social gatherings; or returning to masks
lockdowns were imposed from 13 to 17 February 2021 only. The impact of these policy changes on daily diag-
after 13 Alpha variant cases were detected, from 28 May noses four weeks after implementation was recorded as
to 3 June 2021 (later extended to 10 June) after 30 Kappa the main outcome measure (Fig. 5). The analysis quanti-
variant cases were detected, and from 16 July to 20 July fied how much the risk of a resurgence increased when
(later extended to 27 July) after 18 Delta variant cases restrictions were eased more generously, and enabled
were detected. After each lockdown restrictions were resurgence risk to be incorporated into decision-making
incrementally eased, rather than lifted at once. alongside the costs associated with the lockdown.
Based on this and other evidence, from 11 June schools
Reactive outbreak analysis example: Kappa variant were re-opened, hospitality was reopened with den-
outbreak (May 2021) sity limits, small gatherings were allowed but only out-
Once an incursion was detected, outbreak risk esti- doors, and work from home and mask mandates were
mates were refined by incorporating characterisations of maintained.

Fig. 4  Outbreak time to containment. Following an outbreak, the time required in lockdown to return to < 5 diagnoses per day, according to the
delay in implementing restrictions. Lines show median and inter-quartile range (error bars) across 1000 simulations. Based on the wild type variant
 Scott et al. BMC Public Health (2023) 23:988 Page 7 of 12

Fig. 5  Outbreak analysis in Victoria in 2021, when an outbreak has been detected but limited information is known, pursuing an elimination
strategy. Top: Simulations were run starting from zero cases and a randomly infected seed case. Simulations were only retained if they produced,
through stochastic variation, approximately the same initial outbreaks as occurred. In this example it was 43–65 diagnoses within 7 days of the first
diagnosis, and 53–80 diagnoses within 12 days of the first diagnosis. This is against a background of masks and work from home if possible being
implemented on day 1, and lockdown being implemented on day 3. Bottom: From the retained simulations, scenarios compared whether the
lockdown was maintained, or after 14 days either schools were reopened, venues were opened with density limits, small social gatherings were also
allowed, or a return to masks and working from home only

Prospective outbreak analysis example: context of vaccine from home) could potentially reduce COVID-19-related
rollout and coverage (March 2021) mortality to a similar rate as experienced in the 2017
When vaccination coverage increased, other public influenza season (a year with particularly high influenza-
health responses were also expected to change. In the related mortality), depending on vaccine efficacy. In
context of vaccines becoming available, outbreak risk addition, a trade-off was identified between the degree
(defined as the percentage of simulations reaching dif- of restrictions and the duration, where similar case num-
ferent diagnosis thresholds over the first 90 days after a bers could be achieved with a short duration of strict
single case) was assessed by vaccine type and coverage restrictions, or longer duration of light restrictions. This
(Fig. 6). While outbreak risk reduced with increasing vac- showed that even with high vaccine coverage, without
cine coverage, it was clear that herd immunity through NPIs the health impacts of COVID-19 would be much
vaccination was unlikely to be achieved, supporting an worse than influenza, but that intermittent light restric-
eventual move from an elimination strategy to a control tions could considerably improve outcomes.
strategy.
Managing health system utilization example: Delta variant
Managing health system utilization example: moving epidemic wave (Sept 2021)
from elimination to control (July 2021) Prior to the Delta variant, NPIs were sufficient to contain
Later we investigated the feasibility of a range of possi- outbreaks in unvaccinated populations. However, in mid-
ble control strategies, from intermittent restrictions to 2021 Victoria experienced a large Delta variant epidemic
removal of all non-pharmaceutical interventions (NPIs) wave before high coverage of vaccination was achieved
[12, 13]. Since vaccines were highly efficacious but imper- (as of 5 August only 43% of Victorians over 16 years had
fect, removing all NPIs was found to come at consider- received one dose and 21% two doses). With the Delta
able health cost (Fig.  7). This analysis showed that with variant having a shorter serial interval that reduced the
the Delta variant, intermittent low-level restrictions (for effectiveness of contact tracing, and community fatigue
example, high levels of testing, mask mandates and work leading to declining compliance with restrictions,
Scott et al. BMC Public Health (2023) 23:988 Page 8 of 12

Fig. 6  Outbreak analysis in Victoria in 2021, over 90 days starting from a single case and different levels of population vaccine coverage. Left:
AstraZeneca type vaccine, with assumed protection against infection of 50%, and protection against symptomatic disease of 65%. Right: Pfizer type
vaccines, with assumed protection against infection of 80%, and protection against symptomatic disease of 90%. TPOD = total protection against
disease; POI = protection against infection. Note that vaccine efficacy estimates were based on best available data as at March 2021

community transmission continued to increase despite a phase, community incursions from hotel quarantine reg-
lockdown being imposed. Plans to transition away from ularly occurred, and modelling was used as a critical tool
an elimination approach were accelerated, with a focus to understand outbreak risks and how they varied in dif-
on using NPIs to control the epidemic while conducting a ferent circumstances or with different responses.
mass vaccination campaign. Modelling was used to assess In the elimination phase, model outputs showed out-
the implications of different strategies (i.e., sequences break risk depended on whether the detected case was
and timings of restrictions being eased, relative to the the incursion, first ring of infections or a ‘mystery case’.
vaccination rollout and case numbers) on infections and There were benefits of early lockdown on detection of the
health system demand [14, 15]. This analysis showed that first case and exiting lockdowns slowly through gradual
maintaining testing among those vaccinated was likely to easing of restrictions to minimise resurgence risk from
be necessary for limiting transmission, and early actions undetected cases. Key metrics in this period were the
to achieve transmission reduction could have significant number of cases, the intensity of restrictions, and the
downstream implications due to rising vaccine cover- duration of restrictions, as returning to COVID-zero was
age (Fig. 8). The roadmap that was developed ultimately viewed as the most efficient way to meet other targets.
included incremental easing of restrictions in line with The emergence of COVID-19 vaccines meant that
different vaccination coverage thresholds. catastrophic health outcomes could be avoided with-
out the need for some of the stricter border control
Discussion and lockdown measures. This, combined with the fact
After Australia’s initial response successfully eliminated that the elimination of community transmission was no
community COVID-19 transmission from most jurisdic- longer realistic due to the emergence of more infectious
tions in early 2020, and as the dire health consequences of variants, led to a move from an elimination to a control
widespread transmission in the absence of a vaccine were strategy. While high vaccine coverage was an essential
observed internationally, Australia embarked on a phase element, our analyses indicated that sustainable con-
lasting until mid-2021 where similar catastrophic health trol strategies would require vaccines plus intermittent
outcomes were avoided by using tight border controls, ‘light’ restrictions, introduced early in times of case
contact tracing and restrictions to achieve and maintain number escalation, to avoid excess case and health sys-
the elimination of community transmission. During this tem demand escalation. During the critical transition
 Scott et al. BMC Public Health (2023) 23:988 Page 9 of 12

Fig. 7  Projected COVID-19 deaths in Victoria over a 12-month period under different outbreak response scenarios, triggered by case numbers
(left), and average time spent under restrictions (right). Top: Grey bars show the projected number of deaths on a log scale, with the lighter shading
indicating the proportion that are among the vaccinated population. Scenarios are based on 95%/70%/70% vaccine coverage being achieved
among people aged 60+/12–59/<12 years, the vaccine having an assumed 80% protection against infection and 92% protection against death,
and 5 cases per day were seeded into the community through reduced quarantine measures. Light restrictions = mandatory masks, density limits
and work from home if possible; medium restriction = light restrictions + additional limits on gathering sizes; lockdown = mandatory masks, work
from home, schools closed, retail closed, hospitality take-away only, social gatherings up to two outdoors only. Bottom: example time series output
from one simulation, where increasing restrictions are triggered with increased hospital number thresholds, to maintain disease control

period from an elimination to a control strategy, and answered directly with data, this meant that the ration-
as vaccine coverage increased, the focus of analyses ale for the use of modelling was always clear.
shifted to health system demand, with case numbers The utility of modelling was also increased by the co-
becoming less important. design of scenarios with government and other relevant
In all analyses conducted, model evidence was most stakeholders [16]. Scenarios were only modelled if (a)
crucial for decision making when used to investigate they could not be shown to be superior / inferior based
questions that could not be answered in any other way. on existing data; (b) they were considered reasonable by
For example, where relevant data was available, direct epidemiologists and public health teams; and (c) they
analysis was significantly faster to inform decisions and were being seriously considered by government given
required fewer assumptions to be made. Model analy- other economic, social, and political implications. Many
ses were therefore typically used to support decisions policy options could be ruled out before even making it
that needed to be made pre-emptively (e.g., how long to to the modelling stage, because they were not likely to
lockdown for, when minimal information was available) be feasible for other reasons. For example, working with
[16]. Since the primary aims of each study could not be stakeholders ensured that scenarios could incorporate
Scott et al. BMC Public Health (2023) 23:988 Page 10 of 12

Fig. 8  2021 roadmap with reduced testing for people vaccinated (blue), maintained testing for people vaccinated (red), and an additional 15%
reduction in non-household transmission (green). Scenarios include schools returning to in person learning throughout October; childcare
returning and mobility restrictions easing in October; limited outdoor gatherings at 70% two-dose vaccine coverage among people 16 + years;
indoor gathering with density limits at 80% two-dose coverage among people 16 + years and mandatory vaccine requirements. Dashed vertical
lines represent estimated dates of reaching 70% and 80% two-dose coverage among people 16 + years

operational or capacity constraints in the delivery of ser- evidence. However, in the case of an outbreak, decisions
vices such as testing or vaccination. Co-designing the often need to be made quickly, with significant impacts on
scenarios also helped align model outputs with other the lives of large numbers of people. It is therefore imprac-
evidence under consideration, to make the modelling as tical to have outputs peer-reviewed in advance, which can
informative as possible. For example, running epidemic take months. Review and validation of raw model outputs
projections for similar scenarios to those separately by government analysts on standby was conducted in place
used for economic forecasts facilitates integrating both of such checks, and where feasible identical requests were
sources of evidence. In addition, the continued involve- often put to other modelling groups [17, 18] to be con-
ment of the modelling team with public health teams ducted simultaneously and independently [19, 20].
allowed ongoing refinement and improvement of the Detailing how these analyses were conducted is impor-
modelling, and validation against past accuracy. tant because the role of modelling in informing policy is
There were many challenges associated with the need not well understood. Epidemiological modelling is a spe-
for fast decision-making. Ideally, modelling analyses would cialized discipline that experienced increased media atten-
undergo peer review before being used to inform decisions. tion because of its utility to generate evidence to support
The purpose of the peer-review process is to ensure that decision-making in the COVID-19 pandemic; however, it
model design, assumptions and inputs are fit-for-purpose, is often not communicated well through the media [19].
consistent with best practice, and incorporate the latest There are common misconceptions about the differences
 Scott et al. BMC Public Health (2023) 23:988 Page 11 of 12

between forecasting (trying to predict the specific course must be used in conjunction with other information
of the epidemic) and scenario analyses (comparing projec- sources to make policy decisions.
tions under different policy options to estimate the impact
of policies and inform decisions) [21]. This often results in Supplementary Information
a dichotomisation of modelling being viewed by the public The online version contains supplementary material available at https://​doi.​
as being right or wrong, with many of the key insights gen- org/​10.​1186/​s12889-​023-​15936-w.

erated by the analyses being missed. This is compounded

Additional file 1. 
by the presence of uncertainty at multiple levels, including
in data, model parameters, and stochastic model outputs.
Acknowledgements
Communicating uncertainty and confidence is challeng-
NS and MH are the recipients of National Health and Medical Research Council
ing, but critical to interpreting model findings. A strength fellowships. The authors gratefully acknowledge the support provided by the
of modelling that can be difficult to communicate is that Victorian Government Operational Infrastructure Support Program.
even in the presence of high uncertainty in forward epi-
Authors’ contributions
demic projections, there may be low uncertainty in what is All authors conceived the studies together, based on epidemiology at the
a superior intervention or policy option [21]. time. All authors designed scenarios. NS, RGA, DD, RSD developed the model
and implemented the scenarios in the model. NS and RGA drafted the manu‑
There are many limitations in the use of modelling to
script. All authors contributed to the final version of the manuscript.
inform COVID-19 decisions. First, models are simplifica-
tions of the real world and cannot capture the full range Funding
Parts of this work were directly funded by the Victorian Government Depart‑
of human behaviour. Second, there are many things that
ment of Health and Human Services (now Department of Health).
are unknown about COVID-19 disease dynamics, that
models must approximate with the evolving results of Availability of data and materials
This study involved the use of an agent-based model, and input parameters
clinical trials and observational studies. Third, parame-
for the model are available in the supplementary materials. The model code is
ters continue to change over time. As evidence for model available from https://​github.​com/​Insti​tutef​orDis​easeM​odeli​ng/​covas​im.
inputs like vaccine efficacy or human behaviour become
clearer over time as more data becomes available, results Declarations
change and this can make analyses done at different
Ethics approval and consent to participate
times appear to be contradictory. Fourth, compro-
Not applicable as the study did not involve any participants. Model inputs are
mises are required to constrain levels of detail in order available in the supplementary material.
to meet timelines for when decisions need to be made.
Consent for publication
Finally, epidemiological models typically only focus on
Not applicable.
health outcomes (and sometimes health-related costs).
Government decisions have repercussions throughout Competing interests
NS, RGA, DD, RSD, and MH have received funding from the Victorian Depart‑
the community for a wide range of stakeholders, and
ment of Health (DoH), NSW DoH and the Federal Government for modelling
policy decisions must necessarily account for factors not related to COVID-19. MH receives funding from the Victorian Government
included in the model. It is important that model analyses (DoH, Department of Families, Fairness and Housing, and Department of Jobs,
Precincts and Regions) and the Macquarie Foundation to undertake work
acknowledge this and avoid making policy recommenda-
monitoring the impact of COVID-19 on the community. JN, DW BS and EMW
tions outside of their ability to meet specific health tar- have nothing to declare.
gets. Overall, greater transparency around modelling and
Author details
how it was used would help to educate the community, 1
 Disease Elimination Program, Burnet Institute, 85 Commercial Rd, Melbourne,
media and governments more broadly on the strengths Victoria, Australia. 2 Department of Epidemiology and Preventive Medicine,
and limitations of these analyses, and the ways in which Monash University, Melbourne, Victoria, Australia. 3 Victorian Government
Department of Health, Victoria, Australia. 4 School of Population and Global
modelling is (and is not) used in decision making.
Health, The University of Melbourne, Parkville, Victoria, Australia. 5 Department
of Infectious Diseases, The Alfred and Monash University, Melbourne, Victoria,
Australia. 6 Department of Infectious Diseases, Doherty Institute, The University
Conclusion of Melbourne, Parkville, Victoria, Australia.
Model evidence offers the greatest value to support
COVID-19 decision making when decisions need to be Received: 12 February 2023 Accepted: 19 May 2023
made pre-emptively, or to answer questions that cannot
be answered with data analysis alone. Models need to
adapt to evolving COVID-19 strategies and should have
References
scenarios and outcome metrics co-designed with policy- 1. Fotheringham P, Anderson T, Shaw M, Jewitt J, Storey H, Hutchings O,
makers and other stakeholders. Epidemiological models Cartwright J, Gupta L. Control of COVID-19 in Australia through quar‑
cannot cover all outcomes from policy changes, particu- antine: the role of special health accommodation (SHA) in New South
Wales, Australia. BMC Public Health. 2021;21(1):1–8.
larly when timelines for decision-making are short, and
Scott et al. BMC Public Health (2023) 23:988 Page 12 of 12

2. Victorian Government Department of Health and Human Services: State‑

ment from the Premier – 26 October 2020. 2020. Accessed 23 May 2022
from https://​www.​dhhs.​vic.​gov.​au/​updat​es/​coron​avirus-​covid-​19/​state​
ment-​premi​er-​26-​octob​er-​2020.
3. Premier of Victoria: Victoria’s Roadmap: Delivering The National Plan.
2021. Accessed 23 May 2022 from https://​www.​premi​er.​vic.​gov.​au/​victo​
rias-​roadm​ap-​deliv​ering-​natio​nal-​plan.
4. Premier of Victoria: More Freedoms To Come As Victoria Hits 90 Per
Cent Vax Rate (18. November 2021). 2021. Accessed 23 May 2022
from https://​www.​premi​er.​vic.​gov.​au/​more-​freed​oms-​come-​victo​
ria-​hits-​90-​cent-​vax-​rate.
5. Kerr CC, Stuart RM, Mistry D, Abeysuriya RG, Rosenfeld K, Hart GR, Núñez
RC, Cohen JA, Selvaraj P, Hagedorn B, et al. Covasim: an agent-based
model of COVID-19 dynamics and interventions. PLoS Comput Biol.
2021;17(7):e1009149.
6. Scott N, Palmer A, Delport D, Abeysuriya R, Stuart R, Kerr CC, Mistry D,
Klein DJ, Sacks-Davis R, Heath K, et al. Modelling the impact of relaxing
COVID-19 control measures during a period of low viral transmission.
Med J Aust. 2021;214(2):79–83.
7. Abeysuriya RG, Delport D, Stuart RM, Sacks-Davis R, Kerr CC, Mistry D,
Klein DJ, Hellard M, Scott N. Preventing a cluster from becoming a new
wave in settings with zero community COVID-19 cases. BMC Infect Dis.
2022;22(1):1–15.
8. Abeysuriya R, Delport D, Hellard M, Scott N. Estimating risks associated
with early reopening in Victoria. Burnet Institute Public Health Report.
2020.
9. Covasim model GitHub repository. https://​github.​com/​Insti​tutef​orDis​
easeM​odeli​ng/​covas​im. Accessed 24 May 2023.
10. Davies NG, Abbott S, Barnard RC, Jarvis CI, Kucharski AJ, Munday JD,
Pearson CA, Russell TW, Tully DC, Washburne AD. Estimated transmis‑
sibility and impact of SARS-CoV-2 lineage B. 1.1. 7 in England. Science.
2021;372(6538):eabg3055.
11. Volz E, Mishra S, Chand M, Barrett JC, Johnson R, Geidelberg L, Hinsley
WR, Laydon DJ, Dabrera G, O’Toole Á. Assessing transmissibility of SARS-
CoV-2 lineage B. 1.1. 7 in England. Nature. 2021;593(7858):266–9.
12. Abeysuriya R, Hellard M, Scott N. COVASIM modelling: impact of vaccines
on epidemic outcomes. Burnet Institute Public Health Report. 2021.
13. Abeysuriya R, Hellard M, Scott N. Long-term COVID-19 control requires
high vaccination and intermittent control measures. Burnet Institute
Public Health Report. 2021.
14. Abeysuriya R, Delport D, Sacks-Davis R, Hellard M, Scott N. Modelling the
Victorian roadmap. Burnet Institute Public Health Report. 2021.
15. Abeysuriya R, Delport D, Sacks-Davis R, Hellard M, Scott N. Melbourne
COVID-19 roadmap modelling. Burnet Institute Public Health Report.
2021.
16. McBryde ES, Meehan MT, Adegboye OA, Adekunle AI, Caldwell JM, Pak
A, Rojas DP, Williams BM, Trauer JM. Role of modelling in COVID-19 policy
development. Paediatr Respir Rev. 2020;35:57–60.
17. Thompson J, McClure R, Blakely T, Wilson N, Baker MG, Wijnands JS, et al.
Modelling SARS-CoV-2 disease progression in Australia and New Zealand:
an account of an agent-based approach to support public health
decision-making. Aust N Z J Public Health. 2022;46(3):292–303.
18. Thompson J, Carvalho N, Blakely T. The probability of the 6-week
lockdown in Victoria (commencing 9 July 2020) achieving elimination of
community transmission of SARS‐CoV‐2. Med J Australia. 2021;215(2):96–
6. e91.
19. James LP, Salomon JA, Buckee CO, Menzies NA. The use and misuse of
Ready to submit your research ? Choose BMC and benefit from:
mathematical modeling for infectious disease policymaking: lessons for
the COVID-19 pandemic. Med Decis Making. 2021;41(4):379–85.
• fast, convenient online submission
20. Thompson J, McClure R, Scott N, Hellard M, Abeysuriya R, Vidinaarachichi
R, et al. A framework for considering the utility of models when facing • thorough peer review by experienced researchers in your field
tough decisions in public health: a guideline for policy-makers. Health • rapid publication on acceptance
Res Policy Syst. 2022;20(1):1–7.
• support for research data, including large and complex data types
21. Jahn B, Friedrich S, Behnke J, Engel J, Garczarek U, Münnich R, et al. On
the role of data, statistics and decisions in a pandemic. Adv Stat Anal. • gold Open Access which fosters wider collaboration and increased citations
2022;106(3):349–82. • maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in pub‑ Learn more biomedcentral.com/submissions
lished maps and institutional affiliations.