Submitted Manuscript: Confidential template updated: February 28 2012 Considerations for the design of vaccine efficacy trials during public health emergencies Natalie E. Dean1*, Pierre-Stéphane Gsell2, Ron Brookmeyer3, Victor De Gruttola4, Christl A. Donnelly5, M. Elizabeth Halloran6,7, Momodou Jasseh8, Martha Nason9, Ximena Riveros2, Conall H. Watson10, Ana Maria Henao-Restrepo2, Ira M. Longini, Jr.1* 1 Department of Biostatistics, University of Florida, Gainesville, FL, USA. 2 World Health Organization, Geneva, Switzerland. 3 Department of Biostatistics, University of California, Los Angeles, CA, USA. 4 Department of Biostatistics, Harvard University, Boston, MA, USA. 5 MRC Centre for Global Infectious Disease Analysis, Department of Infectious Disease Epidemiology, School of Public Health, Imperial College London, London, UK. 6 Vaccine and Infectious Disease Division, Fred Hutchinson Cancer Research Center, Seattle, WA, USA. 7 Department of Biostatistics, University of Washington, Seattle, WA, USA. 8 Medical Research Council, The Gambia at London School of Hygiene & Tropical Medicine, Fajara, The Gambia. 9 Biostatistics Research Branch, National Institute of Allergy and Infectious Diseases, Rockville, MD, USA. 10 Department of Infectious Disease Epidemiology, London School of Hygiene & Tropical Medicine, London, UK. *To whom correspondence should be addressed: nataliedean@ufl.edu or ilongini@ufl.edu, Dauer Hall, PO Box 117450, Gainesville, FL 32611, USA. +1 (352) 294-1945 One Sentence Summary: As part of the WHO research and development Blueprint for action to prevent epidemics, we describe key considerations for the design and analysis of trials and studies to evaluate experimental vaccines during public health emergencies. Abstract: Public Health Emergencies (PHEs) provide a complex and challenging environment for vaccine evaluation. Under the R&D Blueprint Plan of Action, the World Health Organization (WHO) has convened a group of experts to agree on standard procedures to rapidly evaluate experimental vaccines during PHEs while maintaining the highest scientific and ethical standards. The Blueprint priority diseases, selected for their likelihood to cause PHEs and the lack of adequate medical countermeasures, were used to frame our methodological discussions. Here, we outline major vaccine study designs to be used in PHEs and summarize high-level recommendations for their use in this setting. We recognize that the epidemiology and 1 transmission dynamics of the Blueprint priority diseases may be highly uncertain and that the unique characteristics of the vaccines and outbreak settings may affect our study design. To address these challenges, our group underscores the need for novel, flexible, and responsive trial designs. We conclude that assignment to study groups using randomization is a key principle underlying rigorous study design and should be utilized except in exceptional circumstances. Advance planning for vaccine trial designs is critical for rapid and effective response to a PHE and to advance knowledge to address and mitigate future PHEs. Introduction The recent Ebola and Zika public health emergencies (PHEs) have demonstrated that the global community was not prepared to evaluate vaccines in affected countries, despite several decades of research into vaccine development on emerging pathogens (1). Preclinical and early clinical studies had not been completed for vaccine candidates. There was inadequate coordination between governments, non-governmental organizations (NGOs), and the private sector. Infrastructure for conducting clinical research in affected areas was limited and strained by the outbreak response. The timeline for writing, approving, and implementing protocols was dramatically compressed. Epidemics of pathogens with no licensed vaccine will undoubtedly emerge in the future, and the public health community must be prepared to rapidly evaluate experimental vaccines in such circumstances. Our group of statisticians, clinical trialists, infectious disease modelers, and researchers was convened by WHO under the R&D Blueprint Plan of Action (2) with the mission to develop a consensus on study designs to rapidly evaluate vaccine candidates that address scientific, ethical and logistical issues arising in PHEs. We used the Blueprint priority diseases (3) to frame our discussions, to illustrate our rationale on key methodological considerations, and to anticipate future challenges. The list of Blueprint priority diseases is to be updated annually by an expert panel. The 2018 list includes Crimean-Congo haemorrhagic fever (CCHF), Ebola virus disease and Marburg virus disease, Lassa fever, Middle East respiratory syndrome coronavirus (MERS-CoV) and Severe Acute Respiratory Syndrome (SARS), Nipah and henipaviral diseases, Rift Valley fever (RVF), Zika, and Disease X (i.e., a future unknown threat). The main goal of a vaccine efficacy trial is to obtain efficacy and effectiveness data that can support broader use of a vaccine under a defined regulatory framework. In the context of an outbreak, vaccine evaluation also provides a way to give access, in the affected communities, to the most promising experimental vaccines and potentially to help control the current outbreak should the vaccine prove to be effective. In this process, we need to ensure that the experimental vaccine is demonstrated to be safe and effective and that it is used with an adequate community engagement and delivery strategy. Conducting vaccine evaluation in PHEs is associated with methodological and operational challenges (4, 5). The epidemiology of the infectious disease, technological aspects, sociocultural aspects, and outbreak circumstances affect the choices we make when designing a vaccine trial or study. We generally have limited knowledge about the transmission dynamics and the natural history of the Blueprint priority diseases. These pathogens are prone to epidemics where the spatiotemporal 2 incidence of the disease may be highly variable and unpredictable. Unlike endemic diseases, outbreaks end or are contained to a point such that only sporadic cases occur. Furthermore, outbreaks may typically last only a few weeks and it may take one to two weeks for an outbreak to be detected and confirmed. In settings with poor surveillance, it may take even longer. These epidemiological and operational aspects make it difficult for studies to identify, enroll, and vaccinate at-risk participants prior to exposure, as well as to define the appropriate endpoints to estimate vaccine efficacy and effectiveness. Given the sense of urgency that may arise, very little may also be known about the vaccine candidate itself in terms of safety and immunogenicity in humans, but also in terms of thermostability and other properties. Importantly, vaccine evaluation may also take place in a setting with unvalidated and unstandardized diagnostics and/or serologic assays, which poses considerable challenges for case ascertainment and endpoint measurement. Outbreak circumstances are complex, and each outbreak has different characteristics. Typically, a PHE may trigger the rapid development of a large number of vaccine candidates that could be tested in affected countries if the outbreak persists. As a result, trial sponsors may compete for study sites and populations. In addition, research in epidemic management is relatively new. The conduct of research needs to be fully integrated into the international effort to control the disease, and should not be performed at the expense of the broader response to a PHE. Finally, there may be fears and misconceptions among the affected communities. Involving communities in the study implementation and complying with good participatory practices for research (6) are essential to increase acceptability of the intervention and preserve the integrity of the trial. Because of the epidemiological situation and outbreak-working environment, we may not be able to conduct the perfect study. To address these challenges our group underscores the need for innovative, responsive and flexible study designs. As part of the Blueprint working group on vaccine evaluation, we present a summary of major vaccine study designs and design elements to be considered during PHEs of emerging and re-emerging pathogens for which there is no licensed vaccine. Results It is widely acknowledged that double-blind placebo-controlled, individually-randomized vaccine trials performed in a variety of sites and study populations provide robust evidence that may inform licensure and broader use of a vaccine. However, in special circumstances, in the context of PHEs, trialists may be compelled to consider alternative study designs. Here, we outline major study design elements and challenges that are specific to the Blueprint priority diseases and to the context of PHEs, and we illustrate some of the trade-offs and methodological options. 1. Study endpoints The challenges For a given pathogen, study endpoints should be selected to support the broader intended use of a vaccine, as described in the WHO vaccine target product profile (TPP) for a given pathogen (Table 1), and that are representative of the public health burden caused by that particular 3 pathogen. Two types of vaccines that are commonly desired in TPPs are: (a) fast-acting vaccines amenable to be used reactively during outbreaks to interrupt chains of transmission and terminate outbreaks, and (b) durable vaccines to be used preventatively in targeted populations to maximize the public health impact of the vaccine. Preventive vaccines are especially valuable for protecting against endemic diseases such as Lassa fever and may be prioritized for use in highrisk populations, such as health-care workers. For pathogens without a developed TPP (e.g., for Disease X), the same basic principles from other diseases are expected to apply, especially where the new pathogen produces an acute viral disease with a similar pattern of zoonotic spillover. In practice, it may not be feasible to have sufficient vaccine trial sample sizes with endpoints that are representative of the public health burden, such as clinical disease. In addition, if there are poor or limited diagnostics or limited infrastructure, endpoints requiring laboratory confirmation may be hard to detect. For instance, although cases of microcephaly represent the major public health burden associated with Zika infection, the choice of more frequent clinical events as a primary outcome measure for vaccine efficacy trials is likely to be necessary for feasible sample sizes (7). The justification of a mild, more common, endpoint as the primary endpoint in vaccine trials would be predicated on the assumption that the benefit of the vaccine on the selected endpoint is reasonably likely to predict clinical benefit for cases of microcephaly or other severe complications. The methodological options Methodological options include clinical disease endpoints, infection endpoints, or correlates of vaccine-induced protection. Clinical disease endpoints, such as severe disease or disease of any severity, may be clinically or laboratory confirmed. A clinical disease endpoint without laboratory confirmation should only be considered for pathogens with a highly distinct clinical syndrome, and these studies should consider laboratory testing of a random sample of cases to internally estimate the sensitivity and specificity of the case definition (8). For infection endpoints, detection of acute infection in the absence of clinical disease may require frequent laboratory testing and be operationally challenging. Detection of seroconversion would require an assay that can distinguish natural infection from vaccine-induced immunity. Studies are encouraged to collect serological data at baseline and post-vaccination to measure potential immunological correlates of vaccine-induced protection (9). Where available, validated immunological correlates can be used to infer the efficacy of a vaccine, but they are unlikely to exist for emerging pathogens or for novel vaccine platforms. Nonetheless, correlate data can be used along with other data sources to demonstrate a reasonable likelihood that the vaccine is efficacious when a clinical disease or infection endpoint is not feasible. For Zika vaccine efficacy trials, though there are likely many more asymptomatic infections than clinical disease cases, selecting an endpoint related to Zika infection would rely on a robust laboratory capacity and active surveillance system. However, licensed diagnostics for Zika infection are limited and serologic assays are cross-reactive with other arboviruses. Virologically-confirmed Zika clinical disease is a more feasible primary endpoint for a Zika vaccine efficacy trial because of the challenges of detecting infection endpoints, but Zika clinical disease will require a larger overall trial (7). Because Zika symptoms are non-specific and may be mistaken for other arboviral diseases, laboratory confirmation is critical. 4 Ebola (reactive use) Lassa (preventive use) MERS-CoV (reactive use) Nipah (reactive use) Zika (reactive use) Indication for Use For immunization of at-risk persons residing in the area of an on-going outbreak to protect against Ebola virus disease caused by circulating species of filovirus; to be used in conjunction with other control measures to curtail or end an outbreak. Preferred vaccine characteristics At least 80% efficacy in preventing Ebola virus disease in healthy adults, adolescents and children. For active immunization of persons considered potentially at-risk, based on specific risk factors, to protect against Lassa Fever disease. At least 90% efficacy in preventing Lassa virus infection or Lassa fever disease. All age groups. For active immunization of atrisk persons in the area of an on-going outbreak for the prevention of Middle East respiratory syndrome (MERS) caused by MERS-CoV; to be used in conjunction with other control measures to curtail or end an outbreak. At least 90% efficacy in preventing Middle East Respiratory Syndrome caused by MERS-CoV in healthy adults. All age groups. For active immunization of atrisk persons in the area of an on-going outbreak for the prevention of Nipah disease; to be used in conjunction with other control measures to curtail or end an outbreak. At least 90% efficacy in preventing Nipah virus infection or Nipah disease in healthy adults For the prevention of Zika virus-associated clinical illness of any severity in subjects 9 years of age or older. At least 80% efficacy in preventing virologically-confirmed Zika illness. Rapid onset of immunity (preferably less than 2 weeks). Preferred target population All age-groups and populations at high present risk of Ebola virus disease caused by circulating species of filovirus. Suitable for administration to pregnant women. Suitable for administration to pregnant women. Evidence of prevention of virus shedding. Rapid onset of immunity (less than 1 week). All age groups and populations at high risk of Nipah disease. Rapid onset of immunity (less than 2 weeks after first dose). Evidence of prevention of viremia. Women of reproductive age (including adolescent and preadolescent girls 9 years of age or older), and boys/men of the same ages. Table 1 – Examples of preferred characteristics extracted from the WHO vaccine Target Product Profiles (TPP) (10) for Ebola, Lassa, MERS-CoV, Nipah and Zika, some of the pathogens prioritized by WHO. The TPPs are pathogen rather than product specific, and define a mandatory set of product attributes such as indication, target population, dosing regimen, duration of protection, route of administration, safety and efficacy requirements. They also include criteria pertaining to product presentation, storage and shelf life. TPPs are developed through expert consultations 5 and intend to provide early technical guidance into the various product specific vaccine TPPs that are developed by individual vaccine manufacturers. Take home message The demonstration of the vaccine benefit based on a laboratory-confirmed clinical disease endpoint is the recommended way to evaluate a vaccine because it is often most representative of the public health burden of interest. In some settings, infection or other endpoints may be justified as proxies of the public health burden of interest. The use of immunological correlate data may be necessary if clinical disease or infection endpoints are not feasible. Study endpoints may differ from the vaccine TPP desired from a public health perspective, but the benefit must be validated in future studies. 2. Target population The challenges Trial target population should also be representative of the target population defined in the vaccine TPPs, or based on what is known about the epidemiology of the pathogen (Table 1). Likewise, it may not be feasible to have a sufficient sample size for a study population that is representative of the public health burden: for example, prevention of virologically-confirmed Zika infection in women of reproductive age (7). Typically, trials are implemented in sites with established high clinical attack rates and draw participants from a general population representative of those who would ultimately receive the vaccine. However, because the incidence of new cases is extremely variable in PHEs, it may be challenging to identify a population in a given area that is at-risk and fully susceptible to disease transmission. The methodological options Study populations can be drawn from the general population in areas at high geographic risk for transmission. Studies may narrow the target population to those with other risk factors that make them at highest risk of infection, such as occupation or contact with high-risk individuals. For example, individuals with direct contact with camels and their household contacts are at increased risk of MERS-CoV. A targeted approach may require a smaller overall sample size if the incidence is truly higher in these individuals, though it may be harder to identify, enroll, and track these participants than a general population. To take into account the high variability in disease transmission during an epidemic, we define a responsive target population as a study population that is triggered by the occurrence of a new case. In this regard, a responsive target population is designed to track the epidemic and focuses the intervention where the risk goes. For instance, the study population enrolled in the Ebola ring vaccination trial in Guinea (11) was a responsive contact-based study population where identification and enrollment of study participants was triggered by a confirmed case. For vectorborne diseases, such as Zika, the study population may be defined by geographic proximity to a case. This approach relies on a sensitive and rapidly responding surveillance system to inform the study in real-time as well as on a mobile and flexible vaccine delivery and possibly a cold chain. Such a design works best for single-dose vaccines that evoke a quick immune response 6 and for infectious diseases that spread relatively slowly through predictable contact networks. For rapidly spreading diseases, it may be necessary to use broader inclusion criteria in order to capture later generations of transmission. For example, while the typical ring strategy includes contacts and contacts of contacts of cases, one may add third-order contacts or everyone else residing within a fixed distance of the case. It may also be advantageous to monitor pre-selected sites to speed responsive vaccination. Lassa vaccine trials could include heightened surveillance in areas where cases are most frequently detected with rapid vaccination of participants when transmission is observed. The take home message Responsive trials are appropriate in the event of an epidemic where the transmission dynamics are extremely variable in space and time. Because they focus the intervention where the transmission and risk exposure are occurring, the statistical power is expected to increase and required sample size decrease. Computational disease modeling can be used to predict trial accrual rates and inform sample size selection (12). 3. Randomization The challenges Randomization provides assurance that the groups being compared are similar except for the vaccine being studied. The use of randomization was strongly debated in the context of the WestAfrican Ebola outbreak (13–15) because the use of randomization may deny persons an opportunity to have access to a potentially effective vaccine in a situation with high mortality and lack of adequate medical countermeasures. Groups of experts argued that randomized trials are the most reliable and rapid way to identify the relative benefits and risks of investigational products and that every effort should be made to implement designs with random group assignment during outbreaks and epidemics (14, 16). Our group concurs with the above statement. Randomized trials are the study design of choice in PHEs, and deviation from the use of randomized designs should occur only under very exceptional circumstances following a robust risk-benefit analysis. For instance, if there is sufficient evidence of the safety and effectiveness of an investigational vaccine and there is no satisfactory alternative, the use of randomization may raise ethical concerns and acceptability among the affected populations. The methodological options A schematic of the different forms of appropriate randomized vaccine trials is shown in Figure 1. The unit of randomization can be at the individual or cluster level with various levels of stratification as pertinent. 7 Figure 1: Schematic of randomized trial designs Randomization at the individual level In an individually randomized controlled trial (iRCT), participants are randomized within each study site. Sites could be defined responsively or from natural groupings of people at high risk of infection (e.g. health-care workers). iRCTs are statistically efficient designs, especially when there is substantial heterogeneity in incidence across study sites. The primary analysis estimates the individual-level reduction in susceptibility to disease or infection (“direct vaccine effect” or sometimes “vaccine efficacy” (VE)). Population-level effects of vaccination, including indirect protection, are typically not estimable (17). If indirect vaccine protection is very high, one concern is that transmission within the study site could be dramatically reduced in both arms such that it becomes difficult to measure VE (18). More than one vaccine candidate may be suitable for efficacy testing, in which case multi-arm trials sharing a single placebo or comparator vaccine arm are expected to require fewer resources than multiple, independent two-arm trials (19). From a public health perspective, this approach is attractive because it provides a method to simultaneously evaluate multiple vaccine candidates, has the potential to diversify the number and supply of vaccines available, and helps avoid monopoly situations. This approach has been determined to be optimal for Zika vaccine trials where future transmission will probably occur in different geographic clusters in pockets of still susceptible populations. Factorial trials permit simultaneous evaluation of a vaccine and an innovative non-vaccine intervention (e.g. vector control) targeting the same disease. For example, participants may be individually randomized to vaccine, and the non-vaccine intervention may be individual- or cluster-randomized. For diseases that spread in the environment, sites could be clusterrandomized to water, sanitation, and hygiene interventions, as has been implemented for cholera. 8 Factorial trials (iRCT of vaccination with either individual or cluster randomization to the nonvaccine intervention) conserve resources by utilizing the same population and trial infrastructure (20). Where the non-vaccine intervention is effective at reducing disease in the study population, though, the power to detect vaccine efficacy will also be reduced. Randomization at the cluster level In parallel cluster randomized controlled trials (parallel cRCTs), study sites (e.g. high-risk communities) or small groups (e.g. households) are randomized as a unit to vaccine or comparator. Clusters may be defined responsively, such as the contact-based rings in the Ebola ring vaccination trial (11), naturally capturing infectious disease transmission networks (21, 22). The primary analysis estimates total vaccine effectiveness, which measures the individual-level benefit of the vaccine resulting from the combination of direct and indirect vaccine effects (17). If data collection are expanded to include non-participants, the trial can generate estimates of indirect and overall effects. A form of this strategy was used in the Ebola ring vaccination trial (24). As parallel cRCTs may be difficult to blind, they are subject to a number of biases that can reduce interpretability of the results (23). Furthermore, clustered designs are less statistically efficient than individually randomized designs, especially when there is high heterogeneity across clusters. In stepped wedge cRCTs, vaccine is delivered to all clusters but in a randomized order. In PHEs, these designs have important disadvantages, primarily because they are complex to plan, implement, and analyze (21). Stepped wedge cRCTs are inflexible, as all participants and facilities must be enrolled before the first dose of vaccine can be administered (23). Stepped wedge cRCTs probably result in the slowest trials and are not well-suited for endpoints with spatiotemporally variable incidence (25–27). Two-stage randomized designs, in which clusters are randomized to a level of vaccine coverage (e.g. 20% or 80%) and participants are individually randomized to achieve this coverage, are one of the only designs to support relatively unbiased estimation of both direct and indirect vaccine effects (28). An important disadvantage of the design is its complexity, and there is no precedent for such design in vaccine trials. The take home message Despite the exceptional circumstances of a PHE, randomization, whether at the individual or cluster level, remains a key principle in vaccine evaluation. Deviation from the use of randomized designs should occur only under very exceptional circumstances. For PHEs, we recommend randomized trial designs that are compatible with the enrollment of a responsive target population. For estimating vaccine efficacy, individual randomization within responsively defined sites will typically require the smallest overall sample size. cRCTs can provide an individual-level measure of vaccine efficacy through total effectiveness, as well as populationlevel indirect effects, while iRCTs only measure direct vaccine effects (17). 4. Comparator The challenges 9 A common model for evaluating and deploying a new vaccine against a disease for which there is no existing vaccine is that it is first tested in a trial controlled with a placebo or with an unrelated vaccine. The use of blinding (or masking), as is possible with the use of a control, reduces the potential for biases, such as selection, detection, and performance bias (29). Like with randomization, the use of a placebo has been strongly debated in the context of the WestAfrican Ebola outbreak (30) and will likely be debated in future PHEs. The methodological options Researchers should consider whether the risks associated with use of the placebo – that is the risks of the placebo intervention itself and those of withholding or delaying a vaccine with evidence of efficacy and effectiveness – are minimal, preventable or reversible. Risks greater than this may constrain the use of placebos. In PHEs, a delayed vaccination comparator may be adopted in which individuals/clusters are allocated to either immediate or delayed vaccination. Motivations for the use of a delayed comparator include improving acceptability, providing vaccine to individuals in greatest need, and averting more cases and promoting epidemic control if the vaccine is efficacious. However, if the vaccine is ineffective or dangerous, more people are exposed to the vaccine than would be in a trial with placebo or unrelated vaccine control. Trials using delayed vaccination are expected to have lower power than placebo-controlled trials, and the VE estimates may be biased (31). To reduce bias, the length of the delay should be relatively long compared to the disease incubation period and the time required for the immune response to develop among vaccinated people. A delayed vaccination approach was used as a comparator arm was implemented in the Ebola ring vaccination trial in Guinea (24, 32), and the Ebola iRCT trial in Sierra Leone (33). In settings where an existing vaccine has already been established to provide clinically meaningful benefit, an experimental vaccine may have potential advantages other than efficacy, such as having a more favorable tolerability or safety profile, being more convenient to store, transport, or administer, or less costly. It might be sufficient for the experimental vaccine to have similar rather than superior efficacy relative to the existing vaccine, which can be evaluated in a non-inferiority trial (34). Depending on the size of the non-inferiority margin (minimum threshold for an unacceptable loss of efficacy), non-inferiority trials may require large sample sizes that make them challenging in the PHE setting. The take home message Although the use of placebo or unrelated control vaccine provides a robust methodological standard, and can allow for blinding to protect against many real or perceived biases, the use of delayed vaccination can be explored in certain circumstances. 5. Primary analysis The challenges The estimated vaccine effects may be sensitive to the primary analysis, especially the inclusion of cases with illness onset shortly after vaccination. Cases that occur immediately after vaccination are likely the result of infection prior to vaccination and/or prior to the development 10 of a robust immune response. For responsive vaccination strategies, the period of highest incidence in the target population may be around the time of vaccination. The methodological options The primary analysis can be conducted per protocol, intention-to-treat (ITT), or modified ITT. The per protocol analysis restricts the population to eligible, fully compliant participants receiving all doses as allocated per protocol. The analysis often includes a delay, usually starting after the final dose of the vaccine plus the maximum incubation period, to allow the immune response to develop and to account for the time between infection and symptom onset. The goal of the per protocol analysis is to estimate the intrinsic efficacy of the vaccine to support licensure decisions and planning, but it is subject to post-randomization biases such as differential loss to follow-up. Alternatively, an ITT analysis includes all cases occurring after randomization or all cases occurring after the first dose of vaccine/placebo. The ITT analysis yields a practical, though more context-specific, estimate of vaccine effectiveness because it includes cases who may have been infected before the vaccine induced an immune response, as well as individuals who fail to comply with the protocol, potentially for reasons relating to the vaccine itself. As a result, the ITT estimate of VE tends to be attenuated compared to the per protocol estimate, and the difference between the ITT and per protocol estimates of VE may be especially large if many infections occur during the per protocol delay (31). In the modified ITT approach, a sensitive test is used to retrospectively exclude individuals infected at baseline (35), though this requires the availability of both baseline samples and a reliable test. Although ITT is generally regarded as the preferred approach in other types of clinical trials, VE trials frequently conduct a per protocol primary analysis because compliance is typically high (36). The take home message Though only a single primary analysis may be selected, both ITT and per protocol estimates of VE should be reported. 6. Data monitoring The challenges It is essential to rapidly identify safe and efficacious vaccines so that they may be transformed into an intervention that can influence the course of the outbreak. It is also important to discard futile or unsafe vaccines at the earliest opportunity so that limited resources can be rededicated to other promising candidates. In outbreaks, transmission among humans may decline to extremely low levels or stop entirely, precluding accrual of further evidence to directly evaluate vaccine efficacy. The methodological options Independent Data and Safety Monitoring Committees should be in place to safeguard the interests of study participants and to enhance the integrity and credibility of the vaccine trial (37). The design of the vaccine trial should include specification of data monitoring boundaries 11 allowing for early termination of the trial for benefit or for futility while controlling the trial’s type 1 error rate and preserving power. Group sequential guidelines, such as an O’Brien-Fleming boundary, provide a widely implemented approach (38, 39). The number and timing of interim analyses can be flexibly defined through an alpha spending method (40). If a trial is terminated early for efficacy, there should also be a plan in the protocol for next steps, which may include vaccinating all eligible, consenting, unvaccinated participants. These participants should then be followed for safety outcomes since the product would be unlicensed at that time. Following the promising results of the rVSV-ZEBOV vaccine against Ebola virus disease (32), ring vaccination with immediate vaccination only was used in Guinea in response to a flare-up of Ebola transmission several months after West Africa was declared Ebola-free, with the vaccine deployed under compassionate use criteria (41). Ring vaccination with rVSV-ZEBOV has also been used during the 2018 Ebola outbreaks in the Democratic Republic of Congo (42). In settings of changing epidemiology, the protocol should clarify how study data would be analyzed, including alpha spending if the full sample size is not reached. A waning epidemic could trigger study closure with a final analysis, study pause until the next outbreak occurs in that area, or study continuation to collect additional safety and immunogenicity data. Keeping the study open would also be desirable in case there is an unexpected surge in transmission. This decision could be guided by an evaluation including transmission modelling to assess the probability of future cases in the current outbreak or future outbreaks in the study area. This type of modelling has been used to inform likely case accrual for Ebola vaccine trials (43). Pausing the study protocol until the next outbreak occurs is proposed as a valuable strategy for accumulating evidence for the efficacy of an intervention. Especially for diseases with limited person-to-person transmission that primarily spillover from an animal reservoir, such as Nipah, any individual outbreak may be too small to fully power a trial. Where such a “master trial” approach is not feasible, at minimum there should be a prospectively defined strategy for merging separate trials of the same intervention, such as a fixed-effects meta-analysis. Research protocols should be aligned as much as possible, with central coordination by WHO with the ministries of health in the affected countries. The take home message The study protocol should include a flexible data monitoring strategy for efficacy and futility, and it should pre-specify plans for a waning epidemic. It is recommended that this include planning to continue the trial into a future outbreak. Discussion In this document, we have outlined major study designs and design elements to be considered in PHEs. Given the circumstances of PHEs and the epidemiological situation, we have underscored the need for responsive and flexible study designs while maintaining the highest scientific and ethical standards possible. Study endpoints should be selected to support the broader intended use of a vaccine and should reflect the public health burden of interest. The study population can be responsively defined or target high-risk individuals to increase statistical power. Individual or cluster randomization can be implemented, and trials can simultaneously evaluate multiple experimental vaccines to use limited resources more efficiently. Placebo control or the use of an 12 unrelated vaccine control is recommended, with trials blinded whenever possible, though delayed vaccination can be considered in certain settings. Both a per protocol and ITT analysis should be reported. Trials should pre-specify a monitoring strategy that is robust to changing epidemiology. A key principle is that randomized designs should be utilized whenever possible. Observational studies (e.g., cohort studies, case-control and test-negative designs (17, 44, 45)) should only be considered in certain limited settings because the quality of inference will always be viewed as inferior relative to a randomized design. A setting where observational studies may be useful is when the product of interest has received conditional licensure but needs to be further evaluated. Like in any observational study, collection of and adjustment for potential confounders is critical. Results of observational studies are easiest to interpret when the effect of the intervention is large enough so as to overshadow random error and bias, especially due to confounding (46). In rare settings where deemed ethical, human challenge studies, in which participants are intentionally exposed to the pathogen, may be used to support regulatory decisions, provided that that the human challenge model system is adequately predictive of efficacy in the field (47). Human challenge studies can use classical experimental designs and relatively small sample sizes to directly assess efficacy, safety, and immunogenicity of an experimental vaccine. To navigate through the various study design elements and options outlined here and to promote scientific discussion among methodologists, an interactive, web-based decision support tool has been developed (48). Mathematical models of infectious disease are also valuable to explore different assumptions and analyze their impact on the statistical power of a given vaccine trial design in a given epidemic scenario (49). Furthermore, our work on vaccine study design is one component of the larger Blueprint effort at WHO. Other workstreams include establishing a Global Coordination Mechanism to facilitate dialogue between relevant stakeholders. The Coalition for Epidemic Preparedness Innovations (CEPI) is one partner engaged in this work. CEPI aims to support the early development of experimental vaccines for prioritized pathogens, which is important for advancing candidates to efficacy testing (50). Many of the principles described here for vaccine studies can be expanded to therapeutic and prophylactic antimicrobial agents. By expanding these study designs and plans for all potential emerging infectious disease threats on the Blueprint priority disease list, we will be able to rigorously evaluate vaccine and antimicrobial efficacy and effectiveness at the earliest opportunity when an outbreak occurs, to mitigate current and future outbreaks. References and Notes: 1. S. Moon, D. Sridhar, M. A. Pate, A. K. Jha, C. Clinton, S. Delaunay, V. Edwin, M. Fallah, D. P. Fidler, L. Garrett, E. Goosby, L. O. Gostin, D. L. Heymann, K. Lee, G. M. Leung, J. S. Morrison, J. Saavedra, M. Tanner, J. A. Leigh, B. Hawkins, L. R. Woskie, P. Piot, Will Ebola change the game? Ten essential reforms before the next pandemic. The report of the Harvard-LSHTM Independent Panel on the Global Response to Ebola, Lancet 386, 2204–2221 (2015). 2. M. P. Kieny, P. Salama, WHO R&D Blueprint: a global coordination mechanism for R&D preparedness, Lancet 389, 2469–2470 (2017). 3. WHO, R&D Blueprint | List of Blueprint priority diseases. (available at http://www.who.int/blueprint/prioritydiseases/en/). 13 4. M. Nason, Statistics and logistics: Design of Ebola vaccine trials in West Africa, Clin. Trials 13, 87–91 (2016). 5. A. Vandebosch, R. Mogg, N. Goeyvaerts, C. Truyers, D. Watson-Jones, G. Herrera-Taracena, Simulation-guided phase 3 trial design to evaluate vaccine effectiveness to prevent Ebola virus disease infection : Statistical considerations , design rationale , and challenges, Clin. Trials 13, 57–65 (2016). 6. WHO, R&D Blueprint | C. Developing new norms and standards tailored to the epidemic context. (available at http://www.who.int/blueprint/what/norms-standards/en/). 7. WHO, R&D Blueprint | ZIKV workshop 1-2 June 2017 (available at http://www.who.int/blueprint/what/normsstandards/zikv_workshop-1-2june2017/en/). 8. M. E. Halloran, I. M. Longini, Using validation sets for outcomes and exposure to infection in vaccine field studies, Am. J. Epidemiol. 154, 391–398 (2001). 9. WHO, Correlates of vaccine-induced protection: methods and implications (WHO Press, Geneva, Switzerland, WHO/IVB/13., 2013; www.who.int/vaccines-documents/). 10. WHO, R&D Blueprint | B. Accelerating R&D processes. (available at http://www.who.int/blueprint/what/research-development/en/). 11. Ebola ça Suffit Ring Vaccination Trial Consortium, The ring vaccination trial: a novel cluster randomised controlled trial design to evaluate vaccine efficacy and effectiveness during outbreaks, with special reference to Ebola, Br. Med. J. 351, h3740 (2015). 12. M. E. Halloran, K. Auranen, S. Baird, N. E. Basta, S. E. Bellan, R. Brookmeyer, B. S. Cooper, V. DeGruttola, J. P. Hughes, J. Lessler, E. F. Lofgren, I. M. Longini, J.-P. Onnela, B. Özler, G. Seage, T. A. Smith, A. Vespignani, E. Vynnycky, M. Lipsitch, Simulations for Designing and Interpreting Intervention Trials in Infectious Diseases, BMC Med. In Press (2017), doi:10.1101/198051. 13. WHO, Ethical considerations for use of unregistered interventions for Ebola virus disease: Report of an advisory panel to WHO.WHO/HIS/KER/GHE/14.1 (2014) (available at http://www.who.int/csr/resources/publications/ebola/ethical-considerations/en/). 14. T. R. Fleming, S. S. Ellenberg, Evaluating interventions for Ebola : The need for randomized trials, 13, 6–9 (2016). 15. R. Upshur, J. Fuller, Randomized controlled trials in the West African Ebola virus outbreak, Clin. Trials 13, 10– 12 (2016). 16. National Academies of Sciences Engineering and Medicine, Integrating Clinical Research into Epidemic Response: The Ebola Experience (National Academies Press, Washington, DC, 2017). 17. M. E. Halloran, I. M. Longini Jr., C. J. Struchiner, I. M. Longini, C. J. Struchiner, Design and Analysis of Vaccine Studies (Springer, New York, 2010; http://www.springer.com/public+health/book/978-0-387-40313-7). 18. I. M. Longini, M. E. Halloran, M. Haber, R. T. Chen, Measuring vaccine efficacy from epidemics of acute infectious agents, Stat. Med. 12, 249–263 (1993). 19. M. K. B. Parmar, J. Carpenter, M. R. Sydes, More multiarm randomised trials of superiority are needed, Lancet 384, 283–284 (2014). 20. D. J. Reda, in Clinical Trials Design in Operative and Non Operative Invasive Procedures, (Springer, Cham, 2017), pp. 69–77. 14 21. R. J. Hayes, L. H. Moulton, Cluster Randomised Trials (CRC Press, Boca Raton, FL, 2nd Editio., 2017). 22. R. J. Hayes, N. D. Alexander, S. Bennett, S. N. Cousens, Design and analysis issues in cluster-randomized trials of interventions against infectious diseases., Stat. Methods Med. Res. 9, 95–116 (2000). 23. B. Giraudeau, P. Ravaud, Preventing bias in cluster randomised trials, PLoS Med. 6 (2009), doi:10.1371/journal.pmed.1000065. 24. A. M. Henao-Restrepo, A. Camacho, I. M. Longini, C. H. Watson, W. J. Edmunds, M. Egger, M. W. Carroll, N. E. Dean, I. Datta, M. Doumbia, B. Draguez, S. Duraffour, G. Enwere, R. Grais, S. Gunther, P.-S. Gsell, S. Hossmann, S. V. Watle, M. K. Konde, S. Keita, S. Kone, E. Kuisma, M. M. Levine, S. Mandal, T. Mauget, G. Norheim, X. Riveros, A. Soumah, S. Trelle, A. S. Vicari, J. A. Røttingen, M.-P. Kieny, Efficacy and effectiveness of an rVSV-vectored vaccine preventing Ebola virus disease: final results from the Guinea ring vaccination, openlabel, cluster-randomised trial (Ebola ça Suffit!), Lancet 389, 505–518 (2017). 25. S. E. Bellan, J. R. C. Pulliam, C. A. B. Pearson, D. Champredon, S. J. Fox, L. Skrip, A. P. Galvani, M. Gambhir, B. A. Lopman, T. C. Porco, L. A. Meyers, J. Dushoff, Statistical power and validity of Ebola vaccine trials in Sierra Leone: A simulation study of trial design and analysis, Lancet Infect. Dis. 15, 703–710 (2015). 26. J. Cohen, K. Kupferschmidt, Ebola vaccine trials raise ethical issues, Science (80-. ). 346, 289–290 (2014). 27. A. Doussau, C. Grady, Deciphering assumptions about stepped wedge designs: the case of Ebola vaccine research, J. Med. Ethics 42, 797–804 (2016). 28. M. G. Hudgens, M. E. Halloran, Toward Causal Inference With Interference, J. Am. Stat. Assoc. 103, 832–842 (2008). 29. J. P. T. Higgins, D. G. Altman, P. C. Gøtzsche, P. Jüni, D. Moher, A. D. Oxman, J. Savovic, K. F. Schulz, L. Weeks, J. A. C. Sterne, Cochrane Bias Methods Group, Cochrane Statistical Methods Group, The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials, Br. Med. J. 343, d5928 (2011). 30. World Health Organization, Guidance for Managing Ethical Issues in Infectious Disease Outbreaks (WHO Press, Geneva, Switzerland, 2016). 31. N. E. Dean, M. E. Halloran, I. M. Longini, Design of Vaccine Trials During Outbreaks With and Without a Delayed Vaccination Comparator, Ann. Appl. Stat. 12, 330–347 (2018). 32. A. M. Henao-Restrepo, I. M. Longini, M. Egger, N. E. Dean, W. J. Edmunds, A. Camacho, M. W. Carroll, M. Doumbia, B. Draguez, S. Duraffour, G. Enwere, R. Grais, S. Gunther, S. Hossmann, M. K. Konde, S. Kone, E. Kuisma, M. M. Levine, S. Mandal, G. Norheim, X. Riveros, A. Soumah, S. Trelle, A. S. Vicari, C. H. Watson, S. Keita, M.-P. P. Kieny, J. A. Røttingen, Efficacy and effectiveness of an rVSV-vectored vaccine expressing Ebola surface glycoprotein: interim results from the Guinea ring vaccination cluster-randomised trial, Lancet 386, 857–866 (2015). 33. M.-A. Widdowson, S. J. Schrag, R. J. Carter, W. Carr, J. Legardy-Williams, L. Gibson, D. R. Lisk, M. I. Jalloh, D. A. Bash-Taqi, S. A. S. Kargbo, A. Idriss, G. F. Deen, J. B. W. Russell, W. McDonald, A. P. Albert, M. Basket, A. Callis, V. M. Carter, K. R. C. Ogunsanya, J. Gee, R. Pinner, B. E. Mahon, S. T. Goldstein, J. F. Seward, M. Samai, A. Schuchat, Implementing an Ebola Vaccine Study - Sierra Leone, MMWR Suppl. 65, 98–106 (2016). 34. T. R. Fleming, Current issues in non-inferiority trials, Stat. Med. 27, 317–332 (2008). 35. P. B. Gilbert, D. Grove, E. Gabriel, Y. Huang, G. Gray, S. M. Hammer, S. P. Buchbinder, J. Kublin, L. Corey, S. G. Self, A Sequential Phase 2b Trial Design for Evaluating Vaccine Efficacy and Immune Correlates for Multiple HIV Vaccine Regimens, Stat. Commun. Infect. Dis. 3 (2011), doi:10.2202/1948-4690.1037. 15 36. A. D. Horne, P. A. Lachenbruch, K. L. Goldenthal, Intent-to-treat analysis and preventive vaccine efficacy, Vaccine 19, 319–326 (2001). 37. S. S. Ellenberg, T. R. Fleming, D. L. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective (John Wiley & Sons, New York, 2002). 38. P. C. O’Brien, T. R. Fleming, A multiple testing procedure for clinical trials., Biometrics 35, 549–556 (1979). 39. S. J. Pocock, Group Sequential Methods in the Design and Analysis of Clinical Trials, Biometrika 64, 191–199 (1977). 40. K. K. G. Lan, D. L. DeMets, Discrete Sequential Boundaries for Clinical Trials, Biometrika 70, 659–663 (1983). 41. P. S. Gsell, A. Camacho, A. J. Kucharski, C. H. Watson, A. Bagayoko, S. D. Nadlaou, N. E. Dean, A. Diallo, A. Diallo, D. A. Honora, M. Doumbia, G. Enwere, E. S. Higgs, T. Mauget, D. Mory, X. Riveros, F. T. Oumar, M. Fallah, A. Toure, A. S. Vicari, I. M. Longini, W. J. Edmunds, A. M. Henao-Restrepo, M. P. Kieny, S. Kéïta, Ring vaccination with rVSV-ZEBOV under expanded access in response to an outbreak of Ebola virus disease in Guinea, 2016: an operational and vaccine safety report, Lancet Infect. Dis. 17, 1276–1284 (2017). 42. WHO, Ebola situation reports: Democratic Republic of the Congo (2018) (available at http://www.who.int/ebola/situation-reports/drc-2018/en/). 43. A. Camacho, R. M. Eggo, S. Funk, C. H. Watson, A. J. Kucharski, W. J. Edmunds, Estimating the probability of demonstrating vaccine efficacy in the declining Ebola epidemic: a Bayesian modelling approach, BMJ Open 5, e009346 (2015). 44. W. A. Orenstein, R. H. R. Bernier, T. J. Dondero, A. R. Hinman, J. S. Marks, K. J. K. Bart, B. Sirotkin, Field evaluation of vaccine efficacy, Bull. World Health Organ. 63, 1055–1068 (1985). 45. M. L. Jackson, J. C. Nelson, The test-negative design for estimating influenza vaccine effectiveness, Vaccine 31, 2165–2168 (2013). 46. S. Piantadosi, Clinical Trials: A Methodologic Perspective: Second Edition (John Wiley & Sons, Hoboken, NJ, Second., 2005). 47. WHO Expert Committee on Biological Standardization, Human challenge trials for vaccine development: regulatory considerations (2017; http://www.who.int/biologicals/expert_committee/WHO_TRS_1004_web_Annex_10.pdf). 48. S. E. Bellan, R. M. Eggo, P.-S. Gsell, A. J. Kucharski, N. E. Dean, R. Donohue, M. Zook, F. Odhiambo, I. Longini, M. Brisson, B. E. Mahon, A. M. Henao-Restrepo, Guiding Vaccine Efficacy Trial Design During Public Health Emergencies: An interactive web-based decision support tool, BioRxiv (2018), doi:http://dx.doi.org/10.1101/252783. 49. J. Asher, C. Barker, G. Chen, D. Cummings, M. Chinazzi, S. Daniel-Wayman, M. Fischer, N. Ferguson, D. Follmann, M. E. Halloran, M. Johansson, K. Kugeler, J. Kwan, J. Lessler, I. M. Longini, S. Merler, A. Monaghan, A. P. Y. Piontti, A. Perkins, D. R. Prevots, R. Reiner, L. Rossi, I. Rodriguez-Barraquer, A. S. Siraj, K. Sun, A. Vespignani, Q. Zhang, Preliminary modeling results for Zika virus transmission in 2017, bioRxiv , 1–26 (2017). 50. B. Brende, J. Farrar, D. Gashumba, C. Moedas, T. Mundel, Y. Shiozaki, H. Vardhan, J. Wanka, J. A. Røttingen, CEPI—a new global R&D organisation for epidemic preparedness and response, Lancet 389, 233–235 (2017). 16 Acknowledgments: The authors are members of the WHO R&D Blueprint workplan for designing clinical trials in Public Health Emergencies, and have participated in a series of consultations. The authors sincerely thank WHO and other participants of the WHO Blueprint consultation of vaccine evaluation during Public Health Emergencies for their valuable contributions to this work: Steven E. Bellan, Nele Berthels, Alejandro Costa, Rich Donohue, Peter M. Dull, Rosalind M. Eggo, Susan S. Ellenberg, Thomas R. Fleming, Maria de Lourdes Garcia-Garcia, Julian P.T. Higgins, Peter W. Horby, Philip R. Krause, Adam J. Kucharski, Barbara E. Mahon, Frank O. Odhiambo, Marie-Pierre Preziosi, Peter G. Smith, Jonathan A.C. Sterne, Alessandro Vespignani, and Yazdan Yazdanpanah. Funding: NIH R37-AI032042 (NED, IML, MEH), NIH U54-GM111274 (NED, IML, MEH), NIH R01-AI139761 (NED, IML, MEH). NIHR grant PR-OD-1017-20002 (CHW), NIH NIAID R37-AI051164 (VDG), UK Medical Research Council MR/R015600/1 (CAD). NED, IML and PSG drafted the manuscript. All authors reviewed and edited the manuscript. Competing interests: The authors declare no conflict of interest. 17