www.nature.

com/npjdigitalmed

REVIEW ARTICLE OPEN

Evolving regulatory perspectives on digital health technologies

for medicinal product development
Seya Colloud1 ✉, Thomas Metcalfe1, Scott Askin2, Shibeshih Belachew3, Johannes Ammann1, Ernst Bos1, Timothy Kilchenmann1,
Paul Strijbos1, Damien Eggenspieler4, Laurent Servais 5,6, Chloé Garay7, Athanasios Konstantakopoulos7,10, Armin Ritzhaupt8,
Thorsten Vetter9, Claudia Vincenzi9,11 and Francesca Cerreta9,11

Digital health technology tools (DHTTs) present real opportunities for accelerating innovation, improving patient care, reducing
clinical trial duration and minimising risk in medicines development. This review is comprised of four case studies of DHTTs used
throughout the lifecycle of medicinal products, starting from their development. These cases illustrate how the regulatory
requirements of DHTTs used in medicines development are based on two European regulatory frameworks (medical device and the
medicinal product regulations) and highlight the need for increased collaboration between various stakeholders, including
regulators (medicines regulators and device bodies), pharmaceutical sponsors, manufacturers of devices and software, and
academia. As illustrated in the examples, the complexity of the interactions is further increased by unique challenges related to
DHTTs. These case studies are the main examples of DHTTs with a regulatory assessment thus far, providing an insight into the
applicable current regulatory approach; they were selected by a group of authors, including regulatory specialists from
1234567890():,;

pharmaceutical sponsors, technology experts, academic researchers and employees of the European Medicines Agency. For each
case study, the challenges faced by sponsors and proposed potential solutions are discussed, and the benefit of a structured
interaction among the different stakeholders is also highlighted.
npj Digital Medicine (2023)6:56 ; https://doi.org/10.1038/s41746-023-00790-2

INTRODUCTION EMA reviews whether the data generated by DHTTs are

Digital health technology tools (DHTTs) for use in conjunction with appropriate to support the benefit–risk assessment of medicinal
medicinal products are being developed to empower patients to products subject to the centralised procedure. This can take place
better manage their own treatment1 and have the potential to either during the assessment of the MAA of a medicinal product
transform clinical trials2–4. Thanks to DHTTs, clinical trial endpoints during which the DHTT-derived evidence has been submitted, or
can be measured in a home setting and may provide higher preferably earlier in the context of the DHTT development using
external validity and sensitivity in detecting medicinal products’ the EMA Qualification of Novel Methodologies (QoNM) platform6.
efficacy3. This, in turn, could lead to leaner clinical trials, reducing The EMA qualification process assesses whether data derived by
the burden on patients2 and bringing medicines to patients faster. a DHTT are acceptable to support a MAA (or MAA variation or
As DHTTs are being used to collect data and substantiate the extension) and whether the methodology allows for a valid and
safety and efficacy of medicinal products in clinical trials for clinically meaningful interpretation of the concept of interest in a
marketing authorisation applications (MAAs), medicines regulators reliable and robust manner. Applicants may request Qualification
(European Medicines Agency [EMA] and European Union [EU] Advice at any time during the development process to ensure
national competent authorities [NCAs]) must ensure that the appropriate method development7. An iterative qualification
clinical evidence generated is representative, robust and scienti- process is possible and desirable to allow refinement of validation
fically valid. DHTTs that also meet the definition of a medical plans as knowledge progresses. The qualification procedure can
device are subject to medical device regulatory oversight by the have various outcomes, outlined in Fig. 2. A satisfactory
relevant device body (NCAs with device competence and Notified qualification procedure leads to publication of a Qualification
Bodies [NBs]); both device bodies also play a role in the risk Opinion7. If the novel methodology cannot yet be qualified, the
classification of medical devices5. For this reason, DHTT develop-
sponsor will receive confidential Qualification Advice and pub-
ment in the EU takes place at the intersection of the medical
lication of a Letter of Support may be offered if preliminary data
device and the medicinal product regulatory frameworks, making
are considered promising. Publication of a Qualification Opinion
it a challenging environment for sponsors of DHTTs and medicine
attests the adequacy of a methodology for its context of use to
developers to navigate. A high-level summary of the different
regulatory authorities and their function is outlined in Fig. 1. The generate data for medicinal product benefit–risk assessment7.

1
F. Hoffmann-La Roche Ltd., Basel, Switzerland. 2Novartis Pharma AG, Basel, Switzerland. 3Biogen Digital Health International GmbH, Baar, Switzerland. 4SYSNAV, Vernon, France.
5
Muscular Dystrophy UK Oxford Neuromuscular Centre, Department of Paediatrics, University of Oxford, Oxford, UK. 6Division of Child Neurology, Centre de Références des
Maladies Neuromusculaires, Department of Paediatrics, University Hospital Liège and University of Liège, Liège, Belgium. 7Eli Lilly and Company Ltd., Basingstoke, UK. 8Bristol
Myers Squibb, Uxbridge, UK. 9European Medicines Agency, Amsterdam, The Netherlands. 10Present address: GE Healthcare S.A., Athens, Greece. 11These authors contributed
equally: Claudia Vincenzi, Francesca Cerreta. A. Konstantakopoulos was an employee of Eli Lilly and Company Ltd., Basingstoke, UK, during the conceptual design, drafting and
review of the paper. His current affiliation is GE Healthcare S.A., Athens, Greece. GE Healthcare S.A. did not contribute to this manuscript. A. Ritzhaupt was not an employee of
Bristol Myers Squibb, Uxbridge, UK, during the conceptual design, drafting and review of the manuscript. Bristol Myers Squibb did not contribute to this manuscript.
✉email: seya.colloud.sc1@roche.com

Published in partnership with Seoul National University Bundang Hospital

 S. Colloud et al.
2

Medicinal Product Approval Medical Device Certification Clinical Investigation Approval

EMA for centralised procedure,

CMDh for decentralised
NCAsa NBs NCAs
procedure and mutual recognition,
and NCAs for national authorisations

Function Function Function

• Medicinal product approval • Medical device expertise • Medicinal product and medical device expertise (for
• Novel endpoint and methodologies qualification (EMA (including medical device software) some NCAs it is within two separate entities)
only) • NB Opinion for the device part of an integral • Approval of clinical trials for medicinal products
• Approval of integral medicinal product–medical device drug–device combination • Approval of clinical investigations for medical devices
combination products regulated as medicinal products • Responsible for providing the CE marking • NB accreditation responsibility
• Accredited by NCAs

Applicable legislative framework Applicable legislative framework Applicable legislative framework

• Pharmaceutical legislation (Directive 2001/83/EC, • Medical device regulation (Regulation [EU] 2017/745) • Medical device regulation (Regulation [EU] 2017/745)
Regulation [EC] 726/2004) • Clinical trial regulation (Regulation [EU] 536/2014)
• In Vitro Diagnostic Medical Device Regulation • Clinical Trial Directive (Directive 2001/20/EC)
(Regulation [EU] 2017/746) • GCP Directive (Directive 2005/28/EC)

Legislation applicable to case studies Legislation applicable to case studies Legislation applicable to case studies

CS1 CS2b CS3c CS2d CS3e CS1 CS2 CS3 CS4

1234567890():,;

Case study key

Floodlight™ MS and
SV95C Diabetes pen Digital monitoring
Konectom™

CS1 CS2 CS3 CS4

Fig. 1 Summary of regulatory bodies, their function, and the legislative framework relevant to digital health technology tools. Summary
of the regulatory bodies responsible for medicinal product approval, medicine device certification and clinical investigation approval. Arrows
represent formal interaction between regulatory bodies. Listed are the relevant legislative frameworks for each regulatory body and indicated
below are the case studies where these frameworks are applicable. aFor Article 117, interaction with NBs occurs via the manufacturer. bFor the
clinical trial tool. cFor the injector pen. dFor the SaMD functionality in clinical practice. eFor the smartphone app. CE Conformitè Europëenne,
CMDh Co-ordination Group for Mutual Recognition and Decentralised Procedures–human, CS case study, EC European Council, EMA European
Medicines Agency, EU European Union, GCP Good Clinical Practice, MDSW medical device software, MS multiple sclerosis, NB Notified Body,
NCA National Competent Authority, SV95C Stride Velocity 95th Centile.

It is possible to obtain scientific advice on medicinal products medicines development and highlight the benefit of increased
both from NCAs and the Committee for Medicinal Products for collaboration between various stakeholders including device and
Human Use (CHMP); for medical devices, scientific advice can be medicine regulators and cross-industry collaboration. A group,
sought from NCAs with competency on medical devices. NCAs, including regulatory specialists from pharmaceutical sponsors,
with competency on medicinal products and medical devices, can technology experts, academic researchers and employees of the
also be invited to attend Innovation Task Force (ITF) meetings8, EMA, collected and analysed the cases. Some members of this
which provide a forum for early dialogue with applicants on group have previously published on the use of digital technolo-
innovative aspects in medicines development including emerging gies in medicines development10 and supported the development
therapies and technologies. To date, NBs are not allowed to give of the EMA’s Q&A11 on this topic.
pre-certification services, e.g., providing advice, before an
application is lodged by the manufacturer, and therefore these
services have to take place under the scope of the application. CASE STUDIES
In its ‘Regulatory Science Strategy to 2025’ (RSS), the EMA The four case studies presented include DHTTs at various stages of
envisages the creation of an integrated evaluation pathway for the medicines development across a variety of disease areas including
assessment of medical devices, in vitro diagnostics and borderline multiple sclerosis (MS), diabetes and cancer. Some of the DHTTs
products9. Among the goals of the integrated pathway assess- analysed may be classified as medical devices, depending on their
ment, three of them are relevant also for DHTTs: 1. to establish a intended purpose in clinical practice. Case studies 1–2 are
process for multi-stakeholder scientific advice to support devel- examples where sponsors are leveraging the QoNM for Medicines
opment of medicine–device combinations, qualification meth- Development to obtain regulatory endorsement of their DHTT
odologies and the use of companion diagnostics; 2. to create a methodologies, independent of a particular medicinal product.
process to consult medical device authorities and/or NBs (as Case study 3 is an example where the potential impact of DHTTs
applicable) for device-related aspects throughout the product on the benefit–risk assessment of the medicinal product may also
lifecycle, including post-authorisation safety-related events; 3. to be discussed as part of scientific advice and later assessed at the
adapt consultation processes to address digital technologies and time of MAA or post-approval. Case study 4 is an example where
wearables9. the DHTT may only be subject to medical device regulatory
The four case studies presented illustrate the complexity of the oversight in Europe. For each case, the industry authors have
different regulatory frameworks that span across DHTTs used in highlighted the core challenges faced in navigating the regulatory

npj Digital Medicine (2023) 56 Published in partnership with Seoul National University Bundang Hospital
 S. Colloud et al.
3
Public consultation

SAWP 3 Outcome 2:
CHMP 1, CHMP 2, CHMP 3,
discussion, Qualification Opinion
submission discussion adoption
continue
EMA validation, appointment of draft of draft of draft • Publicly available
towards
of coordinators, and QT Qualification Qualification Qualification
Qualification • Opinion on the acceptability
Opinion Opinion Opinion
Opinion of a specific use of the
proposed method in a
Day –60 Day –15 Day 0 Day 30 Day 60 Day 70 Day 90 Day 100 Day 130 Day 190 research and development
context, non-clinical or
Electronic clinical studies, based on
SAWP 3
submission QT meeting, the assessment of
DM with discussion, CHMP 1,
request Voluntary SAWP 1 submitted data
Start of applicant, continue adoption of
via IRIS, preparatory discussion, Draft report
procedure SAWP 2 towards Qualification
submission meeting LoI to
discussion Qualification Advice
of draft applicant
Advice
dossier

Outcome 1: Letter of Support

Qualification Advice
• Confidential • Publicly available (if sponsor agrees)
• Provides advice on • Based on Qualification Advice, when the
methods and protocols to novel methodology cannot yet be qualified
further development but preliminary data are promising
towards qualification • Aims to encourage data sharing and to
• Based on the evaluation facilitate further studies towards qualification
of preliminary data and
scientific rationale

Fig. 2 Summary of the EMA qualification procedure for digital technology-based methodologies to support approval of medicinal
products. Procedure summary for the qualification for digital technology-based methodologies to support the approval of medicinal
products, including the potential application outcomes. CHMP Committee for Medicinal Products for Human Use, DM discussion meeting, EMA
European Medicines Agency, LoI List of Issues, QT qualification team, SAWP scientific advice working party.

environment14,15. The device is passive, meaning it does not

Box 1. Data provided to support the qualification of the SV95C require patients to complete any tasks during their daily lives13,
measure and has been through extensive validation both in controlled and
uncontrolled environments with control and patient populations.
The SV95C application demonstrated how participants’ ambulation could be
reliably measured, and included the following evidence: Algorithms transform the data from the device into physical
variables, such as stride length and speed, and then compute
● The duration of recording (optimal 180 h, minimal 50 h) was calculated to
obtain a minimal variability and maximum sensitivity of the endpoint at
clinical variables, such as the SV95C, from the physical variables13.
baseline18 The EMA qualification validates that the SV95C is accurate, reliable,
● The external validity was demonstrated through a moderate correlation at sensitive to change and is relevant to patients. More specifically,
baseline with three measures that have been used as primary endpoints in the SV95C that quantifies a patient’s maximal ambulation velocity
previous pivotal trials, namely the Six-Minute Walking Test (6MWT)32, the
North Star Ambulatory Assessment33 and the Four-Stair Climb34. The 95th in a continuous manner in a home environment and reflects most
percentile was selected to ensure the best standardised response mean18 components (except for additional information like endurance, or
● There was a need to understand the factors that could potentially influence confounders like motivation or fatigue at time of assessment) of
the variability of the measure, such as patients’ compliance to wearing the
device during recording periods, duration of recording periods, and time of the well-established Six-Minute Walking Test (6MWT)13.
the day and days of the week during which the measure is performed. It
appeared that only the days of the week during which the measure is
recorded could influence it slightly because trial participants walk more
Regulatory aspects
slowly during the weekend, and a plan was thus proposed to mitigate the The ActiMyo wearable device, together with the SV95C digital
risk of influence18 endpoint, is a DHTT that spans across the medical device and the
● Definition of the minimal clinically important difference (MCID) based on a
population distribution, and an analogy with the 6MWT (for which MCID
medicinal product regulatory frameworks. The Medical Device
was defined the same way18) Directive (now updated to the Medical Device Regulation [MDR]
● A set of age- and sex-matched controls and a plan for longitudinal follow-up 2017/745), the Good Clinical Practice Directive (GCP; Directive
of controls18
● The feasibility of collection of such an endpoint in a global study, with data
2005/28/EC) and a selection of ISO standards were considered in
on trial participants’ compliance18 the development16,17. The hardware component of the wearable
sensors is a Conformitè Europëenne (CE)-marked Class I Medical
Device. It has been developed under the ISO13485 Quality
Management System. Through the Qualification Opinion, the
frameworks and potential solutions have been explored, with the EMA endorsed SV95C as a secondary endpoint for ambulant
aim of further supporting the development and use of DHTTs. Duchenne muscular dystrophy (DMD) trial participants aged over
5 years when measured with a valid and suitable wearable
device18. The Qualification Opinion is not specific to ActiMyo, but
CASE STUDY 1—QUALIFICATION OF A DIGITAL ENDPOINT FOR hardware and software performance are included in the Opinion.
MEASUREMENT OF REAL-WORLD AMBULATION IN DUCHENNE This approach is common practice in EMA qualifications, as they
MUSCULAR DYSTROPHY (STRIDE VELOCITY 95TH CENTILE)
aim at qualifying the approach or endpoint and not to endorse a
What is the technology/how is it used? specific device.
Stride Velocity 95th Centile (SV95C) is a digital clinical outcome The application to the EMA was prepared by a multidisciplinary
assessment (COA) and is the first digital endpoint, collected by a group of experts (paediatric neurologists, biostatisticians, phy-
wearable device, to have received endorsement by the EMA siotherapists, engineers, regulatory affairs specialists and patient
through publication of a Qualification Opinion12,13. A wearable representatives). A Letter of Intent for request of qualification was
medical device based on magneto-inertial technology (ActiMyo®) submitted in June 2017 and the final Qualification Opinion was
was used to measure patient movements in a non-controlled adopted by the CHMP in April 201918.

Published in partnership with Seoul National University Bundang Hospital npj Digital Medicine (2023) 56
 S. Colloud et al.
4
The qualification application was based on several global experts earlier in the development and qualification process could
natural history and pivotal trials18 (summarised in Box 1). This was support the transition of DHTTs used as clinical trial endpoints to
the first qualification procedure conducted by the EMA for a the collection of real-world evidence in clinical care settings post-
digital endpoint, so there were limited precedents to inform the approval. The evidentiary package and device status would need
specific requirements. The continuous dialogue throughout the to be tailored accordingly to accommodate the real-world
process with the EMA was extremely helpful to come to an clinical use.
understanding of the evidentiary requirements and data inter-
pretation. The absence of an approved medicinal product in DMD
made the qualification of SV95C as a primary endpoint difficult CASE STUDY 2—MULTIDIMENSIONAL DIGITAL ENDPOINTS TO
because longitudinal data including demonstration of sensitivity ASSESS NEUROLOGICAL FUNCTION VIA SMARTPHONE
to treatment effect would be needed. In this case, it was accepted SENSOR-BASED TECHNOLOGY IN A REAL-WORLD
that sensitivity to change could be illustrated by response to ENVIRONMENT IN MULTIPLE SCLEROSIS
steroids, which are routinely used18. What is the technology/how is it used?
Two biotechnology/pharmaceutical companies aiming at improv-
Challenges ing measures of MS disease progression have engaged into a pre-
A follow-on Qualification Opinion application has been submitted competitive agreement seeking to increase the chance of
to the EMA to upgrade the use of SV95C as a primary efficacy (regulatory) acceptance of digital endpoints derived from two
endpoint and generalise the application of SV95C to other different smartphone-based applications (apps), which share
neuromuscular diseases characterised by a proximal muscle common concepts. Their objective is to characterise and quantify
weakness leading to progressive difficulties in ambulation. disability based on multiple active tests (e.g., Cognitive Test
Notably, for the application to be generalisable, the clinical [information processing speed], Draw a Shape and Pinching Tests,
meaningfulness of SV95C for any other indications should be Two-Minute Walk Test and U-Turn Test) and passive monitoring of
adequately established. With traditional assessments used in functional performances in neurological domains of cognition,
clinical practice, relevant concepts of interests are being measured upper extremity function, gait, balance and overall mobility.
across multiple indications. As an example, the 6MWT is used to Floodlight™ MS consists of several software components: includ-
measure distance walked in 6 min in DMD and also in MS, ing a smartphone app and five CE-marked medical device
Parkinson’s disease and chronic obstructive pulmonary dis- software (MDSW) components intended to provide an objective
ease19–21. It is important to determine a clinically meaningful measurement of the function of people living with MS in between
threshold for the measure to be qualified in different contexts of clinical visits22,23. Konectom™ is also a smartphone-based, CE-
use or indications. This threshold might differ across indications, marked MDSW with nine assessments, intended to be used as a
but the concept of distance walked and how it relates to performance-based and patient-reported outcome assessment
ambulation has relevance across those conditions. Many digital tool to quantify neurological impairments (motor and cognitive
COAs, such as SV95C, currently used in clinical trials are not functions) in people living with MS24.
available in clinical practice. Leveraging these endpoints as The Floodlight MS and Konectom apps can be used as data
primary endpoints in confirmatory clinical trials, even if they have collection tools to characterise treatment effects within clinical
been qualified from a regulatory standpoint, could be a challenge trials and as patient management tools in clinical practice to
for sponsors since recognition as a standard of care measure by a inform patient care. A parallel development approach could
broad range of therapeutic area experts is essential to translate enable common disease measurements, hence, to be used not
results in clinical practice. Clinical experience and consensus on only during development of the medicinal product but also during
measurement is essential to support the contextualisation of patients’ treatment; it could generate better quality real-world
clinical trial data and medicine labelling information. datasets and potentially provide earlier treatment access for
patients.
Solutions
From a regulatory standpoint, a solution to bridge the evidence Regulatory aspects
requirements of SV95C for use in other conditions would be to rely The Floodlight MS and Konectom apps are subject to the MDRs in
on the analytical and technical validation of SV95C (e.g., Europe, currently classified as Class IIa MDSWs. In the context of
addressing parameters such as accuracy, precision, selectivity, using these apps in medicinal product clinical trials, they are
sensitivity, reproducibility and stability) whilst evaluating the subject to GCP and particularly Computer Software Validation
clinical validity, meaningfulness and utility of the endpoint in the (CSV)25. Digital endpoints derived from these smartphone apps
new condition. Historically, however, assessments such as the could be subject to the qualification procedure. To explore this
6MWT have been adopted empirically without formally qualifying possibility, an ITF meeting was held with the EMA to initiate
their validity nor defining their context of use. Similarly, if discussions on Floodlight MS and further Qualification Advice
wearable devices capable of capturing SV95C would be available could follow for Floodlight MS/Konectom-derived digital
to practitioners, the generalisability of SV95C could be additionally endpoints.
supported by real-world use and experience in conditions where
rapid limb movement is a relevant concept of interest.
To support the broad recognition and use of digital COAs, Challenges
industry co-authors would welcome the opportunity for a clinical Digital endpoint development is complex and includes several
network of therapeutic area experts, device experts and patients important steps leading to a validated disease measurement score
to be consulted together at an early stage. To date, the (Fig. 3). The EMA qualification procedure mainly focuses on the
involvement of NBs during the qualification process is in part first and last step in this process and proposes to address
limited by the MDR, which prevents NBs from being consulted. A technicalities of algorithm development by providing information
more collaborative approach would be beneficial in preparation of regarding CE-marked hardware specifications in the Qualification
more complex products at the interface of medicines and medical Opinion11. This approach does not examine how the science
devices. Whilst DHTTs used in clinical practice are not in the scope behind feature derivation would be addressed (e.g., software
of the QoNM and hence are outside the EMA remit, industry requirements, including pre-processing steps enabling raw data
authors believe that involving device bodies and other clinical denoising, normalisation and segmentation) and which

npj Digital Medicine (2023) 56 Published in partnership with Seoul National University Bundang Hospital
 S. Colloud et al.
5
Development of a Validated Score for Use as a Digital Endpoint

Step One Step Two Step Three

Define a common conceptual model of Define the derived features well and select reliable Use these well-defined features to derive a score,
the disease ones, e.g., requires an understanding of the key which is in concordance with existing
Define the concepts of interest that are parameters influencing the data capture method gold-standard measures
patient relevant as well as the mathematical method to derive
the measure
Assign the features derived from the digital tests to
one of these concepts In a pre-competitive collaboration model, all parties
(e.g., celerity represents walking speed) should agree on these two first steps

Fig. 3 Summary of steps involved in the development of a validated digital endpoint. Three steps to develop a validated score for use as a
digital endpoint.

environmental factors may influence the results, which is a critical NB because the company had an active open contract with the
part of digital endpoint development. When assessing new NB for other CE-marked products; this is not always the case for
medicinal products, the EMA needs reassurance that the pharmaceutical sponsors starting their device development. NB
algorithms used to derive endpoint results, and their subsequent participation brought expertise on how to approach components
modifications, perform as intended and are in line with the of software validation whilst NCAs informed on particularities of
published Qualification Opinion to collect and interpret the data. clinical investigation. One of the questions raised by the sponsor
At the time of MAA, the EMA requires that the company provides was how to claim equivalence of previous versions of the
the relevant confirmation of software equivalence with the software generating the endpoint to newer versions. In this
software used during the qualification from NBs or medical discussion, the advice of medical device regulators (NBs in
device NCAs (which only the developer of the method could
particular) was very beneficial, considering the degree of
access) or adequate bridging data to demonstrate comparability
algorithm changes and suggesting leveraging the Medical
(which would be very difficult to generate without the software
code). For analogue endpoints, the input data are reproducible in Devices Directive (MEDDEV) 2.7/1 guidance and MDR Annex
most cases and bridging for comparability is also achievable. In XIV 3 on equivalence26,27. The medical device regulators from
the case of a digital endpoint looking at the concept of celerity, for NCAs on the other hand, were able to advise whether the version
example (Fig. 3), to derive the feature ‘celerity’ captured from a of the software included in clinical trials qualified in that context
test consisting of walking 2 min in a straight line, a sponsor would of use as a medical device, which was essential information for
need to know several mathematical parameters used to calibrate the set-up of the study.
the test, to normalise the data avoiding noise and understand The QoNM has been a very helpful tool to support harmonisa-
what parameters drive variability in order to ensure the test tion of methods in medicinal product development. It has fostered
performed is equivalent to the originator. To then demonstrate it the use of common methodologies for disease measurement,
is equivalent, several patients would need to be tested in a gait lab including endpoints used in clinical trials. Once a method is
showing that the same input data (walking in a straight line for disclosed, in most instances, it can be used with limited reliance
2 min) provide the same output (e.g., the measure of celerity) on the original developer of the method. With digital endpoints,
across the two apps. Without the input and output data of the demonstration of comparability of DHTTs is difficult and limits the
originator or the code behind the measure, this equivalent result is ability of sponsors to use DHTTs without the collaboration of the
very difficult to achieve. Confirming equivalence of two different technology developer who owns the intellectual property of the
apps to derive endpoints is therefore challenging without having code. In addition, the level of information needed to bridge one
access to the software or input and output datasets or the software-processing method to another is too complex to be
software code. This leads to applicants relying on the developer of included in a document, so it is now done within the Qualification
the DHTT in order to use qualified digital endpoints in their trials.
Opinion for traditional analogue endpoints. This static approach
Patients, healthcare professionals (HCPs), sponsors and regulators
would also fail to address fast-evolving technology. The industry
would benefit from having a harmonised set of easily accessible
digital endpoints to drive adoption and support an easier authors would see a repository of qualified software codes and
contextualisation of treatment outcomes in the future, with fewer associated methods to derive digital endpoints as a possible
parallel development and no duplication of efforts. solution to guarantee the equivalence of endpoints across
Pre-competitive frameworks can support progression and registrational studies. Having an independent third-party organi-
consensus on digital measures; however, they are challenged by sation governing such a repository could facilitate that qualifica-
proprietary considerations and the ease with which technologies tion standards are met before submission of the request for a
could be appropriated if the details on input requirements and Qualification Opinion. Furthermore, by establishing a licensing
systems design were to be disclosed. model, industry and academia partners would be incentivised to
publish their software on this platform for use by various sponsors,
Solutions attesting to a level of quality.
For Floodlight MS, the sponsor held early interactions with the The industry authors are of the opinion that experts from device
EMA ITF. Representatives of NCAs overseeing clinical investiga- bodies should be involved in the qualification procedure to advise
tions for medical devices participated in this discussion, along sponsors on the validity of pre-processing steps and quality
with two sponsor-invited NB representatives who attended in assessment procedures for feature processing (step 2 in Fig. 3).
the role of observers to enable an informal exchange. So far, this Subsequently, the same device experts could advise other
meeting is the only example of interaction concomitantly sponsors on the equivalence of their technology to the qualified
involving the sponsor, EMA, NCAs and NBs. As per MDRs, NBs one. Such an approach would facilitate harmonisation of
were limited in their ability to consult, hence setting up the measurements, comparability of generated datasets, and could
meeting took several months. It was only possible to engage the help drive ubiquity of use for digital endpoints.

Published in partnership with Seoul National University Bundang Hospital npj Digital Medicine (2023) 56
 S. Colloud et al.
6

Companion App
Regulatory Oversight

• EMA
Provides scientific recommendations on
Insulin Pen Regulatory Oversight the benefit–risk assessment of the
medicinal product forming an integral
product with the autoinjector
Pre-filled syringe
• EMA
Authorises as a medicinal product (Medical Device Regulation Article 117 for medicine– Medical Device Coordination
device combinations) (Regulation [EU] 2017/745) Group
• NB Guidance on Qualification and
Provides results of the conformity assessment of the device (with the relevant general Classification of Software in Regulation
safety and performance requirements set out in Annex I) to the Medical Device Regulator (EU) 2017/745 – MDR and Regulation
• NCA (EU) 2017/746 – IVDR
If interventional drug or device clinical trials were to be conducted with the tool

• NB
Reusable injector pen CE marking, if the intended purposes
• EMA and device classification require it. A
Authorises insulin cartridge as a medicinal product smartphone app for use with a
connected autoinjector pen may require
• NB
the app to be certified as Class IIa
Authorises the reusable injector pen
or IIb MDSW
• NCA
If interventional drug or device clinical trials were to be conducted with the tool

Fig. 4 Regulatory considerations for an insulin pen-connected medical device. The roles and responsibilities of regulatory bodies in the
assessment of medical devices used in combination with a medicinal product. CE Conformitè Europëenne, EMA European Medicines Agency,
EU European Union, IVDR In Vitro Diagnostic Devices Regulation, MDR Medical Device Regulation, MDSW medical device software, NB Notified
Body, NCA National Competent Authority.

CASE STUDY 3—DIGITAL DIABETES MANAGEMENT SYSTEMS: injector pen to be used with an insulin cartridge is authorised
CONNECTED INSULIN PEN separately as a medical device16. Depending on the intended use,
What is the technology/how is it used? an app compatible for use with a connected autoinjector pen may
Several connected care systems are being developed to support have some functionalities that require the app to be certified as
treatment and management of diabetes28. Here we present a Class IIa or IIb MDSW. Additionally, the add-on device to a pre-
connected care system comprised of an insulin pen that connects filled pen or reusable pen may need to be certified as a medical
to a smartphone app to automatically track insulin doses injected device. The conformity assessment procedure for a MDSW is
dependent on the risk class and may require the involvement of a
by patients.
NB29.
Many patients struggle to manage their diabetes effectively
because of the complexities associated with the treatment. Not
only does insulin therapy require many steps and decisions, but Challenges
also depends on manual recording of glucose and insulin data, The introduction of ‘connected’ devices raises challenges for
which is burdensome and has poor patient compliance28. sponsors on the nature and level of data necessary to support the
Automating the recording of blood sugar, insulin dose measure- MAA of the associated medicinal product. The evidence in relation
ment and time of injection could not only ease the burden of to MDR compliance further varies depending on the classification
manual recording, but may improve the accuracy of patient data. of the DHTTs and whether integral, co-packed or supplied
A more reliable and complete dataset has the potential to improve separately. The roles and responsibilities for the assessment of
treatment management and outcome, which in turn could such medicinal product–medical device combinations (Fig. 4) can
positively impact diabetes self-management. also fall in a grey area.
The EMA’s responsibility is to provide a scientific opinion on the
benefit–risk assessment of the medicinal product of which the
Regulatory aspects
autoinjector pen is an integral part. With the implementation of
A medical device used in combination with a medicinal product the MDR, and specifically with MDR Article 117, in the review of
may have an impact on the benefit–risk of the product, for the ‘connected system’, there may be some areas of overlap
example it may decrease (or increase, if malfunctioning) the risk of between the scope of the medical device regulatory framework
medication errors. Therefore, the available regulatory pathway for (i.e., conformity with relevant General Safety and Performance
a digitally based diabetes management system differs depending Requirements) and the medicinal product benefit–risk assessment.
on the type of drug–device combination. The connected pen The safety and performance of the medical device part could
might be provided as a pre-filled or a reusable injector pen, and affect the benefit–risk of the medicinal product. The scope of the
the connected component may be integral to the pen or an add- EMA review will consider the impact of the digital health
on device. These are important considerations, as the pre-filled application on the benefit–risk assessment of the medicinal
pen containing insulin is authorised as a medicinal product (MDR product (e.g., accuracy of dose administration recording), as well
Article 117 for Medicine–Device Combinations Regulation [EU] as the approach taken by the applicant to evaluate and manage
2017/745); however, it will require a NB Opinion for the approval its impact, for instance, in the case of medication errors. Sponsors
of the MAA (MDR Regulation [EU] 2017/745), whilst the reusable can seek CHMP input on data needed to support the MAA through

npj Digital Medicine (2023) 56 Published in partnership with Seoul National University Bundang Hospital
 S. Colloud et al.
7
the scientific advice procedure, which is well established7. evaluate the equivalence of evidence generated in one indication
However, the major challenge that sponsors face is navigating to another, or across medicinal products, a critical understanding
two regulatory frameworks simultaneously, which may have areas of the medicinal product’s efficacy and safety features as well as
of overlap. To facilitate the approvability of these DHTTs, it is specific disease area expertise are required. NBs responsible for
beneficial to have a clear understanding of the type of data the CE certification do have clinical experts, but they might not be
required, clinical and other, to support any claims made in the familiar with the details of a medicinal product’s safety, efficacy
respective medicinal product information and/or medical device and class effects. For products regulated as medical devices, such
instructions for use. as DPM modules, it is generally not possible to engage formally
with the EMA, unless there is scope for a claim in the medicinal
Solutions product labelling. Hence, a challenge sponsors of medical devices
face, within the current regulatory framework, is the complexity of
To overcome the current challenges and streamline the qualifica-
involving the NBs, and potentially the EMA, in consultation on
tion and approvals of such DHTTs, sponsors would benefit from
such queries.
concurrent and aligned joint scientific advice between the In addition, several DPM modules developed by manufacturers
medicinal product regulators and the medical device bodies. This are being brought to market claiming comparable functionality
is in line with the development of an integrated evaluation without being required to demonstrate comparability in treatment
pathway for the assessment of medical devices and medicines, as outcomes. Requiring evidence to demonstrate comparability of
mentioned in the EMA RSS to 20259; its creation would facilitate treatment outcomes across tools would be disproportionate for a
timely alignment on advice and assessment of the different DHTTs medical device presenting a low risk to patients. Manufacturers of
components (i.e., medicinal product, medical devices and app). DPM modules aim to demonstrate that their DPM modules are
safe to use, and the technical performance is adequate for their
CASE STUDY 4—DIGITAL MONITORING OF SYMPTOMS IN intended use. However, evidence of comparable functionality
PATIENTS WITH CANCER could better inform patients’ and physicians’ preferences for
evidence-based disease management solutions.
What is the technology/how is it used?
Web-based digital patient monitoring (DPM) software modules are
Solutions
in development to capture symptoms reported by patients with
cancer. This concept is an evolution of more traditional Although DPM module regulation falls outside the EMA’s remit,
(telephone-based) remote patient monitoring. DPM is a tool to the industry authors would welcome the opportunity of a joint
collect patient-reported outcomes in clinical practice and enable discussion bringing together knowledge and expertise available
review by clinicians in real time. Modules are the patient-facing across the NBs and the EMA. Overarching guidance on the
element of the software, which encompass a symptom ques- evidence required to extend the claim of the DPM module from its
tionnaire and in some cases algorithms for processing of the functionality on one medicinal product across a class of medicinal
symptom questionnaire input and educational materials. When products, treatment lines or disease indications, would be
software modules are linked to a clinic’s electronic medical beneficial to sponsors. Additionally, with an increase in similar
records or used with a standalone platform, the combination is DPM modules being brought to market and claiming comparable
described as a DPM solution. The patient-facing area of the DPM functionality, it would be beneficial to developers if NBs defined
module generally focuses on a particular cancer and may contain the general specifications needed to demonstrate equivalence in
or capture medicine-specific information. The aim of this achieving comparable treatment outcome for this type of DHTT.
technology is to improve patient care and healthcare resource
utilisation by better managing the disease and medicinal DISCUSSION
treatment, through capturing patient-reported disease or
treatment-related symptoms and quality-of-life-reported data, This review aims to illustrate challenges of regulating digital
and facilitating seamless non-urgent communication between health technologies that often sit at the intersection of two
patients and HCPs. European regulatory frameworks (medical device and the
medicinal product regulations).
As shown in the four case studies (CSs 1–4), summarised in
Regulatory aspects Table 1, DHTTs have the potential to benefit medicines develop-
DPM modules can improve compliance by enhancing the ment and patient care; however, the complexity of the European
detection of adverse events directly reported by the patient, regulatory framework and current unavailability of a pathway that
which may indirectly lead to improving the safety profile of a allows for formal joint or parallel advice from medicine regulators
medicinal product as well as to improved clinical decision-making. and device bodies, may result in a slower uptake and develop-
In Europe, they are generally considered Class IIa MDSW apps ment of DHTTs.
(depending on the actual intended use) and are subject to In 2020, the European Commission adopted the Pharmaceutical
conformity assessment (MDR)16. As modules have different Strategy for Europe, which provides opportunities to adapt
intended applications and classifications, it may be necessary to legislation to be future proof30. It highlights important aspects
evaluate each component of the DPM software to determine its of supporting the development of medical devices for use in
medical device classification. When it is claimed that the DPM medicine development, such as the need for more collaboration
module has an impact on the benefit–risk of its associated within the regulatory networks. The revision of the legislation
medicinal product, it is important to be able to distinguish the coming with the Pharmaceutical Strategy for Europe is an
contribution to the benefit–risk of each individual component. opportunity to address some of the challenges presented by
DHTTs highlighted in the case studies detailed in this paper.
Challenges Potential solutions, as put forward by the authors, are
discussed below.
DPM modules can inform clinical decision-making related to one
medicinal product within a particular cancer type. The evidence
required to extend the claim of the DPM module from its Increased collaboration between regulatory bodies
functionality on one medicinal product across a class of medicinal The opportunity for enhanced collaboration between the various
products, treatment lines or disease indications, is not clear. To stakeholders within the regulatory system is evident in all the four

Published in partnership with Seoul National University Bundang Hospital npj Digital Medicine (2023) 56
 S. Colloud et al.
8
Table 1. Summary of case studies.

Case study Type of MDSW Regulatory pathway Proposed context of use of the Disease area
DHTT (yes/no) Qualification Opinion

1. Stride Wearable sensors for passive No QoNM for Medicines Development Use as a secondary endpoint in Duchenne
Velocity 95th measurement of disease Device used to support qualification pivotal medicinal product studies, in muscular
Centile phenotype is a CE-marked hardware (ActiMyo®) ambulant Duchenne muscular dystrophy
dystrophy patients aged 5 years and
above, when measured by a valid
and suitable wearable device
2. Smartphone apps for active Floodlight QoNM for Medicines Development Digital measures that detect subtle MS
Floodlight™ and passive measurement of MS tests The Floodlight MS application changes in individual MS disease
MS/ disease phenotype Yes presents assessments that are CE- course trajectories and enable the
Konectom™ marked MDSW for use in clinical development of more sensitive,
practice. Various configurations of responsive and patient-relevant
the Floodlight MS suite of tests are endpoints to complement or replace
used in clinical trials within data existing measures of disease
collection DHTTs. DHTTs used in progression in people living with MS
medicinal product clinical studies to
support data collection only do not
require a CE mark. They should be
GCP compliant
Konectom Konectom is similarly a medical
Yes device app intended to be used as a
performance-based outcome and
patient-reported outcome
assessment tool to quantify
neurological impairments (motor
and cognitive functions) in people
living with MS. Various
configurations of the Konectom
suite of tests are also used in clinical
trials within data collection DHTTs
3. A connected care system Yesa Medicinal product (forming an Insulin pen that connects to a Diabetes
Connected comprised of an insulin pen integral product) or medical device if smartphone app allowing
insulin pen smart component and a the pen and/or the smart automated tracking of doses
in diabetes smartphone app for passive component is reusable injected by patients
dose monitoring CE-marked MDSW for the digital Smartphone apps that digitally track
component on the connected app insulin doses already exist on the
market as MDSW for use in clinical
practice. Some might also automate
other manual processes, such as
tracking blood glucose through a
connection to a blood glucose
monitor and calculation of the bolus
4. Digital A web-based digital patient Yes CE-marked MDSW Indication- or medicine-specific Cancer
patient monitoring software capturing modules to improve patient care
monitoring patient-reported symptoms and healthcare resource utilisation
for cancer by enabling cancer patients and
treatment healthcare professionals to better
manage the disease
CE Conformitè Europëenne, DHTT digital health technology tool, GCP Good Clinical Practice, MDSW medical device software, MS multiple sclerosis, QoNM
Qualification of Novel Methodology.
a
Dependent on the intended use.

cases. Whilst the EMA scientific advice procedure and the EMA advice was very beneficial. This interaction is not a standard
QoNM are well established for medicinal products, there is no interaction, and the introduction of the MDR specifying that the
equivalent procedure for getting advice for the medical device NBs have no ability to consult has made these opportunities for
components used with medicinal products. The opportunity for interaction even more complex.
structured joint feedback from medicine regulators and device The joint advice of the EMA, NBs and device regulators would
bodies on DHTT development would be welcomed by medicinal foster and create efficiencies in the development of DHTTs, and a
product sponsors and technology developers. number of use cases have been presented. For the development
Opportunities to create direct interaction between the EMA and of DHTTs with potential future use in a clinical practice, joint
technical experts from NBs are not foreseen under the current EU advice would be helpful to support the transition from clinical trial
legislation. The Floodlight MS case study (CS2) is an example of an endpoints to clinical practice (CS1). Guidance on the demonstra-
ad-hoc joint interaction (EMA, NCAs, NBs), where the NBs were tion of software equivalence would be beneficial for bridging and
able to advise on what evidence would be needed to claim claiming comparable performance to measure a qualified end-
equivalence of software versions to derive a digital measure; this point (CS2). The input of software validation experts could be

npj Digital Medicine (2023) 56 Published in partnership with Seoul National University Bundang Hospital
 S. Colloud et al.
9
valuable when data generated using a DHTT are submitted as gained on DHTTs, the industry authors would welcome EMA
confirmatory evidence of clinical studies to support a medicine’s guidance on what evidence is necessary to support patient
benefit–risk assessment (CS1 and CS2). In the future, DHTTs relevance of digital endpoints from one to another context of use
intended to be used in conjunction with a medicinal product will and what evidence remains valid across contexts of use. Such
become more complex and may have a significant impact on the guidance would provide important insights and foster the
benefit–risk of medicinal products (CS3 and CS4). In such cases, all development of DHTTs in multiple conditions. Improving the
stakeholders would benefit from an integrated pathway assess- generalisability of digital measures and tools across various
ment of the DHTTs throughout the lifecycle of the product, indications is important to keep pace with the increasing rate of
encompassing the best clinical and technology experts in the EU. innovation in healthcare.
In recognition of the changing European legislation and fast-
evolving landscape of medicinal products and medical devices,
including DHTTs, the EMA has proposed an integrated evaluation CONCLUSION
pathway for the assessment of medicines used in combination The case studies presented outline the different regulatory
with medical devices in their RSS 2025. In the future, as foreseen frameworks that span across DHTTs used in medicines develop-
by the MDR, expert panels on medical devices will also provide ment and highlight the need for increased collaboration as well as
scientific advice to manufacturers on their clinical development potential evolution of the current regulatory frameworks to foster
strategy and proposals for clinical investigations for certain high- this. Enhancing collaboration between stakeholders and engaging
risk medical devices, namely Class III devices and Class IIb active the breadth of expertise available in the EU through multi-
devices intended to administer and/or remove medicinal pro- stakeholder assessments have the potential to expedite the
duct(s). Whilst limited to certain devices and not covering the development and qualification of novel DHTTs for use in medicinal
broad range of DHTTs, this proposal will foster collaboration product development. The use of DHTTs is expected to continue
between parties as multiple parallel assessments remain a to increase in the future; harmonisation and increased collabora-
challenge for developers aiming to guarantee timely access to tion across the different regulatory bodies and between sponsors,
novel therapeutics or methodologies such as DHTTs in Europe. technology developers, HCPs and patients will be critical to ensure
that medicinal product development can benefit from the fast-
More collaboration between companies enabled through paced advances of technologies. A formal strategy to improve the
multi-stakeholder platforms for sponsors integration of the assessment of medicines and medical devices
Sponsors developing DHTTs aiming at capturing endpoints for when needed, would be welcomed by developers for predict-
medicines development would benefit from collaboration plat- ability, consistency and efficiency. The upcoming Pharmaceutical
forms supporting pre-competitive efforts and digital endpoint Strategy for Europe is an important and timely opportunity for
implementation in clinical trials. The qualification process provides Europe and supports the need to address digital transformation in
an opportunity for a high degree of alignment in digital the EU, including how to address the implementation of
measurements within diseases and could facilitate a faster uptake ‘personalised medicine’ and highlighting the importance of
of DHTTs overall. A hindrance to harmonisation and deployment collaboration and regulatory expertise sharing between medicine
in trials is the (understandable) reluctance of developers to and device regulatory bodies30. All the key decision-makers in the
divulge proprietary software codes. One solution could be that the healthcare ecosystem are being brought together to create
qualification provides high-level information on algorithm use and tangible actions and new solutions for our healthcare systems.
its references for eventual use by other sponsors. Alternatively, The EU has the building blocks to solve these challenges and seize
these algorithms could be made available in a sponsor-agnostic the opportunities offered by innovative healthcare solutions to
data space handled by an independent third party, which could create better and sustainable healthcare for generations to come.
provide an interesting solution. Algorithms, software codes or
derivation methods from which qualified endpoints are derived
could be shared and licensed to sponsors for use in studies. Such a DISCLAIMER
space could also support pre-competitive collaboration and drive The views expressed in this article are the personal views of the
convergence in digital measurements. Pre-competitive collabora- author(s) and may not be understood or quoted as being made on
tions would enable alignment of the digital features considered behalf of or reflecting the position of the European Medicines
and consensus on their computational principles when develop- Agency or one of its committees or working parties.
ing a score for specific indications. This would enable equivalence
of digital measures and harmonise practices between developers
of DHTTs. Initial proposals for multi-stakeholder platforms exist, DATA AVAILABILITY
such as one led by the Digital Medicine Society with their digital The findings and data supporting this Review are derived from the authors’ insights
endpoint library31. Ultimately, digital endpoints used to support and references cited within the paper.
authorisation of medicinal products should be interpretable and
comparable across treatments and be open for use by as many Received: 10 March 2022; Accepted: 5 March 2023;
sponsors as possible.

Generalisability of digital measures and tools across various

indications
Qualification of digital methodologies increases the transparency REFERENCES
on the requirements needed to support the use of high-quality 1. Affinito, L., Fontanella, A., Montano, N. & Brucato, A. How physicians can
digital solutions for the benefit of patients and HCPs. This, in turn, empower patients with digital tools. J. Public Health 30, 897–909 (2022).
2. Inan, O. T. et al. Digitizing clinical trials. NPJ Digit. Med. 3, 101 (2020).
enables device developers to adhere to quality principles and
3. Sverdlov, O., van Dam, J., Hannesdottir, K. & Thornton-Wells, T. Digital ther-
specifications required to reliably reproduce a clinical outcome apeutics: an integral component of digital innovation in drug development. Clin.
generated with an equivalent tool. Pharmacol. Ther. 104, 72–80 (2018).
A common question faced by sponsors is how to bridge the 4. Stephenson, D. et al. Precompetitive consensus building to facilitate the use of
evidentiary requirements to support the qualification of an already digital health technologies to support Parkinson disease drug development
qualified DHTT for use in other indications. With more experience through regulatory science. Digit. Biomark. 4, 28–49 (2020).

Published in partnership with Seoul National University Bundang Hospital npj Digital Medicine (2023) 56
 S. Colloud et al.
10
5. European Medicines Agency. Questions & answers for applicants, marketing 29. Medical Device Coordination Group. Guidance on qualification and classification
authorisation holders of medicinal products and Notified Bodies with respect to of software in regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746 –
the implementation of the medical devices and in vitro diagnostic medical IVDR. https://ec.europa.eu/health/sites/health/files/md_topics-interest/docs/
devices regulations ([EU] 2017/745 and [EU] 2017/746). EMA https:// md_mdcg_2019_11_guidance_en.pdf (2019).
www.ema.europa.eu/en/documents/regulatory-procedural-guideline/questions- 30. European Commission. Pharmaceutical strategy for Europe. 2020. https://
answers-implementation-medical-devices-vitro-diagnostic-medical-devices- health.ec.europa.eu/medicinal-products/pharmaceutical-strategy-europe_en.
regulations-eu/745-eu-2017/746_en.pdf (2021). (2021).
6. European Medicines Agency. Qualification of novel methodologies for medicine 31. Digital Medicine Society (DiMe). DiMe’s library of digital endpoints. https://
development 2021. EMA https://www.ema.europa.eu/en/human-regulatory/ www.dimesociety.org/get-involved/library-of-digital-endpoints/ (2022).
research-development/scientific-advice-protocol-assistance/qualification-novel- 32. Butland, R. J., Pang, J., Gross, E. R., Woodcock, A. A. & Geddes, D. M. Two-, six-, and
methodologies-medicine-development-0 (2021). 12-minute walking tests in respiratory disease. Br. Med. J. (Clin. Res. Ed.) 284,
7. European Medicines Agency. Qualification of novel methodologies for drug 1607–1608 (1982).
development: Guidance to applicants. EMA https://www.ema.europa.eu/en/ 33. Muscular Dystrophy UK. North Star Ambulatory Assessment (NSAA). https://
documents/regulatory-procedural-guideline/qualification-novel-methodologies- www.musculardystrophyuk.org/wp-content/uploads/2017/06/
drug-development-guidance-applicants_en.pdf (2014). NSAA_Only_ManualVersion-2.0_May-2017.pdf (2017).
8. European Medicines Agency. EMA/484400/2014 mandate of the EMA Inno- 34. Goemans, N. et al. Prognostic factors for changes in the timed 4-stair climb in
vation Task Force (ITF). EMA https://www.ema.europa.eu/en/documents/ patients with Duchenne muscular dystrophy, and implications for measuring
other/mandate-european-medicines-agency-innovation-task-force-itf_en.pdf drug efficacy: a multi-institutional collaboration. PLoS One 15, e0232870 (2020).
(2014).
9. European Medicines Agency. EMA regulatory science to 2025 strategic reflection.
EMA https://www.ema.europa.eu/en/documents/regulatory-procedural- ACKNOWLEDGEMENTS
guideline/ema-regulatory-science-2025-strategic-reflection_en.pdf (2020). The authors would like to thank Héloïse Mignot (EMA) for her review and feedback of
10. Cerreta, F. et al. Digital technologies for medicines: shaping a framework for publication drafts, and Solange Corriol Rohou (AstraZeneca), Fabian Model
success. Nat. Rev. Drug Discov. 19, 573–574 (2020).
(previously an employee of F. Hoffmann-La Roche Ltd.) and Lada Leyens (F.
11. European Medicines Agency. Questions and answers: Qualification of digital
Hoffmann-La Roche Ltd.) for their insights and contribution to the development of
technology-based methodologies to support approval of medicinal products.
this publication. The authors disclosed receipt of the following financial support for
EMA https://www.ema.europa.eu/en/documents/other/questions-answers-
the research, authorship and/or publication of this article: F. Hoffmann-La Roche Ltd.,
qualification-digital-technology-based-methodologies-support-approval-
Switzerland, provided financial support for publication of this manuscript and the
medicinal_en.pdf (2020).
European Medicines Agency provided financial support for the open access fees of
12. Haberkamp, M. et al. European regulators’ views on a wearable-derived perfor-
this manuscript. F. Hoffmann-La Roche Ltd., Switzerland; Novartis, Switzerland;
mance measurement of ambulation for Duchenne muscular dystrophy regulatory
trials. Neuromuscul. Disord. 29, 514–516 (2019). Biogen, Switzerland; and Eli Lilly and Company Ltd., UK, provided financial support for
13. Servais, L. et al. Stride Velocity 95th Centile: Insights into gaining regulatory their respective studies. Writing and editorial assistance for this manuscript was
qualification of the first wearable-derived digital endpoint for use in Duchenne provided by Nikki Black, PhD, from Articulate Science Ltd., UK, and funded by F.
muscular dystrophy trials. J. Neuromuscul. Dis. 9, 335–346 (2022). Hoffmann-La Roche Ltd.
14. Le Moing, A.-G. et al. A movement monitor based on magneto-inertial sensors for
non-ambulant patients with Duchenne muscular dystrophy: A pilot study in
controlled environment. PLoS One 11, e0156696 (2016). AUTHOR CONTRIBUTIONS
15. Italfarmaco. Clinical study to evaluate the efficacy and safety of givinostat in All authors meet the four criteria for authorship in line with the International
ambulant patients with Duchenne muscular dystrophy (NCT02851797). Clinical- Committee of Medical Journal Editors Recommendations. All authors reviewed,
Trials.gov https://ClinicalTrials.gov/show/NCT02851797 (2016). commented on and approved the manuscript. All authors had full access to the data
16. The European Parliament and the council of the European Union. Regulation (EU) and responsibility for the decision to submit for publication. Case study 1—
2017/745. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/? Qualification of a digital endpoint for measurement of real-world ambulation in
uri=CELEX:32017R0745 (2017). Duchenne muscular dystrophy (Stride Velocity 95th Centile) was developed by D.E.,
17. European Commission. Commission Directive 2005/28/EC. http://data.europa.eu/ L.S., and P.S. Case study 2—Multidimensional digital endpoints to assess neurological
eli/dir/2005/28/oj (2005). function via smartphone sensor-based technology in a real-world environment in
18. European Medicines Agency. Qualification opinion on Stride Velocity 95th Centile multiple sclerosis was developed by S.C. and S.B. Case study 3—Digital diabetes
as a secondary endpoint in Duchenne muscular dystrophy measured by a valid management systems: connected insulin pen was developed by C.G. and A.K. Case
and suitable wearable device. EMA https://www.ema.europa.eu/en/documents/
study 4—Digital monitoring of symptoms in patients with cancer was developed
scientific-guideline/qualification-opinion-stride-velocity-95th-centile-secondary-
by J.A.
endpoint-duchenne-muscular-dystrophy_en.pdf (2019).
19. Canning, C. G., Ada, L., Johnson, J. J. & McWhirter, S. Walking capacity in mild to
moderate Parkinson’s disease. Arch. Phys. Med. Rehabil. 87, 371–375 (2006).
20. Savci, S. et al. Six-minute walk distance as a measure of functional exercise
COMPETING INTERESTS
capacity in multiple sclerosis. Disabil. Rehabil. 27, 1365–1371 (2005). S.C. is an employee of, and owns shares and stocks in, F. Hoffmann-La Roche Ltd.,
21. Karpman, C., DePew, Z. S., LeBrasseur, N. K., Novotny, P. J. & Benzo, R. P. Deter- Switzerland. T.M. is an employee of, and owns shares in, F. Hoffmann-La Roche Ltd.,
minants of gait speed in COPD. Chest 146, 104–110 (2014). Switzerland. S.A. is an employee of Novartis Pharma AG, Basel, Switzerland. S.B. is an
22. F. Hoffmann-La Roche Ltd. Floodlight™ MS. https://floodlightms.com (2021). employee of Biogen Digital Health, Biogen International GmbH, Baar, Switzerland. J.A.
23. Midaglia, L. et al. Adherence and satisfaction of smartphone- and smartwatch- is an employee of, and holds shares and stocks in, F. Hoffmann-La Roche Ltd.,
based remote active testing and passive monitoring in people with multiple Switzerland. E.B. is an employee of F. Hoffmann-La Roche Ltd., Switzerland. T.K. is an
sclerosis: Nonrandomized interventional feasibility study. J. Med. Internet Res. 21, employee of F. Hoffmann-La Roche Ltd., Switzerland. P.S. is an employee of, and
e14863 (2019). owns shares in, F. Hoffmann-La Roche Ltd., Switzerland. D.E. is an employee of
24. Guo, C. et al. Konectom™ smartphone-based digital outcome assessment of SYSNAV, Vernon, France. L.S. is an employee of the University of Oxford, UK, and
cognitive and motor function in multiple sclerosis. MSVirtual 2020 – Poster holds a patent without financial return on a method to assess upper limb function
Abstracts. Mult. Scler. 26, 168 (2020). Abstract P0102. using a wearable device; he has consulted for Biogen, F. Hoffmann-La Roche Ltd.,
25. European Medicines Agency. Guideline on computerised systems and electronic Novartis, Scholar Rock, Pfizer, BioHaven Therapeutics, Sarepta, Santhera, Biophytis,
data in clinical trials. EMA https://www.ema.europa.eu/en/documents/regulatory- Affinia and Regenexbio; he chairs the DSMB of Lupin therapeutics and FibroGen; he is
procedural-guideline/draft-guideline-computerised-systems-electronic-data- also the secretary of the World Muscle Society and the Vice Chair of the scientific
clinical-trials_en.pdf (2021). board of Spinal Muscular Atrophy (SMA) Europe. C.G. is an employee of, and owns
26. European Commission. MEDDEV 2.7/1. https://ec.europa.eu/docsroom/ shares in, Eli Lilly and Company Ltd., Basingstoke, UK; and is a member of The
documents/17522/attachments/1/translations/en/renditions/native (2016). European Federation of Pharmaceutical Industries and Associations as an Eli Lilly and
27. European Commission. Medical Device Regulation Annex XIV 3. https:// Company Ltd. representative. A.K. was an employee of, and owned shares in, Eli Lilly
www.medical-device-regulation.eu/2019/08/14/annex-xiv/ (2019). and Company Ltd., Basingstoke, UK, during the conceptual design, drafting and
28. Levine, B. J., Close, K. L. & Gabbay, R. A. Reviewing U.S. connected diabetes care:
review of the manuscript. A.R. is an employee of Bristol Myers Squibb, UK. T.V. is an
the newest member of the team. Diabetes Technol. Ther. 22, 1–9 (2019).
employee of the European Medicines Agency, Amsterdam, The Netherlands. C.V. is an

npj Digital Medicine (2023) 56 Published in partnership with Seoul National University Bundang Hospital
 S. Colloud et al.
11
employee of the European Medicines Agency, Amsterdam, The Netherlands. F.C. is an Open Access This article is licensed under a Creative Commons
employee of the European Medicines Agency, Amsterdam, The Netherlands. Attribution 4.0 International License, which permits use, sharing,
adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative
Commons license, and indicate if changes were made. The images or other third party
ADDITIONAL INFORMATION material in this article are included in the article’s Creative Commons license, unless
Correspondence and requests for materials should be addressed to Seya Colloud. indicated otherwise in a credit line to the material. If material is not included in the
article’s Creative Commons license and your intended use is not permitted by statutory
Reprints and permission information is available at http://www.nature.com/ regulation or exceeds the permitted use, you will need to obtain permission directly
reprints from the copyright holder. To view a copy of this license, visit http://
creativecommons.org/licenses/by/4.0/.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims
in published maps and institutional affiliations.
© The Author(s) 2023

Published in partnership with Seoul National University Bundang Hospital npj Digital Medicine (2023) 56