Brain–machine interfaces for controlling lower-limb powered robotic systems

Studies were searched and screened in this review following the preferred reporting items for systematic reviews and meta-analyses (PRISMA), as shown in figure 1. Two authors (YH, DE) each conducted a literature search on 7 October 2017 within the PubMed (without using MeSH terms) and EMBASE databases using the advanced search method with a broad set of keywords: (Exoskeleton OR 'Powered exoskeleton' OR actuated OR orthosis OR robot * OR assistive) AND ('Lower limb' OR 'Lower extremity' OR 'Lower extremities' OR 'lower-body' OR walk * OR ambulation) AND (BCI OR 'Brain computer interface' OR BMI OR 'Brain machine interface' OR 'Brain-Controlled') AND (EEG OR electroencephalography OR fNIR * OR Functional Near-Infrared Spectroscopy OR 'Brain-controlled' OR 'motor imagery' OR 'intent'). Duplicates from the two databases and studies not within the inclusion criteria (see below) were excluded. Full texts of the remaining potentially relevant studies were obtained and screened. The two reviewers then compared results. In the case of a disagreement, a third reviewer (TPL) was decisive. Two additional publications that do not contain all the keywords in their titles were manually identified as relevant studies (Kilicarslan et al 2016, Lee et al 2016).

Zoom In Zoom Out Reset image size

The inclusion criteria by which the studies were included are as follow:

• Lower-limb powered exoskeleton or orthosisA variety of design forms exist within lower-limb, powered robotic systems. The terms orthosis and exoskeleton are often used interchangeably. In this review, the term exoskeleton refers to bipedal devices that actuate bilateral hip and knee joints (ankle joints are optional) to assist walking, regardless of whether it is over-ground or on a treadmill. A powered orthosis refers to a device providing actuated assistance to a single joint.
• BMIBoth closed-loop and offline BMIs were included. Although the former is the ultimate goal, offline decoding is often the first step towards an online version. Closed-loop BMIs were highlighted in the results.
• WalkingThis review specifically focuses on robots and BMIs that are directly related to walking. Some orthoses do not require an upright standing or walking posture (Xu et al 2014, Zhang et al 2015). Some BMIs do not control robotic devices to walk, although designed in walking context (Salazar-Varas et al 2015, Costa et al 2016a). They were excluded from this review.
• Neural signalsThis review does not exclude any studies based on the type of neural signal, regardless of recording levels (invasive and non-invasive) or the type of subjects (healthy and motor-impaired humans, monkeys).

Following data were extracted from each identified studies:

• UserHuman/monkey, patients/healthy subjects, and symptoms in the case of patients.
• Robotic deviceManufacturer, functions, shared control strategy, safety features, and speed
• BMITasks, neural features, artifact removal method, decoder, and output commands
• Performance

All identified studies and some of their key features were compiled in table 1. In the rest of this review, the phrase identified studies always refer to these 11 studies.

Table 1. The identified studies in this review.

First author (year)	Device	Subjects		BMI
		Impaired	Healthy	Tasks	Denoising	Neural features	Decoder
Do (2013)	Lokomat	1 SCI	1	Walk and stand	CAR, noisy channels were removedc	1–40 Hz ERD	Bayesian classifierc
Kilicarslan (2013)	Rex	1 SCI	None	Walk and stand	—	Delta band MRCP	Gaussian mixture modelc
He (2014)	NASA X1	1 stroke	2	Kinematics of lower limbs (continuous)	CAR	Delta band MRCP	Kalman filter
García-Cossio (2015)	Lokomat	3 stroke	10	Passive walk, active walk, and stand	CAR, CCA	8–30 Hz ERD	Logistic regression
Kwak (2015)	Rex	None	11	Walk, turn left/right, sit, and stand	60 Hz notch filter	SSVEP	K-nearest neighborsc
Donati (2016)	Lokomat and an exoskeleton	8 SCI	None	A few high level walk-related commands	CAR	MRCP with CSP	Linear discriminant analysisc
Kilicarslan (2016)	Rex	None	2	Walk and stand	${{H}^{\infty}}$ filterc, ASR	Delta band MRCP	Gaussian mixture modela,c
Lee (2016)	Rex (modified)	None	5	Walk versus turn; if turn: left versus right	CAR	14–19 Hz ERD	Random forestsc
López-Larraz (2016)	H2	4 SCI	3	Walk and stand	CAR, noisy trials were removed	7–25 Hz ERD and 0.1–1 Hz MRCP	Sparse discriminant analysisc
Vouga (2017)	EXiO	None	1 monkeyb	Tracking a cursor in 2D plane on screen	—	Neuronal firing rate	Linear regressionc
Zhang (2017)	Rex	1 SCI	1	Walk, stand, turn left/right	ASR	Delta band MRCP	Multiple kernel learning

^aA causal decoding method was implemented offline to simulate real-time conditions. ^bA rhesus macaque. ^cDenotes realtime operations. Abbreviations: SCI, spinal cord injury; CAR, common average reference; CCA, canonical correlation analysis; ASR, artifact subspace reduction; ERD, event-related desynchronization; MRCP, movement-related cortical potential; CSP, common spatial pattern; SSVEP, steady-state visual evoked potential.

The Quality Assessment Tool for Quantitative Studies, based on the Effective Public Health Practice Project (EPHPP) Guidelines were used to evaluate the methodological quality of each identified study (Jackson and Waters 2005). The following three categories were used: strong, moderate, and weak.

Except for one study that tested an exoskeleton for monkeys (Vouga et al 2017), all other studies tested human participants. Out of the 54 human subjects, there were 15 SCI (28%), 4 stroke (7%), and 35 healthy subjects (65%) (see figure 2(A)). For comparison, a recent systematic review of 27 studies of lower-limb powered exoskeletons published in 2015 with 144 disabled participants reported that 58% of the subjects were SCI, 35% were stroke, and 3% were subjects with unspecified etiology (Federici et al 2015). This user population also agrees with the current trending of the user eligibility of FDA-cleared lower-limb exoskeletons. All four FDA-cleared devices (ReWalk, Indego, Ekso, and HAL) are designed for SCI population, while Ekso also supports patients with stroke.

Zoom In Zoom Out Reset image size

Group analysis of possible differences between the patient and healthy subjects is not practical because of the low number of subjects and varying experimental protocols and outcome measures. Only two of the identified studies tested more than ten subjects, albeit healthy able-bodied subjects (García-Cossio et al 2015, Kwak et al 2015). Among the four studies that recruited both healthy subjects and patients: He et al (2014) and Zhang et al (2017) only had one patient each; García-Cossio et al (2015) and López-Larraz et al (2016) used different protocols in the two groups.

For the studies being analyzed, the two devices that appeared most frequently are Rex and Lokomat (Hocoma, Switzerland), shown in figure 3. Lokomat, being a stable device fixed on a platform, has an advantage over mobile exoskeletons in terms of stability. With little risk of falls, Lokomat provides a safe testing environment. The Rex exoskeleton keeps its balance without the support of crutches or any other external system, although in practice, human spotters are often required to guarantee safety (Kwak et al 2015, Kilicarslan et al 2016). The customized exoskeleton used in Donati et al (2016) is also reported to be stable in single support stance without the need of crutches as well. Similar to Lokomat, the machine in Vouga et al (2017) designed for a monkey is stationary on a treadmill. Two identified studies did not use the above devices with self-balancing features. To compensate, NASA X1 users in He et al (2014) and H2 users in López-Larraz et al (2016) used walkers.

Zoom In Zoom Out Reset image size

The time a robot needs to physically carry out each task is an important factor to evaluate the robot performance. Lee et al (2016) reported that Rex takes about 4 s to turn and 8 s to walk. This time is not reported in other studies. For full reviews of existing lower limb exoskeletons, readers are referred to Contreras-Vidal et al (2016), Esquenazi et al (2017) and He et al (2017).

Except for (Vouga et al 2017) recorded brain signals of a monkey with implanted microelectrodes, EEG was used in all other identified studies. The unanimous usage of EEG in all human studies may be the result of its non-invasiveness and portability.

Neural features are usually extracted from the raw input to increase signal-to-noise ratio and reduce computation complexity. The neural features used in the identified studies were categorized into five groups, as shown in figure 2(B).

Neuronal firing rate is a more direct measurement of neural activity than EEG. An increase in the rate at which a population of motor cortical neurons fires is correlated to movement in the preferred direction of the neuronal population (Georgopoulos et al 1986). It can only be accessed invasively, as in the only study with monkey (Vouga et al 2017). All other identified studies relied on EEG recordings.

Movement-Related Cortical Potential (MRCP): Studies on MRCPs date back decades ago when EEG was first observed, revealing time domain EEG features that are closely associated with movements and decision making (Shibasaki and Hallett 2006). Many MRCP components have been identified, such as P300 and N100. MRCP is the most commonly used neural feature among identified studies (Kilicarslan et al 2013, 2016, He et al 2014, Donati et al 2016, López-Larraz et al 2016, Zhang et al 2017). Although the time domain signals were usually sent to a decoder without explicitly specifying which components to analyze among these studies. Donati et al (2016) used common spatial pattern (CSP) to separate MRCPs into subcomponents.

Event-Related Desynchronization (ERD) is observed by a decrease in EEG power (usually alpha and beta bands) associated with movement-related tasks in both physical activity and motor imagery conditions (Pfurtscheller et al 1997). ERD of 7–25 Hz was used in combination with MRCP in López-Larraz et al (2016). In Do et al (2013), the power spectral densities (PSD) were integrated in 2 Hz bins that were centered at 1, 3, ..., 39 Hz, yielding 20 power spectral values per channel. In García-Cossio et al (2015), PSD from 8 to 30 Hz were computed using Welch's method with a Hanning window of 250 milliseconds. In Lee et al (2016), the averaged PSD between 14–19 Hz in a 0.5 s sliding window was computed.

Steady-State Visual Evoked Potentials (SSVEPs) are the brain's response to visual stimulus such as a flickering light. The frequency of the SSVEP response matches the frequency of the visual stimulus. SSVEPs are not directly associated with movement, yet subjects can be trained to make the connection. In Kwak et al (2015), five LED lights operated at 9, 11, 13, 15, 17 Hz were installed on an extended arm stand of the exoskeleton. The users were trained to stare at specific lights for the actions they represent. CCA was used for decomposing the EEG signal in order to extract stimulation frequency related information. CCA finds a pair of linear combinations such that the correlation between two canonical variables X (EEG) and Y (stimulus frequencies) is maximized.

In order to control the robots, the subjects need to engage in certain tasks to generate specific neural signal patterns. In offline studies, the tasks involve physically walking and standing (He et al 2014, García-Cossio et al 2015).

In most of the realtime studies, the users imagined discrete tasks such as walking/turning (Do et al 2013, Kilicarslan et al 2013, Lee et al 2016) or moving one leg (López-Larraz et al 2016). Donati et al (2016) reported a two-phase, long-term protocol: the patients were initially instructed to imagine movements of their hands and arms to control the exoskeleton. Seven months after onset of training, subjects imagined moving their left or right legs to control the stepping of the ipsilateral leg. Vouga et al (2017) designed a continuous virtual tracking task in realtime: a monkey was trained to control a dot on a screen to pursue a moving target. The position of the dot was then mapped to the ankle position of the exoskeleton.

Various decoders were found in the identified studies (figure 2(C)). Based on the continuity of states being decoded, they can be grouped into two categories: (1) Those featuring continuous reconstruction of trajectories, such as linear regression (Vouga et al 2017) and Kalman filter (He et al 2014), and (2) discrete classification, such as sparse discrimination analysis (López-Larraz et al 2016), Bayesian classifier (Do et al 2013), logistic regression (García-Cossio et al 2015), linear discriminate analysis (Donati et al 2016), Gaussian mixture model (Kilicarslan et al 2013, 2016), random forest (Lee et al 2016), multiple kernel learning (Zhang et al 2017), and maximizing canonical correlation coefficients (Kwak et al 2015).

In discrete classifications, the difficulty in achieving high performance generally increases significantly as the number of class increases. In an offline study, with three different tasks, only two of the tasks were being classified at a time in order to simplify the challenge (García-Cossio et al 2015). Cascaded classification was used in a few studies as a practical workaround to reduce the number of classes (Donati et al 2016, Lee et al 2016). For instance, users in Lee et al (2016) would make an initial, binary choice between walk versus turn; if turn was chosen, a follow up binary choice of direction (left versus right) would be made.

3.6.1. Levels of control.

3.6.2. Delay.

BMI adds another source of information to the control of the robot, creating a paradigm known as shared control, illustrated in figure 4. For an overview of the shared control strategies in exoskeletons, readers are referred to Tucker et al (2015). In this section, the review focuses on some unique strategies among the identified realtime BMI studies: the output from BMI decoders were sometimes rejected or delayed by the robot controller to avoid fast alternating or unsafe commands.

Zoom In Zoom Out Reset image size

In Do et al (2013), the BMI issues binary commands of walk and stop to the robot. A binary state machine was then modeled: the state of the machine changes only if the output exceeds a predefined threshold for at least 2 s. The robot response time is therefore at least 2 s. This delay rejects unnecessarily fast alternating commands during transition. In López-Larraz et al (2016), the exoskeleton only moves after detecting five consecutive move commands. Since the commands are issued every 0.25 s, this delays the robot response by at least 1.25 s. This statistic was not reported in other studies.

A manual switch was used in López-Larraz et al (2016) for safety reason. Every trial required the experimenter to explicitly press an activation button, otherwise any BMI commands would be ignored.

Lee et al (2016) is the only study that augmented the robot to automatically scan surrounding obstacles. A Kinect camera and ultrasonic sensors were installed. This customized Rex rejects any user commands that may lead to hitting obstacles. This is a typical example of shared-control: both the user and machine are engaged in the decision-making process for a better and safer control. Internal knowledge without the help of extra sensors can also be used to reject dangerous commands. For example, Kwak et al (2015) supports five actions: walk, turn left/right, stand, and sit. But the sit command can only be triggered from standing. Technically, this feature was implemented in their BMI decoder level instead of the robot control. As both the robots and BMIs become smarter and more deeply integrated, the line between their functions may blur.

Only limited information regarding BMI performance can be identified among these studies, possibly because many of them are proof-of-concept experiments (table 2). When different results were reported for impaired and healthy groups, only the result of the impaired group is presented. The two studies with continuous BMIs were excluded as their performances were measured in a manner that was different from the others: He et al (2014) reported an average correlation coefficient of 0.4 between the measured and predicted joint angles; Vouga et al (2017) defined performance as how much time the cursor spent in the target in increments of 0.6 s. However, only the result (>90%) from the best session was reported.

3.8.1. False positive, false negative, accuracy, and confusion matrix.

3.8.2. Information transfer rate.

Information transfer rate (ITR), expressed in bits per second, is a commonly used criterion to assess the overall performance of BMIs (Wolpaw et al 1998). It is expressed with the following formula,

$\begin{align} \newcommand{\e}{{\rm e}} \displaystyle {\rm ITR}=\frac{1}{T}\times \left[{{\log}_{2}}N+p{{\log}_{2}}p+\left(1-p \right){{\log}_{2}}\frac{1-p}{N-1} \right],\nonumber \end{align} \tag{ 1 }$

where T is the time in seconds needed to convey each command, p is the classification accuracy and N is the number of classes.

Note that equation (1) is only valid under some assumptions, including (a) all the output commands are equally likely to be selected; (b) the classification accuracy is the same for all the classes; (c) the classification error is equally distributed among all the remaining symbols (Yuan et al 2013). Meeting these assumptions can be challenging. For example, the cascaded decoder in Lee et al (2016) and state-based decoder in Kwak et al (2015) treat classes differently. Therefore, ITRs in table 2 should only be viewed as an approximation of the actual values.

Kwak et al (2015) reported ITR as one of their performance metrics and briefly mentioned the above limitation. Furthermore, ITR in López-Larraz et al (2016) was manually estimated by the authors of this review based on other information it reported. The average ITR over these two available records is 0.37 bits s⁻¹. This value agrees with the results found in other BMI studies. Typically, the ITR in BMI studies falls below 1 bit s⁻¹ with either human or monkey subjects, and either invasive or non-invasive recording Tehovnik et al (2013).

3.8.3. Control efficiency.

The methodological quality summary based on the EPHPP quality assessment tool is summarized in table 3. Details of the grading choices are attached in appendix.

4.1.1. Artifact removal.

4.1.2. Safety.

4.1.3. Responsiveness.

4.1.4. Practical challenges.

Users and prescribers of lower limb exoskeletons are concerned with the effectiveness of the devices in providing mobility in indoor and outdoor environments, their use in rehabilitation programs, their safety and possible health benefits, particularly with respect to pain, spasticity and autonomic activity, particularly bowel function. There have been many discussions over the clinical relevance of exoskeleton even without BMIs. Recent systematic reviews provide information on these topics. Some studies showed benefit of the exoskeleton over conventional therapy while other studies found no difference or that comparison therapies were superior.

A scoping review reported that clinical trials of powered robotic exoskeletons for post-stroke rehabilitation were free of serious adverse events and could bring meaningful improvement in sub-acute stroke (Louie and Eng 2016). A review of the usefulness of exoskeletons showed that ReWalk, Indego, and Mina (IHMC Robotics, USA) are effective for walking in a laboratory for individuals with complete lower-level SCI; however the level of scientific evidence of long term benefits is low (Lajeunesse et al 2016). One review concluded that exoskeletons are safe to use in real-world settings and known to yield health benefits (Miller et al 2016). However, this study was later challenged as it included duplicate subjects and studies, rendering its conclusions questionable (Dijkers et al 2016). An extensive review of the tests used in studying robotic aided movement of the lower extremities functions was published with the goal of improving exoskeleton design (Maggioni et al 2016). The protocols, outcomes, inclusion/exclusion criteria, and metrics in published clinical exoskeleton studies with individuals with SCI were found to vary greatly across studies (Contreras-Vidal et al 2016). Recently, a clinical study with 55 spinal cord injury patients proved significant benefits after they were rehabilitated with exoskeletons (Grasmücke et al 2017). On the other hand, there are reviews that found no consistent benefit to rehabilitation using an exoskeleton versus a variety of conventional rehabilitation methods (Federici et al 2015, Fisahn et al 2016).

Current clinical rehabilitation programs, such as treadmill body-weight support gait training, focus on interventions at the peripheral level of the body. The expectation from rehabilitation programs is that the repetitive motor training will trigger neuroplasticity that promotes the recovery of motor functions (Pinter et al 2016). By contrast, BMI approaches directly measures brain activity to control robotic devices, therefore encourage neural activities that drive the designated motor tasks. These approaches encourage voluntary control and motor intent from users, which have been demonstrated as one of the main principles in rehabilitation and may trigger stronger neuroplasticity (Koenig et al 2011, Luu et al 2017). Additionally, better understanding of neural representations of human movements from advanced BMI technology may propel the development of a novel training paradigm for improving the efficacy of rehabilitation in a top-down approach.

Donati et al (2016) is the only identified study that reported clinical improvement after protocols. However, its result is complicated by its mixed usage of treadmill-based robot, overground exoskeleton, virtual reality tasks, zero-G treadmill, BWS, and tactile feedbacks over 12 months.

It is worth noting that although SSVEP-based systems have the best performance in table 2, they may not provide additional benefits by exploiting cortex plasticity. SSVEPs are elicited by the task (looking at LEDs) that are not physiologically related to walking.

The BMI design can be either exogenous (Kwak et al 2015) or endogenous (Do et al 2013, Kilicarslan et al 2013, 2016, He et al 2014, García-Cossio et al 2015, Donati et al 2016, Lee et al 2016, López-Larraz et al 2016, Vouga et al 2017, Zhang et al 2017). The exogenous BMI, based on SSVEPs, have the advantage of providing a high accuracy even in close-loop control (93%, see table 2). In addition, the number of training sessions required by users to reach a successful performance is reduced. For instance, its subjects were instructed to fixate at LED lights with different blinking frequencies for 50 times, 5 s each time. This process is likely to be significantly shorter than typical BMI studies where multiple training sessions are necessary for optimal performance. There are even users that cannot get reasonably good performance despite enough training, a phenomenon known as BMI illiteracy (Vidaurre and Blankertz 2010). These aspects are important for improving users' confidence during the control of exoskeletons. However, one of the biggest limitations of exogenous BMIs is that they require users to find and fixate certain visual stimulus. Thus, this approach decreases user attention at the core task of walking, raising safety concerns, and may be unlikely to provide clinical benefit.

The reported performance among identified studies varies greatly (table 2). For instance, (Kilicarslan et al 2013) report 99% accuracy. However, this accuracy refers to the percentage of correctly classified EEG samples instead of trials, which does not directly translate to how well it performs in practice. In fact, there is little discussion in how to best report the performance. Accuracy do not reflect the number of classes and time allowed in deliberating each command. Additionally, decoding accuracy and ITR may strongly correlate to the skewness of the data. For example, a person chooses to walk straight and the decoder always predicts walk instead of turns. When the same number of trials in each class cannot be guaranteed, F₁ score along with other metrics is recommended because it considers both the precision and the recall ratio of a test, therefore robust to skewness in the data distribution. Above all, control efficiency closely measures the general performance of the system regardless of the technical details. The authors encourage future studies to report it to (1) allow objective comparison between studies; (2) provide an accurate sense of the usefulness of the system in practice.