Sensor-Based Locomotion Data Mining for Supporting the Diagnosis of Neurodegenerative Disorders: A Survey

Low-level motion indicator assessment: The evaluation of different gait features, including simple and complex gait parameters, is commonly considered an important part of dementia assessment, and their locomotion patterns are considered good predictors of diverse adverse health outcomes and mobility “bio-markers” [114]. Different studies have shown that by sensor-based gait cycle analysis of PwD vs cognitively healthy seniors, it is possible to find a correlation between variability in different gait parameters and cognitive decline progression [12, 74]. As an example, the experiments conducted by Kearns et al. [57] revealed that there is a correlation between higher tortuosity of the path with lower MMSE scores, which is assessed by clinicians.

Task-oriented assessment: Task-oriented methods are based on single-task and dual-task (sometimes called Talking While,Walking [122]) walking assessment, which consider the concurrent performance of a motor-motor or motor-cognitive task. The evaluation relies on measuring the interference of the different tasks on each other. Indeed, while the individuals divide their attention to multiple tasks, they have difficulty in regulating the stride-to-stride variations of locomotion, and this difficulty is stronger for cognitively impaired people [27, 112]. Impairment in executive functions (e.g., verbal fluency [71], visual-spatial skills [47], counting aloud backward from 100 to zero [71]) is one of the earliest indicators of cognitive decline. As an example, different experiments have proved that PwD have a more variable within-bout and irregular trunk acceleration pattern, higher step duration and gait complexity, and less variable across-bout walking pattern on single- and dual-task walking (e.g., words enumeration) compared to healthy persons [47, 53, 92]. The execution and evaluation of task-oriented assessment methods is more complex than the one of low-level motion indicator methods but allows the recognition of early-stage cognitive decline.

Pattern-based methods: These methods process trajectory data to recognize abnormal patterns that may indicate dementia according to specific indicators, such as the Martino-Saltzman model [72]. Since ambulation episodes are composed of locomotion and nonlocomotion phases, a key step is to segment locomotion traces to recognize significant trajectories. A wrong segmentation of trajectories may disrupt the effectiveness of the pattern recognition algorithms. Moreover, the execution of daily living activities, especially in indoor environments, may impact the individual’s movements, increasing the complexity of trajectories and incorrect detection [59]. Some pattern-based methods are nonsupervised. For example, in the work of Khodabandehloo and Riboni [59], the authors detect loops (i.e., pacing and lapping episodes) building a buffer on the trajectory segments and considering the percentage of intersection among the buffered segments composing the same trajectory. Other works represent the trajectory as a string of locations, and detect abnormal patterns using the longest repeated substring algorithm [65]. Other authors propose the use of supervised ML algorithms to recognize abnormal trajectories. Zolfaghari et al. [128] represent trajectories as colored pictures to encode features such as speed and sharp points, and they use a multilayer deep neural network for determining whether the trajectory is by PwD, by a person with mild cognitive impairment, or by a cognitively healthy senior. Since pattern-based methods rely on data acquired in naturalistic environments, they are prone to misclassification errors due to noise in sensor data acquisition and interference of external factors (e.g., activities, interaction with other people, obstacles in the home). Hence, these approaches tend to be effective mostly for long-term monitoring [59]. Other methods include the application of low-pass filters to location data for noise reduction especially for data collected from inertial sensors, or the addition of extra features such as statistics extracted from trajectory data [59, 65], activities, and abnormal behaviors [60]. Furthermore, Chaudhary et al. [16] tried to implement an early dementia detection system in an indoor environment using travel patterns of the inhabitant and a recurrent neural network, which automatically extracts the high-level features.

Threshold-based methods: This category of methods detects gait phases by mining thresholds from the data for segmenting events, recognizing gait cycles, and locomotion phases. Meng et al. [75] used accelerometer data and adaptive thresholds to recognize the gait phases of PwD and cognitively healthy seniors. In the work of Vuong et al. [119] the authors presented a solution for detecting locomotion anomalies in dementia patients using a wearable Opal monitor that includes an accelerometer, a gyroscope, and a magnetometer. To recognize direct, pacing, lapping, and random movement patterns, they considered translational acceleration threshold to detect a walk event, then they tracked the movement orientation until the algorithm would detect that the subject has made a complete stop. The classification is done by considering a different range of orientation degrees for different locomotion patterns. Wang et al. [121] proposed a threshold-based method to automatize the TUGT. Event segmentation is based on thresholds to identify peaks and valleys during forward motion, swing points, and stance point. Experimental results showed that PwD need more time for TUGT tasks such as sitting and standing up compared to cognitively healthy subjects. Of course, in such methods, a key factor is the threshold definition. Since different subjects may walk with high variation in cadence and step length, those works need adaptive thresholds for different classes of subjects and specific model calibration for the different individuals. An advantage of threshold-based approaches is the low computational complexity and feasibility for real-time applications at the edge.

Feature-based methods: As explained before, dementia is a complex impairment that impacts several domains. Hence, it is natural to consider different features for its detection. Feature-based methods rely on feature extraction techniques to represent the different characteristics of locomotion data, and on supervised ML algorithms to classify the feature vectors according to the cognitive status of the subject. In feature-based methods, locomotion features used for recognizing dementia include kinematic characteristics of postural control such as displacement on the transverse plane and dispersion radius [20], accelerometer-based gait parameters (e.g., walking speed, step frequency, compensation movements, acceleration variance) [41], gait symmetry and regularity [122], and other gait variables including step counts, step duration, speed, and accelerometer and gyroscope statistical measures [46].

Toosizadeh et al. [112] extracted different features from accelerometers worn by patients on the upper arm, on the wrist, and on the shins, during the execution of dual tasking. Those features considered both upper-extremity function parameters (e.g., elbow angular velocity, acceleration) and gait parameters (e.g., speed, time interval, distance). Analysis of variance tests showed significant correlation between upper-extremity features acquired during the dual task and the MoCA cognitive assessment result of the patient. Kumar et al. [64] used trend analysis on different navigational features, including speed, path efficiency, angle turn, and ambulation fraction, to distinguish PwD from cognitively healthy seniors considering UWB location data acquired in an assisted living facility. Together with feature extraction, some of these methods apply feature selection techniques to increase recognition rates, reduce computational effort, and avoid overfitting [41, 46, 122].

Kondragunta et al. [63] used data acquired from depth cameras during both single and dual tasking, and extracted several features including step length, step time, stride time, and cadence, by applying 3D human pose estimation techniques. They proposed using dynamic time warping on those features to recognize PwD, individuals with mild cognitive impairment, and cognitively healthy seniors. Compared to wearable sensor systems, techniques based on cameras are less obtrusive, but they are prone to detection errors in low-light conditions and are generally more computationally expensive. Moreover, the use of cameras pose serious privacy issues, especially when they are deployed in private homes [9]. A shortcoming of feature-based methods is that the accuracy of recognition strongly depends on feature engineering. Moreover, being based on supervised learning, those methods have a need for large volumes of training data that are expensive to capture in real-world settings.

Feature-based assessment methods: The diagnosis of PD can be challenging in its earlier stages, and Parkinsonian gait is characterized by small steps, a slower gait cycle, smaller swing phase, and lengthier stance phase. Physicians evaluate these features in their diagnosis process to confirm the presence of PD. Gait evaluation can be difficult since it can be influenced by several aspects, such as age and health condition. Despite the considerable interest in Parkinsonian gait analysis, there is no accurate tool to help physicians with gait evaluation. To detect the characteristics of gait, feature extraction methods and ML have been used. However, gait is a physiological characteristic that differs for each person according to age, health, and other intrinsic factors. Therefore, manual preprocessing and feature extraction will always be limited in their capacity. Generally, the feature-based assessment method is used to extract significant features that will help diagnose PD. Various features such as step distance, stride length, stride velocity, stance and swing phases, heel and normalized heel forces, swing time of Parkinsonian patients, and walking speed were extracted from the data collected from wearable sensors and after that examined using an ML-based algorithm to select the most influential features that would help differentiate between the two groups: subjects with PD and healthy people. Generally, supervised learning methods such as Linear Discriminant Analysis (LDA) [84], SVM [14, 22, 54, 84], the naive Bayes classifier [14, 22, 115], RF [22, 115], Multilayer Perceptron (MLP) [115], Decision Tree (DT) [14], K-nearest neighbor [115], and Deep 1D-Convnet [29] are used for distinguishing between PD patients and HCSs. Furthermore, recently novel ML and deep learning based multimodal technologies have been used to identify severity in PD patients’ actions by analyzing speech, movement patterns, and evaluation of motor capabilities [117, 120]. The rationale of this approach is that they may effectively capture discriminative features from time-series data or other IMU sensors collected data without the need of sophisticated feature engineering efforts.

Image processing based assessment methods: Image processing technologies have been employed widely for PD diagnosis. PD patients show large gait variability and slower walking speeds than normal people. Cho et al. [18] introduced a gait analysis system based on a computer vision based technique for the detection of gait patterns of PD. They captured a few videos of both normal subjects and patients with PD. They processed the images from the videos to characterize the subjects. Principal component analysis and LDA were employed to extract features, and the minimum distance classifier was used as the classifier. Seven PwPD and seven healthy people from Buddhist Tzu Chi General Hospital in Taiwan participated in this study. It used the image sequences of human silhouettes during walking and extracted the intrinsic features by LDA. It can identify healthy people and PD patients by their gaits with high reliability, and it is considered a promising aid in the diagnosis of PD. Khan et al. [58] also presented a computer vision based marker-free method to detect abnormal gait in PwPD. In this study, subjects are videotaped with several gait cycles for the gait analysis of PwPD. To develop a silhouette, the subject’s body is segmented through a color segmentation process. A skeleton is formed by computing the medial points of each body segment. Motion cues such as the cyclic motion of legs and the posture lean of the subject during the gait are extracted from the skeleton. Then the comparison study was compared between these two cues with the cues of the probable perfect gait pattern to assess gait impairment. Generally, a computer vision based technique is introduced for gait analysis to classify normal or PD gaits using a camera. In the work of Gong et al. [44], a masked regional convolutional neural network is applied for extracting human silhouettes from video frames based on recorded videos of normal gaits. After that, the gait energy images are constructed based on extracted human silhouettes as features, which are applied to develop an SVM model for classifying healthy and PD gaits in video clips.

Pattern-based assessment methods: These approaches process trajectory data to recognize abnormal patterns identifying PD. Walking is a part of human movement monitored by the human brain. Gait deficits and abnormal walking patterns may appear if the brain fails to control this movement. Walking patterns can be monitored continuously and remotely over time by wearable sensors. The bag-of-words approach [96] was introduced where each individual’s walking time series is defined as a bag of words. The ratio of the words together with ML techniques such as linear SVM, DT, RF, and K-nearest neighbor were employed to distinguish between patients with PD and healthy individuals. Although the bag-of-words method is used to distinguish between PD and healthy age-matched individuals, this method is also applicable to other health conditions. Each individual’s walking time series is converted into signal subsequences using an overlapping sliding window. Then, each subsequence is characterized using a few statistical descriptors and similar subsequences are assigned the same word. Therefore, each person’s data is converted into a bag of words and then evaluated using accelerometers’ data collected from the ankles. The proposed approach illustrates a vital step toward providing healthcare through continuous monitoring and advanced analysis of movement patterns [96]. Another method was introduced to diagnose PD based on a shifted 1D Local Binary Pattern (LBP) using an ML algorithm [30]. The gaitpdb dataset was used for the evaluation task, consisting of three gait datasets based on gait signals from different circumstances. Statistical features were extracted from histograms of gait signals changed by shifted 1D-LBP. ML methods such as naive Bayes, logistic regression, MLP, were employed to extract and classify features that can be successfully used in PD detection from gait [30]. This approach may also be utilized to diagnose other symptoms such as speech. It can also be applied in any real-time application by detecting local changes in a signal.

Nevertheless, it is hard to define significant features or patterns that can help in diagnosing PD. Moreover, ML-based assessment methods require large volumes of expensive training data to employ in the classification task.