Supporter-Type Anterior Cruciate Ligament Prevention System Based on Estimation of Knee Joint Valgus Angle Using Stretch Sensors

Abstract

Anterior cruciate ligament (ACL) injuries are common in sports involving jumping and rapid direction changes, often occurring in non-contact situations. The risk of ACL injury is evaluated by knee flexion and valgus angles; a small knee flexion angle combined with a large valgus angle increases the risk. Monitoring these angles during activities can help athletes recognize their ACL injury risk and adjust their movements. Traditional 3D motion analysis, used for measuring knee angles, is costly and impractical for daily practice. This study proposes a knee supporter with stretch sensors to estimate knee flexion and valgus angles in practice settings, evaluating ACL injury risk and notifying athletes of high-risk movements. The proposed device wirelessly transmits data from three stretch sensors placed on the device to a PC and uses machine learning to estimate the knee angles. The results of the evaluation experiments, conducted with data from five healthy male and female participants in their twenties, indicate that the estimation accuracy for the knee flexion angle, achieved by a model trained using a Random Forest Regressor (RFR) with data from individuals other than the target user, resulted in a Mean Absolute Error (MAE) of 8.86 degrees. For the knee valgus angle, a model trained with the user’s own data using the RFR achieved a MAE of 0.81 degrees.

1. Introduction

Anterior cruciate ligament (ACL) injuries are common among athletes who engage in sports such as volleyball and skiing, which involve jumping and rapid changes in direction [1]. Approximately 250,000 ACL injuries occur in the United States [2]. Although ACL injuries can occur at any age, they are particularly prevalent among younger athletes. These injuries can begin as early as ages 10 to 12, with peak incidences reported between ages 14 to 18 for females and 19 to 25 for males [3].

Surgical intervention is often required to restore the function of a damaged ACL, and athletes typically undergo about six months of rehabilitation before returning to their sport. Post ACL reconstruction, many athletes struggle to regain the pre-injury knee function necessary for their sport. Approximately 30% of high school athletes who undergo ACL reconstruction either quit sports or switch to a different activity [4]. Additionally, there is a high risk of subsequent ACL injuries to either the same or the opposite knee post reconstruction [5], necessitating cautious movements to minimize injury risk.

While rigid knee braces can prevent dangerous movements that lead to ACL injuries, they hinder knee joint movement, limiting athletic performance. Alternatively, evaluating ACL injury risk through knee flexion and valgus angles, as shown in Figure 1, provides a screening method. ACL injuries frequently occur in non-contact situations [6], with increased risk when a small knee flexion angle coincides with the knee valgus [7]. The risk also escalates with larger knee valgus angles during jump landings and direction changes in practice [8]. Thus, measuring these angles during practice can screen athletes at high risk for ACL injuries.

Research on estimating knee flexion and valgus angles (collectively referred to as knee joint angles) commonly uses 2D and 3D motion analysis. Two-dimensional motion analysis is simple, low-cost, and quick, making it suitable for large-scale screenings. However, its accuracy is limited by the potential misinterpretation of hip rotation or lower limb alignment as knee valgus. Conversely, 3D motion analysis provides precise knee joint angle estimates but is impractical for evaluating knee movements during regular games and practices due to its laboratory setting and high costs. This makes it inaccessible for high school students who are prone to ACL injuries, thereby unsuitable for large-scale screenings.

This study aimed to develop a system to evaluate ACL injury risk in athletes’ regular practice environments and notify those performing high-risk movements. This paper proposes a method to estimate knee flexion and valgus angles using a supporter device equipped with stretch sensors, and its accuracy was evaluated.

The remainder of this paper is structured as follows: Section 2 reviews related research, Section 3 describes the proposed system, Section 4 explains the evaluation experiments of the knee joint angle estimation method, Section 5 presents the results, Section 6 discusses the findings, and Section 7 concludes the paper.

2. Related Research

2.1. ACL Injuries and Prevention

ACL injuries frequently occur in sports involving jumping and rapid direction changes, such as volleyball and skiing [1]. Most of these injuries are non-contact [6], resulting from internal risk factors (e.g., physical characteristics, athletic abilities) interacting with external risk factors (e.g., sport-specific movements, environmental factors). Reducing these risk factors can mitigate ACL injury risks. However, modifying sport-specific movements and environmental factors, such as ground conditions influenced by weather, is challenging. Therefore, reducing internal risk factors is crucial for ACL injury prevention.

Several studies have examined internal risk factors for ACL injuries. David et al. reported a significant gender-based discrepancy in ACL injury rates among athletes, observing that female players in soccer, rugby, and basketball experience ACL injuries at a rate approximately four times higher than their male counterparts [9]. Differences in knee anatomy, such as a smaller femoral notch in females, can predispose individuals to ACL injuries [10]. Many factors are involved in ACL injury, including hormone levels [11].

Among modifiable internal risk factors, the knee valgus angle has garnered attention. Hewett et al. reported that athletes exhibiting knee valgus during jump landings are at higher risk for ACL injuries [8]. Withrow et al. found that knee flexion with valgus significantly strains the ACL [12]. Thus, correcting knee valgus is vital for reducing ACL injury risk.

Training programs to prevent ACL injuries by avoiding knee valgus during jump landings are effective. Mehl et al. summarized various prevention programs, including jump, running, flexibility exercises, balance training, and strength training, which reduce ACL injury risks [13]. Noyes et al. reported that jump and weight training decreased knee valgus angles during landings [14]. Early screening and prevention training are effective for young athletes [15], making early detection crucial. Although the knee valgus angle measurement is used for screening, conventional high-accuracy methods are costly and time-consuming, hindering accessibility for the youth.

2.2. Camera-Based Knee Joint Angle Estimation

Currently, 3D motion analysis is commonly used for knee joint angle measurements. Systems like VICON utilize infrared cameras and reflective markers to accurately capture joint coordinates [16]. Heebner et al. used optical motion capture systems to estimate knee joint angles for optimizing ACL injury assessment and rehabilitation [17]. Donnelly et al. analyzed knee valgus moments and ACL injury risk mechanisms during non-contact side steps using similar systems [18]. However, these methods are costly, time-consuming, and restricted to laboratory settings.

Two-dimensional motion analysis, which has been gaining attention recently, offers simplicity, a low cost, and a short execution time, making it suitable for large-scale screenings. Sykes estimated knee joint angles during running using digital video cameras [19]. However, 2D analysis can misinterpret hip rotation or lower limb alignment as knee valgus, limiting accuracy. Both 2D and 3D analyses rely on cameras, restricting angle estimation to the camera’s field of view, and are thus unsuitable for estimating angles during matches or practices.

2.3. Sensor-Based Knee Joint Angle Estimation

Non-camera-based methods estimate knee joint angles using inertial measurement units (IMUs) attached to the user’s shins and knees or embedded in tightly fitting clothing [20,21]. Lou et al. developed a portable rehabilitation monitoring system using wearable IMUs to estimate knee joint range of motion, achieving a mean absolute error (MAE) of $6.0°$ to $8.5°$ [22]. Tedesco et al. created the SKYRE platform for the remote assessment of patients undergoing knee rehabilitation, using nine-axis accelerometers and electromyography sensors, with MAEs ranging from $0.16°$ to $4.94°$, comparable to 3D motion analysis [23].

Methods involving stretch sensors attached to clothing or bandages for knee joint coordinate estimation also exist [24,25]. Ohnishi et al. proposed a system using stretch sensors on knee supporters to estimate knee flexion angles, achieving MAEs from $1.6°$ to $4.2°$ [26]. Wood et al. developed a knee sleeve with 16 piezoresistive sensors to estimate knee flexion and external rotation angles, with root mean square errors of $12.6°$ and $3.5°$, respectively [27]. While methods for estimating knee flexion angles without cameras exist, methods for estimating knee valgus angles are lacking.

3. Proposed System

This section describes a system that measures knee joint flexion and valgus/varus states using stretch sensors during athletes’ training. The system estimates knee joint angles from sensor data and notifies users of their ACL injury risk.

3.1. Intended Environment

This research aimed to develop a system that detects athletes at high risk of ACL injury in their regular training environments using wearable devices and notifies them of the risk. The system implementation met the following four requirements:

• No restriction on movement. The proposed system was intended for use during athletes’ regular training, so it was designed to not restrict their range of motion or activities. Thus, the system used a knee supporter that athletes could wear without discomfort, ensuring no restriction on movement.
• Capable of measuring knee joint state. Preventing ACL injuries requires estimating the knee valgus angle during jump landings and direction changes, as well as the valgus angle when the knee flexion angle is small. Therefore, the system has to measure the knee joint state. The proposed system uses stretch sensors that are elastic and easy to attach to the supporter to measure changes in knee joint angles.
• No need for extensive equipment. The proposed system was intended for use in athletes’ regular training environments, so it did not require extensive systems like multiple cameras for angle estimation. This research aimed to develop a system that used only a knee supporter, stretch sensors, and a PC to estimate knee joint angles and evaluate ACL injury risk.
• No need for specialized skills for attachment and measurement. Continuous knee joint angle measurements need to minimize measurement errors caused by individual characteristics and attachment/detachment. Thus, the system is simple to use without requiring specialized skills. The proposed system allows users to wear a knee supporter device and perform calibration through flexion exercises during warm-up to minimize errors due to attachment/detachment and body shape changes during movement.

3.2. System Configuration

The proposed system comprises a supporter device equipped with stretch sensors and a wireless module, and a PC. The system configuration is shown in Figure 2. The system wirelessly transmits data from stretch sensors on the supporter to the PC. For calibration, users perform predetermined knee flexion and valgus movements at the start of system use. Subsequently, users train normally, and the system continuously estimates knee flexion and valgus angles using machine learning from the sensor data. The system evaluates ACL injury risk based on the estimated angles and notifies users performing high-risk movements, encouraging training to reduce ACL injury risk.

The supporter device is shown in Figure 3. The device uses three C-STRETCH MEASURE stretch sensors from Bando Chemical Industries, Ltd., Kobe, Japan, equipped with wireless modules [28]. Sensor data are transmitted wirelessly to the PC in millimeters. The supporter, a ProFits knee supporter (PS271) from PIP Co., Ltd., Osaka-shi, Japan [29], does not overly restrict knee movement. Based on knowledge from rehabilitation and sports science experts, the stretch sensors are positioned to capture knee flexion and valgus states. Three stretch sensors are placed on the back of the supporter, with one on the patella side for flexion movement, and one each on the inner and outer sides of the knee for valgus movement.

3.3. Calibration Method

In this study, we assumed that the knee joint angle and the stretch sensor values can be represented by a linear function, and we calibrated the estimated knee joint angles accordingly. This calibration process for the stretch sensor was designed to reduce fitting errors and individual differences in knee bending. The calibration values were calculated from the flexion data included in the estimation data and the flexion data at the start of system use.

First, a linear approximation is created from the scatter plot of the stretch sensor values and knee joint angles in the flexion data used for estimation. This approximation can be expressed by Equation (1), where $y_1$ is the knee joint angle, x is the stretch sensor value, $a_1$ is the slope, and $b_1$ is the intercept.

$$y_1=a_{1}x+b_1$$

(1)

Next, a linear approximation is created from the scatter plot of the stretch sensor values and knee joint angles in the flexion data at the start of system use. This approximation can be expressed by Equation (2), where $y_2$ is the knee joint angle, x is the stretch sensor value, $a_2$ is the slope, and $b_2$ is the intercept.

$$y_2=a_{2}x+b_2$$

(2)

Expressing $y_2$ in terms of $y_1$ gives Equation (3).

$$y_2={\displaystyle \frac{a_2}{a_1}}(y_1-b_1)+b_2$$

(3)

Therefore, the knee joint angle $z_1$ estimated by machine learning to $z_2$ can be calibrated using Equation (4).

$$z_2={\displaystyle \frac{a_2}{a_1}}(z_1-b_1)+b_2$$

(4)

For calibration in the flexion direction, the stretch sensor attached to the patella side is used, and for calibration in the valgus direction, the stretch sensor attached to the inner side of the knee is used.

3.4. Knee Joint Angle Estimation Method

In the proposed system, features are extracted from the values obtained from the three stretch sensors placed on the supporter device, and machine learning is used to estimate the knee flexion angle and knee valgus angle. The features are the same for both the left and right sides, so a machine-learning model is created regardless of the side. The features used are the instantaneous values of the three stretch sensors on the outer, central, and inner sides of the knee, normalized, along with the past instantaneous values obtained at 50 Hz. Since the central sensor value becomes larger compared to the other two sensors, the three stretch sensor values are normalized. The maximum and minimum values of the stretch sensors used for normalization are calculated from the stretch sensor values during flexion at the start of system use. The machine learning model uses a regressor, and the data analyses, including feature calculation, are programmed in Python. The combination of machine learning models and features used for estimation will be compared and investigated in the evaluation experiments in Section 4.

3.5. Notification Method

Regarding the notification method, the user can receive notifications at any time during practice. The proposed system immediately notifies the user with a buzzer sound if the estimated knee flexion angle and knee valgus angle indicate a high ACL injury risk. Since the load on the ACL increases when the knee is valgus at a small knee flexion angle, the proposed system notifies the user at the moment such an angle is estimated.

4. Evaluation

An evaluation experiment was conducted to verify whether the proposed method can estimate knee flexion and knee valgus angles, which are indicators of ACL injury risk. In the experiment, participants wearing the knee supporter device used in the proposed system performed movements to screen for and measure ACL injury recovery. The stretch sensor values during these movements were used to evaluate the estimation accuracy of knee flexion and knee valgus angles with various machine-learning models. This experiment was approved by the Ethics Committee of Kobe University Graduate School of Engineering for research involving human participants (Approval No. 05-47).

4.1. Experiment Procedure

To evaluate the machine learning models for estimating knee flexion and knee valgus angles, participants wore the supporter device on both knees and performed various movements. The participants performed movements that changed knee flexion and valgus angles while wearing the supporter device shown in Figure 3, and the stretch sensor values during these changes were collected.

The movements conducted in the experiment were chosen in consultation with experts in rehabilitation and sports medicine, referencing ACL injury screening tests [8,15] and rehabilitation tests [17,30]. The following 13 movements were selected, with examples shown in Figure 4:

• Flexion movement (Squat);
• Maximizing knee valgus at 30° flexion;
• Maximizing knee valgus at 60° flexion;
• Maximizing knee valgus at 90° flexion;
• Walking;
• Running;
• Drop jump from a 30 cm platform;
• Single-leg hop;
• Three single-leg hops;
• Three consecutive jumps;
• Three single-leg cross jumps;
• Cutting to the right;
• Cutting to the left.

Figure 4. Movement examples. White arrows indicate the direction of movement.

Figure 4. Movement examples. White arrows indicate the direction of movement.

Movements involving a single leg were performed on both legs, and each movement was repeated three times consecutively. The device was not removed between movements. The sampling frequency of the stretch sensors used in this experiment was 100 Hz.

The experimental environment is shown in Figure 5. To obtain ground truth data, a VICON NEXUS 2 3D motion analysis system (Vicon Motion Systems, Oxford, UK) consisting of 16 infrared cameras was used to measure at a sampling frequency of 100 Hz. Infrared reflective markers were attached to 43 landmarks on the body. The ground truth knee joint angles were obtained by creating segments for the left and right feet, lower legs, thighs, and pelvis from 24 markers on the lower body, as shown in Figure 6, and the Euler angles between the thigh and lower leg segments were calculated.

The calibration procedure was as follows: Mask all unnecessary reflections from the infrared cameras. In the capture volume, wave an active LED wand to ensure that the markers are visible to the cameras throughout the entire area where 3D data will be captured. The system continues calibration until sufficient calibration information is acquired, with the mean calibration residuals for trials remaining under 1.00 mm. The laboratory coordinate system was defined using a right-handed coordinate system.

Marker coordinates were filtered using a low-pass Butterworth filter of the fourth order with a cut-off frequency of 6 Hz to remove noise. Gaps in the marker data were interpolated using the Pattern Fill or Rigid Body Fill methods [31,32].

Infrared-reflecting markers, 14 mm in diameter, were attached based on anatomical landmarks by two physical therapists, each with over ten years of clinical experience and expertise in motion analysis. The marker set was referenced from our previous study [33]. The local coordinate system for the femur was established with the origin at the midpoint between the lateral and medial epicondyles. The hip joint center was calculated based on a previous study [34], using the anterior superior iliac spine and the greater trochanter. The Z-axis was defined as the line connecting the hip joint center to the origin. The X-axis was defined as the line connecting the lateral and medial epicondyles. The Y-axis was orthogonal to both the X and Z axes. For the tibia, the local coordinate system was established with the origin at the midpoint between the lateral and medial condyles. The Z-axis was defined as the line connecting the midpoint of the medial and lateral malleoli to the origin. The X-axis was defined as the line connecting the lateral and medial condyles. The Y-axis was orthogonal to both the X and Z axes. Using these local coordinate systems, the knee joint angles were calculated based on Euler angles with an XYZ rotation sequence. This method allows for the precise determination of flexion/extension, internal/external rotation, and varus/valgus movements of the knee joint.

Walking and running speeds for the movement tasks were at the discretion of the participants, and the measurement environment consisted of an 8 m walkway. The participants were three healthy males and two healthy females (age $22.8\pm 1.0$ years, height $166.0\pm 8.4$ cm, body mass $59.8\pm 8.8$ kg), all of whom had experience in sports such as baseball and soccer. Hereafter, the five participants are referred to as participants A, B, C, D, and E. Participants A, C, and E were male, and participants B and D are female. None of the participants had a history of knee joint trauma, surgeries, or ACL injuries.

4.2. Evaluation Method

To evaluate the machine learning algorithms and features used for knee joint angle estimation in the proposed system, the accuracy of knee flexion and knee valgus angle estimations was verified through cross-validation using the stretch sensor values from the supporter device worn on each knee. Cross-validation is a method for assessing the generalization performance of a predictive model by partitioning the sample data, using one part as the validation data and the rest as the training data. In this evaluation, within-participant cross-validation and between-participant cross-validation were used as described below:

• Within-participant Cross-Validation
  Three-fold cross-validation using data from the same participant.
• Between-participant Cross-Validation
  Validation using all trial data from one participant as the test data and all trial data from the remaining participants as the training data.

The user’s burden varies depending on the data used for training. If the accuracy is high with within-participant trial data, the user needs to collect their own training data beforehand, which increases the user’s burden but allows high-accuracy knee joint angle estimation. Conversely, if the accuracy is high with training data from different users, the system can be used without the need for the user’s own ground truth data, reducing the user’s burden.

The proposed system aimed to evaluate ACL injury risk during actual exercises without requiring special equipment. Thus, it is ideal to estimate knee joint angles from other users’ training data without collecting the user’s own ground truth data, making good between-participant cross-validation results desirable.

A Random Forest Regressor (RFR) [35] and Linear SVR (LSVR) [36] were used as the machine learning algorithms. These were selected using Auto ML in a preliminary survey using collected data. The features extracted from the three stretch sensors on the supporter device for each leg were the following:

 Feature A: 

Instantaneous values: a single sample of data output from the sensor (total of three features)

 Feature B: 

Instantaneous values and past 20 samples of instantaneous values obtained at 50 Hz (total of 63 features)

These features were normalized for each leg based on the stretch sensor values during flexion, using min-max normalization. Min-max normalization is represented by the following Equation (5):

$$X^{\prime }={\displaystyle \frac{X-X_{min}}{X_{max}-X_{min}}}$$

(5)

where X is the original value, $X_{min}$ is the minimum value of the feature, and $X_{max}$ is the maximum value.

In between-participant cross-validation, the estimated knee joint angles were calibrated using Equation (4) to reduce differences in body size and device attachment among participants. The slope $a_2$ and intercept $b_2$ were calculated from the flexion data of the test participant, while the slope $a_1$ and intercept $b_1$ were obtained from the scatter plots of the flexion data of the remaining participants. Calibration was not performed in within-participant cross-validation because the supporter device was not removed during the experiments.

For the evaluation of the estimation results, the features and machine learning algorithms used in the proposed method were determined based on the results of cross-validation. The evaluation metrics were the mean absolute error (MAE) and root mean squared error (RMSE) between the estimated knee joint angles and the actual angles obtained by the 3D motion analysis system. ACL injury risk is higher when the knee valgus angle is large at a small knee flexion angle. Therefore, it is crucial to detect high-risk athletes, even if it means occasionally misclassifying low-risk athletes. Consequently, when determining the machine learning algorithms based on the estimation results, it is necessary to ensure that the knee flexion angle is not overestimated and the knee valgus angle is not underestimated.

5. Results

This section describes the results of the evaluation experiments detailed in Section 4. Since the proposed system is intended for use during training, we focused on movements frequently performed during training (running) and high-risk movements for injury (three consecutive jumps and cutting) to determine the machine learning algorithm. No differences were observed between the left and right sides in the experimental results, so all subsequent estimation result figures only show the right leg. Additionally, the data for running and cutting for participant A and cutting for participant B were missing and thus excluded from the discussion.

5.1. Knee Flexion Angle

5.1.1. Within-Participant Cross-Validation

Table 1. Estimation errors of knee flexion angle in within-participant cross-validation.

	RFR a		LSVR b
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	4.38	3.33	4.93	4.75
RMSE [$°$]	5.76	4.33	5.96	5.71

^a Random Forest Regressor. ^b Linear SVR.

shows the mean MAE and RMSE of knee flexion angles for all movements in within-participant cross-validation. Table 1 indicates that both the MAE and RMSE were smaller with Feature B, suggesting that when estimating knee flexion angles using the user’s data, it is better to use instantaneous values and the past 20 samples of instantaneous values.

To confirm the estimation accuracy of each machine learning algorithm, we compared the knee flexion angle estimation results using instantaneous values and the past 20 samples of instantaneous values. Figure 7 shows the within-participant cross-validation results for running for participant E using RFR and LSVR. In the analysis of Figure 7a,b, both algorithms did not significantly deviate in their estimations. However, based on the minimum values of the estimation results for each algorithm, the RFR had a higher accuracy for smaller knee flexion angles. Thus, for movements with significant knee flexion angle changes like running, the RFR is more suitable for estimating knee flexion angles.

Figure 8 shows the within-participant cross-validation results for three consecutive jumps for participant E using the RFR and LSVR. In the analysis of Figure 8a,b, both algorithms did not significantly deviate in their estimations. In the analysis of the minimum values of the estimation results from different algorithms, the RFR had a higher accuracy for smaller knee flexion angles. Thus, for movements that impact the knee joint like jumps, the RFR is more suitable for estimating knee flexion angles.

Figure 9 shows the within-participant cross-validation results for cutting for participant D using the RFR and LSVR. From the comparison of Figure 9a,b, it is evident that the RFR accurately estimated both larger and smaller knee flexion angles. Thus, for movements with significant knee valgus angle changes like cutting, the RFR is more suitable for estimating knee flexion angles.

The results of these three movements suggest that when estimating knee flexion angles using the user’s data, it is better to use the RFR as the machine learning algorithm and instantaneous values and the past 20 samples of instantaneous values as features.

5.1.2. Between-Participant Cross-Validation

Table 2. Estimation errors of knee flexion angle in between-participant cross-validation.

	RFR		LSVR
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	8.73	8.86	8.35	9.49
RMSE [$°$]	10.84	11.06	9.93	10.86

shows the mean MAE and RMSE of knee flexion angles for all movements in between-participant cross-validation. Table 2 indicates that both the MAE and RMSE are smaller with Feature A, suggesting that when estimating knee flexion angles using data from other users, it is better to use instantaneous values only.

To confirm the estimation accuracy of each machine learning algorithm, we compared the knee flexion angle estimation results using instantaneous values only. Figure 10 shows the between-participant cross-validation results for running from all participants using the RFR and LSVR. In an analysis of Figure 10a,b, both algorithms showed consistent estimations without major deviations. Additionally, when examining the minimum values of the estimation results for participants C and E, the RFR demonstrated a higher accuracy for smaller knee flexion angles. Thus, for movements with significant knee flexion angle changes like running, the RFR is more suitable for estimating knee flexion angles.

Next, the estimation results from the inter-participant cross-validation of the RFR and LSVR for the triple jump are presented in Figure 11. In the analysis of panels (a) and (b) of Figure 11, neither machine learning algorithm significantly deviated from the estimated values. A comparison of the minimal values in the estimation results for participants A and E, segmented by an algorithm, showed that the RFR had a higher estimation accuracy for smaller knee flexion angles. Therefore, for movements like jumping that impact the knee joints, the RFR is better suited for estimating knee flexion angles.

Figure 12 displays the results from the inter-participant cross-validation of the RFR and LSVR for cutting. In panel (a), the LSVR achieved a better accuracy in estimating smaller knee flexion angles for participant C, while the RFR excels in similar conditions for participant E in panel (b). These findings imply that for actions like cutting, where the knee valgus angle undergoes substantial changes, both machine learning algorithms could be effectively utilized.

The analysis from these three types of movements indicates that using data from other participants to estimate knee flexion angles tends to yield better results with the RFR, particularly when only instantaneous values are employed as features. While the MAE and RMSE values were higher in inter-participant cross-validation compared to intra-participant, the overall differences in estimation across various movements were minimal.

The proposed method ideally allows for the estimation of knee joint angles using only data from other individuals, without the need to collect accurate data from the user beforehand. Therefore, using data from other individuals for estimating knee flexion angles is preferable. Based on these results, it is suggested that the proposed system should utilize the RFR algorithm and only instantaneous values from other individuals’ data for the estimation of knee flexion angles.

5.2. Knee Valgus Angle

5.2.1. Within-Participant Cross-Validation

Table 3. Estimation errors of knee valgus angle in within-participant cross-validation.

	RFR		LSVR
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	1.11	0.81	1.49	1.33
RMSE [$°$]	1.50	1.01	1.76	1.57

shows the mean MAE and RMSE of knee valgus angles for all movements in within-participant cross-validation. Table 3 indicates that both the MAE and RMSE are smaller with Feature B, suggesting that when estimating knee valgus angles using the user’s data, it is better to use instantaneous values and the past 20 samples of instantaneous values as features.

To confirm the estimation accuracy of each machine learning algorithm, we compared the knee valgus angle estimation results using instantaneous values and the past 20 samples of instantaneous values as features. First, Figure 13 shows the within-participant cross-validation results for running for participant C using the RFR and LSVR. In the analysis of Figure 13a,b, the LSVR demonstrated a higher accuracy for larger knee valgus angles. However, since the knee valgus angle is only about 2° at its maximum, which is not large enough to cause an ACL injury, it is difficult to determine which algorithm is more suitable based on this result.

Next, the results of the internal cross-validation for the triple jumps of participant B using both the Random Forest Regressor (RFR) and Linear Support Vector Regressor (LSVR) are shown in Figure 14. In the analysis of the maximum values from Figure 14a,b across machine learning algorithms, the RFR exhibited a superior estimation accuracy for instances when the knee valgus angle was large. Therefore, for movements like jumping, which involve impacts on the knee joint, the RFR was more suitable for estimating the knee valgus angle.

Lastly, the internal cross-validation results for the cutting movement of participant B using both the RFR and LSVR are displayed in Figure 15. In the analysis of Figure 15a,b for each algorithm, the RFR again showed a higher estimation accuracy when the knee valgus angles were larger. Thus, for activities like cutting, where the knee valgus angle changes significantly, the RFR was also found to be more suitable for estimating the knee valgus angle.

From the results of these three types of movements, it is suggested that when using the user’s own data to estimate knee valgus angles, the Random Forest Regressor algorithm and features comprising the user’s instant values and the most recent 20 sample values provide better accuracy.

5.2.2. Between-Participant Cross-Validation

Table 4. Estimation errors of knee valgus angle in between-participant cross-validation.

	RFR		LSVR
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	4.33	3.65	4.02	4.09
RMSE [$°$]	5.24	4.37	4.30	4.33

shows the mean MAE and RMSE of knee valgus angles for all movements in between-participant cross-validation. Table 4 indicates that for the RFR, both the MAE and RMSE are smaller with Feature B, suggesting that when estimating knee valgus angles using data from other users, it is better to use instantaneous values and the past 20 samples of instantaneous values as features. For the LSVR, there was no significant difference in the MAE and RMSE between the features.

To confirm the estimation accuracy of each machine learning algorithm, we compared the knee valgus angle estimation results using instantaneous values and the past 20 samples of instantaneous values. The knee valgus angle estimation results for participant D in the between-participant cross-validation were significantly larger compared to the other participants, so a separate figure was used.

First, Figure 16 shows the between-participant cross-validation results for running for participants B, C, and E using the RFR and LSVR, and Figure 17 shows the results for participant D. In the analysis of Figure 16 and Figure 17 for each machine learning algorithm, it is observed that the LSVR provided estimation values that varied more according to the actual changes in knee valgus angle but underestimated the knee valgus angle compared to the true values for participant E in Figure 16. Additionally, for participant D in Figure 17a,b, the LSVR provided estimation values that varied more according to the changes in knee valgus angle but consistently estimated values large enough to cause ACL injury. Therefore, for movements with significant knee flexion angle changes like running, neither machine learning algorithm seems suitable for estimating knee valgus angles.

5.2.3. Between-Participant Cross-Validation

Figure 18 shows the between-participant cross-validation results for three consecutive jumps for participants A, B, C, and E using the RFR and LSVR, and Figure 19 shows the results for participant D. In the analysis of Figure 18 and Figure 19 for each machine learning algorithm, both algorithms sometimes underestimated or overestimated the knee valgus angle compared to the true values, with large deviations. In the analysis of the maximum values of the knee valgus angle estimation results for participant C in Figure 18, the RFR demonstrated a higher estimation accuracy than the LSVR. Thus, for movements that impact the knee joint like jumps, the RFR might be more suitable for estimating knee valgus angles. However, due to the large errors observed in the results for other participants, using the RFR with data from other users might not be effective for estimating knee valgus angles during jumps.

Finally, Figure 20 shows the between-participant cross-validation results for cutting for participants A, B, C, and E using the RFR and LSVR, and Figure 21 shows the results for participant D. In the analysis of Figure 20 and Figure 21 for each machine learning algorithm, comparing the maximum values of the estimation results for participant E in Figure 20, the RFR less frequently underestimated the knee valgus angle. Thus, for movements with significant knee valgus angle changes like cutting, the RFR might be more suitable for estimating knee valgus angles.

Based on the validation results of these three movements, when estimating knee valgus angles using data from other users, it is suggested to use the RFR as the machine learning algorithm and to use instantaneous values and the past 20 samples of instantaneous values as features.

The results from within-participant and between-participant cross-validation for knee valgus angles showed minimal differences in the MAE and RMSE. However, the estimation results from the between-participant cross-validation exhibited large errors across all movements, with noticeable variations in estimation accuracy. Therefore, it is suggested that the proposed method requires using the user’s own data for training the model to estimate knee valgus angles accurately. Consequently, for estimating knee valgus angles, it is suggested to use the RFR as the machine learning algorithm and to use instantaneous values and the past 20 samples of instantaneous values from the user’s own data.

From the estimation results of knee flexion angles and knee valgus angles, the evaluation of knee flexion angles in Section 5.1 indicates that it was possible to estimate knee flexion angles using data from other users. However, since estimating knee valgus angles requires the user’s own data, it is more accurate for the user if both knee flexion and knee valgus angles are estimated using the user’s own data, which also allows for the same burden on the user. Therefore, it is suggested to use the RFR as the machine learning algorithm and to use instantaneous values and the past 20 samples of instantaneous values from the user’s own data.

When purchasing the proposed system, users need to exercise in an environment where stretch sensor values and knee joint angles can be measured for about an hour, similar to this experiment. They should collect their own data, build a machine-learning model using this data, and integrate the model into the system for use.

5.3. Consideration

Regarding cases where estimation did not go well, Figure 21 shows that the knee valgus angle estimation results for participant D had significant deviations from the actual values. Figure 22 shows the between-participant cross-validation results for participant D’s knee valgus angle during cutting, both before and after calibration. The estimation results before calibration were not as significantly large as those after calibration. Therefore, it is possible that the low estimation accuracy for participant D’s knee valgus angle was due to issues with the calibration performed during between-participant cross-validation. Figure 23 shows the scatter plots of the stretch sensor values on the medial side of the knee during flexion and the actual knee valgus angles for each participant. As the knee valgus angle increases, the knee is in a valgus position, and as the knee valgus angle decreases, the knee is in a varus position. From Figure 23, it can be seen that for participants A and B, the knee valgus angle decreases as the stretch sensor value increases, while for participants C, D, and E, the knee valgus angle increases as the stretch sensor value increases. Notably, participant D, who had a low estimation accuracy, showed a significant change in the knee valgus angle. These individual characteristics, such as whether the knee is in a varus or valgus position during flexion, may have caused the calibration to be ineffective.

Next,

Table 5. Estimation errors of knee flexion angle in between-participant cross-validation for participants C, D, and E.

	RFR		LSVR
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	10.35	9.14	9.05	8.86
RMSE [$°$]	12.86	11.48	11.12	10.72

shows the mean MAE and RMSE of knee flexion angles for all movements for participants C, D, and E. From Table 5, it can be seen that both the MAE and RMSE are smaller with Feature B, suggesting that when estimating knee flexion angles using data from other users, it is better to use instantaneous values and the past 20 samples of instantaneous values as features. To confirm the estimation accuracy of each machine learning algorithm, we compared the knee flexion angle estimation results using instantaneous values and the past 20 samples of instantaneous values. Figure 24 shows the between-participant cross-validation results for cutting using the RFR and LSVR. In the analysis of the minimum values of the estimation results for participant E in Figure 24, the RFR had a higher accuracy for smaller knee flexion angles.

Table 6. Estimation errors of knee valgus angle in between-participant cross-validation for participants C, D, and E.

	RFR		LSVR
	Feature A	Feature B	Feature A	Feature B
MAE [$°$]	2.14	2.01	1.87	1.83
RMSE [$°$]	2.57	2.38	2.11	2.04

shows the mean MAE and RMSE of knee valgus angles for all movements for participants C, D, and E. From Table 6, it can be seen that both the MAE and RMSE are smaller with Feature B, suggesting that when estimating knee valgus angles using data from other users, it is better to use instantaneous values and the past 20 samples of instantaneous values as features. To confirm the estimation accuracy of each machine learning algorithm, we compared the knee valgus angle estimation results using instantaneous values and the past 20 samples of instantaneous values. Figure 25 shows the between-participant cross-validation results for cutting using the RFR and LSVR. In the analysis of the maximum values of the estimation results for participant D in Figure 25, the RFR had a higher accuracy for larger knee valgus angles. Additionally, in the analysis of the estimation results for participant D in Figure 21a and Figure 25a, the accuracy of the maximum values in Figure 25a had improved.

Based on these results, it is suggested that by dividing participants into two groups based on their knee movement characteristics during flexion and creating separate machine learning models, it might be possible to estimate knee flexion and knee valgus angles using models trained with data from other users, using the RFR.

6. Discussion

In this paper, we proposed a system that uses stretch sensors attached to a knee supporter to measure the knee angles during practice in order to notify them of ACL injury risk. Based on the evaluation results of the knee flexion angle estimation accuracy experiment, the MAE was 0.81$°$ when using a model trained with the RFR on the user’s own data. Experts indicated that in rehabilitation settings, estimating knee valgus angles of 10$°$ or more would be practical for actual use. In this experiment, there were few movements where the knee valgus angle changed by 10$°$ or more. However, during the cutting motion for participant D, as shown in Figure 25, the knee valgus angle was approximately 10$°$, suggesting that it might be possible to estimate knee valgus angles of 10$°$ or more even in between-participant cross-validation. Although it cannot be concluded that the current experiment can estimate conditions that would likely result in ACL injury, the results suggest that by creating training data with groups exhibiting similar flexion tendencies, it might be possible to estimate knee valgus angles of 10$°$ or more using only other users’ data. Since movements that result in knee valgus angles changing by 10$°$ or more could potentially cause ACL injuries, it is challenging to collect such data. Therefore, in future work, we will create a knee model and collect data on knee joint angles in conditions where the knee bends enough to cause injury to verify if the system can estimate knee angles that would result in actual ACL injuries.

In this study, the MAE for the knee flexion angle was approximately 3.3 degrees, comparable to the range of 1.6 to 4.2 degrees reported in Ohnishi et al.’s study [26]. Furthermore, this paper uniquely contributes by also estimating the knee valgus angle. Additionally, as an injury prediction system, the ability to obtain moment-to-moment knee angles is an advantage, allowing for the specific analysis of movements that may lead to injuries.

In this experiment, all participants used the same M-sized supporter. If the proposed system’s device comprised supporters of different sizes, the stretch sensor values might differ from the current training data even if the sensors were placed in the same positions. However, the results indicate that knee flexion and valgus angles can still be estimated using other users’ training data after calibration, even if the stretch sensor values vary among participants. Therefore, it is considered that knee flexion and valgus angles can be estimated with the same training data even for different sizes of supporter-type devices.

In this experiment, the ground truth knee joint angles were obtained using VICON [17], necessitating laboratory conditions. Consequently, participants might not have been able to run at full speed, and the current training data may not directly apply to actual running motions at higher speeds. If this is the case, using stretch sensors with a higher sampling frequency than the current 100 Hz for the supporter-type device might address the issue.

The supporter-type device used three stretch sensors. Although increasing the number of stretch sensors might improve the estimation accuracy, it would require additional wireless modules to transmit the sensor data, potentially increasing the device’s weight and hindering user movement. Since it was possible to estimate knee flexion and valgus angles with three stretch sensors, it is considered that the ACL injury risk can be evaluated without increasing the number of stretch sensors.

As the proposed system is intended for use during exercise, there is a possibility that the supporter may shift during use. In the future, measures such as adding position markers to the supporter to indicate proper alignment or making the supporter more resistant to shifting will be necessary.

7. Conclusions

In this paper, we proposed a system that measures the knee conditions of athletes during practice using stretch sensors installed on a knee supporter and notifies the ACL injury risk based on the estimated knee flexion and valgus angles derived from the sensor data. In the evaluation experiment, participants wearing the supporter-type device performed activities such as running and jumping, and the stretch sensor values and knee joint angles during these activities were collected to verify whether the proposed system could estimate knee flexion and valgus angles.

The evaluation results show that the knee flexion angle estimation accuracy was good even with models trained on data from other users. The method using the RFR as the machine learning algorithm and instantaneous stretch sensor values as features achieved an MAE of 8.73$°$. For the knee valgus angle, models trained on the user’s own data had a better estimation accuracy. The method using the RFR and instantaneous stretch sensor values along with the past 20 samples of instantaneous values achieved an MAE of 0.81$°$. The estimation of knee valgus angles required the user’s own data, and since the user could measure the ground truth of knee flexion angles under the same conditions, it is suggested that the user’s own data should also be used for knee flexion angle estimation.

Even using the gold standard 3D motion captures, the knee valgus angles for intense motion may show large variations. Since this was an evaluation of how much the data deviate from the motion capture data, the ground truth value might not be accurate for movements where the motion captures were unreliable. The difficulty in obtaining truthful values for the frontal plane motion of the knee is a limitation of this research.

Therefore, when purchasing the proposed system, users need to exercise in an environment similar to that of this experiment, where stretch sensor values and knee joint angles can be measured for about an hour. They should collect their own data, build a machine-learning model using this data, and integrate the model into the system for use. Although the estimation accuracy of knee valgus angles with data from other users was low, the results suggest that it might be possible to estimate knee flexion and valgus angles by creating machine learning models based on trends in stretch sensor values and knee valgus angles during flexion. Hence, if the knee bending characteristics can be confirmed in advance, the proposed system could potentially be used without the user needing to collect stretch sensor values and knee joint angles during exercise.

In future work, we plan to create a lower limb model to verify whether the system can estimate knee joint angles for movements that are difficult to obtain in vivo and could potentially result in ACL injuries.