research-article

Event-triggered critic learning impedance control of lower limb exoskeleton robots in interactive environments

Authors:

Jiangping Hu, and

Bijoy Kumar GhoshAuthors Info & Claims

Volume 564, Issue C

https://doi.org/10.1016/j.neucom.2023.126963

Published: 01 February 2024 Publication History

Abstract

In this paper, we present an event-triggered critic learning impedance control algorithm for a lower limb rehabilitation exoskeleton robot in an interactive environment, where the control objective is specified by a desired impedance model. In comparison to many other traditional impedance controller design algorithms, in this paper, the impedance control problem is transformed into an optimal control problem. Firstly, the interactive environment accounts for the interaction between the exoskeleton, the human, and the environment, and is modeled by a linear time-invariant exogenous system. Secondly, in contrast to time-triggered control design mechanisms, the event-triggered controller is updated only when the system states deviate from prescribed threshold values. To obtain the event-triggered optimal controller, a critic neural network is developed through the framework of reinforcement learning. A modified gradient descent method is introduced to update the weights of the critic network with an additional stable term employed to eliminate the need for an initial admissible control. Meanwhile, with the simultaneous application of historical and transient state data to the critic neural network, the persistent excitation conditions are relaxed. The Lyapunov method is used to rigorously demonstrate the stability of the overall system. Finally, the effectiveness of the proposed algorithm is demonstrated via simulation.

References

[1]

Dollar A.M., Herr H., Lower extremity exoskeletons and active orthoses: Challenges and state-of-the-art, IEEE Trans. Robot. 24 (1) (2008) 144–158.

[2]

Huang R., Cheng H., Guo H., Lin X., Zhang J., Hierarchical learning control with physical human-exoskeleton interaction, Inform. Sci. 432 (2018) 584–595.

[3]

Ju Z., Ouyang G., Wilamowska-Korsak M., Liu H., Surface EMG based hand manipulation identification via nonlinear feature extraction and classification, IEEE Sens. J. 13 (9) (2013) 3302–3311.

[4]

Li Z., Kang Y., Xiao Z., Song W., Human–robot coordination control of robotic exoskeletons by skill transfers, IEEE Trans. Ind. Electron. 64 (6) (2017) 5171–5181.

[5]

Wen Y., Si J., Gao X., Huang S., Huang H.H., A new powered lower limb prosthesis control framework based on adaptive dynamic programming, IEEE Trans. Neural Netw. Learn. Syst. 28 (9) (2017) 2215–2220.

[6]

Sankai Y., HAL: hybrid assistive limb based on cybernics, Robot. Res. (2011) 25–34.

[7]

Yang Y., Huang D., Dong X., Enhanced neural network control of lower limb rehabilitation exoskeleton by add-on repetitive learning, Neurocomputing 323 (2019) 256–264.

[8]

Li Y., Ge S.S., Impedance learning for robots interacting with unknown environments, IEEE Trans. Control Syst. Technol. 22 (4) (2014) 1422–1432.

[9]

Jung S., Hsia T.C., Robust neural force control scheme under uncertainties in robot dynamics and unknown environment, IEEE Trans. Ind. Electron. 47 (2) (2000) 403–412.

[10]

He W., Li Z., Chen C.L.P., A survey of human-centered intelligentrobots: Issues and challenges, IEEE/CAA J. Autom. Sin. 4 (4) (2017) 602–609.

[11]

Sharifi M., Behzadipour S., Vossoughi G., Nonlinear model reference adaptive impedance control for human–robot interactions, Control Eng. Pract. 32 (2014) 9–27.

[12]

Yu X., Li Y., Zhang S., Xue C., Wang Y., Estimation of human impedance and motion intention for constrained human–robot interaction, Neurocomputing 390 (2020) 268–279.

[13]

Aghili F., Robust impedance-matching of manipulators interacting with uncertain environments: Application to task verification of the space stations dexterous manipulator, IEEE/ASME Trans. Mechatron. 24 (4) (2019) 1565–1576.

[14]

Kim D., Koh K., Cho G.-R., Zhang L.-Q., A robust impedance controller design for series elastic actuators using the singular perturbatio theory, IEEE/ASME Trans. Mechatron. 25 (1) (2020) 164–174.

[15]

Akdoğan E., Aktan M.E., Koru A.T., Arslan M.S., Atlíhan M., Kuran B., Hybrid impedance control of a robot manipulator for wrist and forearm rehabilitation: performance analysis and clinical results, Mechatronics 49 (2018) 77–91.

[16]

He W., Xue C., Yu X., Li Z., Yang C., Admittance-based controller design for physical human–robot interaction in the constrained task space, IEEE Trans. Autom. Sci. Eng. 17 (4) (2020) 1937–1949.

[17]

Li Z., Huang B., Ye Z., Deng M., Yang C., Physical human–robot interaction of a robotic exoskeleton by admittance control, IEEE Trans. Ind. Electron. 65 (12) (2018) 9614–9624.

[18]

Huo W., Mohammed S., Amirat Y., Impedance reduction control of a knee joint human-exoskeleton system, IEEE Trans. Control Syst. Technol. 27 (6) (2018) 2541–2556.

[19]

Li Z., Huang Z., He W., Su C., Adaptive impedance control for an upper limb robotic exoskeleton using biological signals, IEEE Trans. Ind. Electron. 64 (2) (2017) 1664–1674.

[20]

Liu L., Leonhardt S., Ngo C., Misgeld B.J., Impedance-controlled variable stiffness actuator for lower limb robot applications, IEEE Trans. Autom. Sci. Eng. 17 (2) (2019) 991–1004.

[21]

Li X., Liu Y.-H., Yu H., Iterative learning impedance control for rehabilitation robots driven by series elastic actuators, Automatica 90 (2018) 1–7.

[22]

Wang Z., Lee J., Wei Q., Zhang A., Event-triggered near-optimal tracking control based on adaptive dynamic programming for discrete-time systems, Neurocomputing 537 (2023) 187–197.

[23]

Modares H., Lewis F.L., Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning, Automatica 50 (7) (2014) 1780–1792.

Digital Library

[24]

Liu D., Wang D., Wang F., Li H., Yang X., Neural-network based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems, IEEE Trans. Cybern. 44 (12) (2014) 2834–2847.

[25]

Peng Z., Zhao Y., Hu J., Luo R., Ghosh B.K., Nguang S.K., Input–output data-based output antisynchronization control of multi-agent systems using reinforcement learning approach, IEEE Trans. Ind. Inform. 17 (11) (2021) 7359–7367.

[26]

Yu X., He W., Li H., Sun J., Adaptive fuzzy full-state and output-feedback control for uncertain robots with output constraint, IEEE Trans. Syst. Man Cybern. Syst. 51 (11) (2021) 6994–7007.

[27]

Peng Z., Zhang Z., Luo R., Kuang Y., Hu J., Cheng H., Ghosh B.K., Event-triggered optimal control of completely unknown nonlinear systems via identifier-critic learning, Kybernetika 59 (3) (2023) 365–391.

[28]

Zargarzadeh H., Dierks T., Jagannthan S., Optimal control of nonlinearcontinuous-time systems in strict-feedback form, IEEE Trans. Neural Netw. Learn. Syst. 26 (10) (2015) 2535–2549.

[29]

Liu D., Yang X., Wang D., Wei Q., Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints, IEEE Trans. Cybern. 45 (7) (2015) 1372–1385.

[30]

Sun K., Sui S., Tong S., Fuzzy adaptive decentralized optimal control for strict feedback nonlinear large-scale systems, IEEE Trans. Cybern. 48 (4) (2018) 1326–1339.

[31]

Tong S., Sun K., Sui S., Observer-based adaptive fuzzy decentralized optimal control design for strict-feedback nonlinear large-scale systems, IEEE Trans. Fuzzy Syst. 26 (2) (2018) 569–584.

[32]

Mu C., Sun C., Wang D., Song A., Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation, Neurocomputing 260 (2017) 432–442.

Digital Library

[33]

Lv Y., Na J., Zhao X., Huang Y., Ren X., Multi-H ∞ controls for unknown input-interference nonlinear system with reinforcement learning, IEEE Trans. Neural Netw. Learn. Syst. 7 (2021),. early access.

[34]

Peng Z., et al., Data-driven reinforcement learning for walking assistance control of a lower limb exoskeleton with hemiplegic patients, in: 2020 IEEE International Conference on Robotics and Automation, IEEE, 2020, pp. 9065–9071.

[35]

Kong L.H., He W., Yang C.G., Sun C.Y., Robust neurooptimal control for a robot via adaptive dynamic programming, IEEE Trans. Neural Netw. Learn. Syst. 32 (6) (2021) 2584–2594.

[36]

Zhang Q., Zhao D., Zhu Y., Event-triggered H ∞ control for continuous-time nonlinear system via concurrent learning, IEEE Trans. Syst. Man Cybern. 47 (2017) 1071–1081.

[37]

Dong L., Zhong X., Sun C., He H., Adaptive event-triggered control based on heuristic dynamic programming for nonlinear discrete-time systems, IEEE Trans. Neural Netw. Learn. Syst. 28 (7) (2017) 1594–1605.

[38]

Vamvoudakis K.G., Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems, IEEE/CAA J. Autom. Sin. 1 (3) (2014) 282–293.

[39]

Zhu Y., Zhao D., He H., Ji J., Event-triggered optimal control for partially unknown constrained-input systems via adaptive dynamic programming, IEEE Trans. Ind. Electron. 64 (5) (2017) 4101–4109.

[40]

Wang D., He H., Liu D., Improving the critic learning for eventbased nonlinear H ∞ control design, IEEE Trans. Cybern. 47 (10) (2017) 3417–3428.

[41]

Lv Y., Wu Z., Zhao X., Data-based optimal microgrid management for energy trading with integral Q-learning scheme, IEEE Internet Things J. 17 (2023),. early access.

[42]

Yang X., He H., Liu D., Event-triggered optimal neuro-controller design with reinforcement learning for unknown nonlinear systems, IEEE Trans. Syst. Man Cybern. Syst. 49 (9) (2019) 1866–1878.

[43]

Zhao B., Liu D., Event-triggered decentralized tracking control of modular reconfigurable robots through adaptive dynamic programming, IEEE Trans. Ind. Electron. 67 (4) (2020) 3054–3064.

[44]

Lv Y., Ren X., Tian J., Zhao X., Inverse-model-based iterative learning control for unknown MIMO nonlinear system with neural network, Neurocomputing 519 (2023) 187–193.

[45]

Wang D., Mu C., He H., Liu D., Event-driven adaptive robust control of nonlinear systems with uncertainties through NDP strategy, IEEE Trans. Syst. Man Cybern. Syst. 47 (7) (2017) 1358–1370.

Cited By

Wang JZhang JTavoosi JShirkhani M(2024)Machine Learning-Based Multiagent Control for a Bunch of Flexible RobotsComplexity10.1155/2024/13304582024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/1330458
Fu QYang T(2024)Enhancing Service Offloading for Dense Networks Based on Optimal Stopping Theory in Virtual Mobile Edge ComputingJournal of Grid Computing10.1007/s10723-024-09765-322:2Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10723-024-09765-3

Index Terms

Event-triggered critic learning impedance control of lower limb exoskeleton robots in interactive environments

Index terms have been assigned to the content through auto-classification.

Recommendations

Adaptive neural network asymptotic control design for MIMO nonlinear systems based on event-triggered mechanism
Abstract
This article presents an adaptive neural network event-triggered asymptotic tracking control strategy for multi-input and multi-output (MIMO) nonlinear systems with state constraints and unknown dynamics. The prescribed state ...
Read More
Event-Triggered Exponential Synchronization of Nonfragile Neural Networks
Abstract
Based on event-triggered mechanism, the exponential synchronization for a class of neural networks (NNs) with distributed and discrete delays under nonfragile controller is researched in this article. Firstly, whether the controller gain ...
Read More
Exponential passivity of discrete-time switched neural networks with transmission delays via an event-triggered sliding mode control
Abstract
This paper investigates the exponential passivity of discrete-time switched neural networks (DSNNs) with transmission delays via an event-triggered sliding mode control (SMC). Firstly, a novel discrete-time switched SMC scheme is ...
Highlights
- Network communication is embedded into the DSNNs modeling, and the network-induced time delays are consiederd here.
Read More

Comments

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 564, Issue C

Jan 2024

278 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 February 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Wang JZhang JTavoosi JShirkhani M(2024)Machine Learning-Based Multiagent Control for a Bunch of Flexible RobotsComplexity10.1155/2024/13304582024Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1155/2024/1330458
Fu QYang T(2024)Enhancing Service Offloading for Dense Networks Based on Optimal Stopping Theory in Virtual Mobile Edge ComputingJournal of Grid Computing10.1007/s10723-024-09765-322:2Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10723-024-09765-3

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents