Abstract Reinforcement learning (RL) in the context of control systems offers wide possibilities ... more Abstract Reinforcement learning (RL) in the context of control systems offers wide possibilities of controller adaptation. Given an infinite-horizon cost function, the so-called critic of RL approximates it with a neural net and sends this information to the controller (called “actor”). However, the issue of closed-loop stability under an RL-method is still not fully addressed. Since the critic delivers merely an approximation to the value function of the corresponding infinite-horizon problem, no guarantee can be given in general as to whether the actor’s actions stabilize the system. Different approaches to this issue exist. The current work offers a particular one, which, starting with a (not necessarily smooth) control Lyapunov function (CLF), derives an online RL-scheme in such a way that practical semi-global stability property of the closed-loop can be established. The approach logically continues the work of the authors on parameterized controllers and Lyapunov-like constraints for RL, whereas the CLF now appears merely in one of the constraints of the control scheme. The analysis of the closed-loop behavior is done in a sample-and-hold (SH) manner thus offering a certain insight into the digital realization. The case study with a non-holonomic integrator shows the capabilities of the derived method to optimize the given cost function compared to a nominal stabilizing controller.
Abstract Tillage is one of the most important and expensive operations in agriculture. A crucial ... more Abstract Tillage is one of the most important and expensive operations in agriculture. A crucial measure in soil protection and tillage quality is assessment of soil roughness. The objective of the current study is to utilize spectral analysis while investigating the effects of tillage with a heavy duty tine at work depths of 5, 10, 15, 20 cm and speeds of 4, 7, 10, 13 km h −1 on the soil cross-section. In order to pick a practically appropriate longitudinal step size, a high-resolution measurement was performed. The conditions of this measurement included 4 km h −1 speed with 10 cm working depth. Spectral analysis of selected soil cross-section parameters was performed to assess a cutting frequency below which the significant spectral contents were located. Such an analysis led to the conclusion on a practical longitudinal resolution of 5 cm. This measure is more reliable than picking several cross-section measurements at random. Influence of different tillage depths and speeds on selected soil cross-section parameters was investigated. It was observed that the volumes of disturbed and loosened soil were mainly influenced by the tillage depth while the furrow was more affected by the work speed.
Abstract Reinforcement learning (RL) in the context of control systems offers wide possibilities ... more Abstract Reinforcement learning (RL) in the context of control systems offers wide possibilities of controller adaptation. Given an infinite-horizon cost function, the so-called critic of RL approximates it with a neural net and sends this information to the controller (called “actor”). However, the issue of closed-loop stability under an RL-method is still not fully addressed. Since the critic delivers merely an approximation to the value function of the corresponding infinite-horizon problem, no guarantee can be given in general as to whether the actor’s actions stabilize the system. Different approaches to this issue exist. The current work offers a particular one, which, starting with a (not necessarily smooth) control Lyapunov function (CLF), derives an online RL-scheme in such a way that practical semi-global stability property of the closed-loop can be established. The approach logically continues the work of the authors on parameterized controllers and Lyapunov-like constraints for RL, whereas the CLF now appears merely in one of the constraints of the control scheme. The analysis of the closed-loop behavior is done in a sample-and-hold (SH) manner thus offering a certain insight into the digital realization. The case study with a non-holonomic integrator shows the capabilities of the derived method to optimize the given cost function compared to a nominal stabilizing controller.
Abstract Tillage is one of the most important and expensive operations in agriculture. A crucial ... more Abstract Tillage is one of the most important and expensive operations in agriculture. A crucial measure in soil protection and tillage quality is assessment of soil roughness. The objective of the current study is to utilize spectral analysis while investigating the effects of tillage with a heavy duty tine at work depths of 5, 10, 15, 20 cm and speeds of 4, 7, 10, 13 km h −1 on the soil cross-section. In order to pick a practically appropriate longitudinal step size, a high-resolution measurement was performed. The conditions of this measurement included 4 km h −1 speed with 10 cm working depth. Spectral analysis of selected soil cross-section parameters was performed to assess a cutting frequency below which the significant spectral contents were located. Such an analysis led to the conclusion on a practical longitudinal resolution of 5 cm. This measure is more reliable than picking several cross-section measurements at random. Influence of different tillage depths and speeds on selected soil cross-section parameters was investigated. It was observed that the volumes of disturbed and loosened soil were mainly influenced by the tillage depth while the furrow was more affected by the work speed.
Uploads