1. Introduction
Almost all types of phenomena can be described by differential equations that make it possible to capture the studied entity’s changes over time. In nature, these behaviors are usually nonlinear, leading to the appearance of hybrid systems in which linear components coexist with nonlinear elements, leading to more complex phenomena and interesting geometric patterns associated with chaotic behavior [
1]. A vital feature of this type of dynamics is a high sensitivity to initial conditions, which implies a rapid divergence between states at a virtually identical starting point, leading to poor predictions [
2,
3]. However, various behaviors in nature (e.g., the weather), in the human body (neuronal transmission), in mechanical systems, and in chemical reactions, to name a few, exhibit chaotic properties. Due to their universality, they are a recurring theme in scientific research [
4,
5,
6,
7].
Machine learning-based methods are being increasingly explored for their accurate processing, cataloging, and classification of large amounts of information. This type of deep learning algorithm has its roots in the early 1980s in the work of John Hopfield, who presented a model capable of storing and retrieving patterns of information [
8]. Since then, the research and the development of more robust and efficient algorithms have intensified, and they are currently used to solve ordinal or temporal problems, such as language translation, speech recognition, and image captioning, among others.
Since chaotic systems and their high sensitivity to initial conditions permeate almost all areas of nature, predicting and identifying their trajectories over time is not trivial; this is why different approaches have been used in designing and using neural algorithms, yielding extensive results [
9]. As feature examples, in [
10] the authors compared three techniques for predicting chaotic time series (artificial neural networks, adaptive neuro-fuzzy inference systems, and least squares support vector machines) to determine the best in the short-term estimation and prediction of temporal behaviors with different Lyapunov exponents. In [
11] an echo state network to improve the prediction horizon of chaotic behaviors was discussed, where the system values were optimized using metaheuristics. In addition, an improvement, on the previously mentioned paper, in the prediction horizon and a reduction in the number of neurons used was reported in [
12]. Finally, in [
13], the authors used Recurrent Neural Networks (RNN) and Deep Recurrent Neural Networks (DRNN) to estimate the states of chaotic systems with optimal results, in terms of loss and accuracy.
An essential characteristic of these RNN algorithms is that their nucleus is based on two main fronts: On the one hand are the algorithms that are structured from pre-training based on epochs, as presented in [
13,
14]. These epochs relate to the number of times the data are processed through the algorithm to adjust the weight parameters and improve the performance and convergence to a network solution. On the other hand are the algorithms that perform their training and identification online while the system to be studied is evolving [
15,
16]. Therefore, the weight parameter estimation is adjusted online due to training processes, such as the filtered error algorithm [
17] and the Extended Kalman Filter (EKF) [
18].
In particular, regarding the online algorithms, Recurrent Wavelet First-Order Neural Networks (RWFONNs) have been mainly used to implement neural control of electrical machines and emulated energy storage systems, to identify and control electrical systems, robotic manipulators, and crewless aerial vehicles [
19,
20], and for the identification and control of multi-stable systems [
21]. Magallón and coworkers, in turn, have presented an RWFONN that can identify systems based on the jerk equation (an Unstable Dissipative System of type I (UDS I) [
22] and a memristive system without equilibrium points [
23]), where the values and structure in the network are the same for both systems [
17]. In this work, the authors identify a Mean Square Error between the time series that is practically zero. One of the main advantages of this technique is that the network can recognize complex trajectories as the systems evolve; this is possible because the training and validation processes are online, unlike most neural algorithms. In addition, the authors have recently presented a real-time identification of chaotic trajectories using an RWFONN and a Field-Programmable Analog Array (FPAA), which physically implements the chaotic dynamics of a third-order jerky system with reconfigurable analog electronics [
24].
Considering that the development of algorithms based on neural networks seems to require a constant calculation of gains, the development of new identification structures, and the optimization of parameters, this work extends the results shown in [
17] for the development of an online general-purpose neural network, addressing the question: Is it possible to have a global neural network that can identify different chaotic systems without changing the neural algorithm (topology and/or hyperparameters)? The identification presented here by the studied network has Mean Square Error values of practically zero in all the studied cases. This measure is complemented by the coefficient of determination (
), which indicates the neural network’s ability to reproduce the observed phenomenon. The results contribute to developing robust structures capable of identifying multiple chaotic systems with no common topology.
The rest of the article has the following structure: The description of the neural network used and the chaotic systems investigated can be found in
Section 2. The methodology used and the results obtained are described in
Section 3. The last part of the article deals with the discussions and preliminary conclusions.
3. Methodology and Results
Considering the neural network described by Equation (
3) and the filtered error model of Equation (
6), this paper used the neural network structure (RWFONN) described by Equation (
8), under the assumption that
(for
):
where
u is the same control input of the system to be identified. In this case, the functions
,
, and
, are parameters of the network described in
Table 3 (see [
17] for further details of the procedure for determining the neural network values), and
,
and
. In addition, derived from Equation (
6), the algorithm for filtered errors is defined as follows:
Finally, the Morlet wavelet activation function implemented by the neuron has the following form:
where
and
are expansion and dilation terms.
Remark 1. In [17], it is stated that the input “u” should contain information of the system, marked as in Equation (8), where is a function of the system to be identified. However, we found that a more straightforward input can be used, since more of the benchmark systems utilized do not present an input function. To illustrate this, during the identification process of the Aizawa and Rucklidge models (see Table 3), we considered . The control input is to identify the remaining systems. Although the use of or even can be studied, such cases are not considered in the rest of this work. 3.1. Metrics Used for the Identification Quantification
The degree of identification between the variables of the neural network () and the state variables of the chaotic oscillators () was performed using the following two statistical metrics:
The Mean Square Error (MSE): Also known as Mean Squared Deviation (MSD), it measures the average of the squares of the errors between the estimated values and the actual values of any phenomenon. MSE is a risk function that corresponds to the expected value of the squared error loss and is defined as:
(r-squared): This value quantifies the amount of information that a model can reproduce from the original system, i.e., it indicates how well the neuron reproduces the behavior of the analyzed oscillator and is defined by:
where
is the covariance between the original data (
) and the information obtained by the neural system (
),
is the standard deviation of the time series from the chaotic system, and
is the standard deviation of the temporal behavior of the neuron. The closer the value of
is to 1, the greater the amount of information the neuron reproduces from the analyzed oscillator.
3.2. Results
The identification of the chaotic systems was performed as follows:
The oscillator to be identified was “connected” to the RWFONN considering
,
and
, as it is described in Equation (
4) for the same structure and values.
The neural network identified the three state variables of the oscillator.
A time series was obtained from the neural network that was the same size as that injected into the system by each state variable.
The MSE and were calculated for each oscillator, comparing a time series of each state variable of the oscillator with those estimated by the neural network. For the comparison, the transient state of the neural network was not removed.
Remark 2. The expected values in the identification metrics were MSE and ; this guaranteed that the neuron could correctly identify the states of the connected chaotic system.
All the results in this paper were calculated with a fourth-order Runge–Kutta integrator (RK4) implemented in MatLab 2024 and a step size of , with a total observation time of 1500 s, which implied a total of 150,000 samples in each time series.
The initial conditions from the basin of attraction of each chaotic system were used. The neural network and the filtered error system were initialized with values similar but not identical to the oscillator under study.
Figure 2 shows a simulation time window for the identification of each of the analyzed systems. The
x-variable time series of the oscillator is shown in blue, while the
-series estimated by the neural network is shown in red. To clarify: The variable
of the neural network estimated the variable
x of the oscillator connected to the RWFONN. Different initial conditions were used to initialize the oscillator and the neural network; see
Table 4. Theorem 2 states that the filtered error bounds are calculated when the systems are initialized with identical initial conditions. However, the performance tests were performed with different initial conditions in the oscillator and neural network to ensure correct identification (without bias) and robustness to disturbances and noise. Using identical initial conditions would only shorten the identification time, resulting in a shorter transient state. The same figure shows how the dynamics of the neural network stabilized in less than a second and perfectly identified the states of the oscillator. The results based on the MSE value for identifying the oscillators described in
Table 2 are shown in
Table 5.
Table 6 shows the convergence times of the neural identification of each variable of the chaotic systems. On the other hand, the values of the determination coefficients are listed in
Table 7.
4. Discussion
The presented online Recurrent Neural Network could identify all the proposed chaotic systems. First, values in the Mean Square Error very close to zero were obtained (as
Table 5 depicts). In the case of the Rucklidge oscillator—the one that presented “the worst” values, with a value in the MSE for the state variable
z of
—it was the only one that presented values of absolute zero in the identification of the other two state variables. A similar case occurred when the identification properties of the neural network were evaluated under the determination coefficient (
Table 7). All the oscillators, except for the Rucklidge case, had values equal to 1, indicating that the neural network used can fully reproduce the behavior of each state variable of the oscillators studied. Although the Rucklidge system was not perfectly identified, the values obtained met our criteria as they were close enough to the objective value (MSE ≃ 0 and
for
), which is why they were considered a satisfactory identification of the system.
On the other hand, the time the neural network requires to identify the chaotic system unambiguously was very short. A correct identification was possible in less than one second in all cases, as shown in
Table 6 and
Figure 2. Remark 2 mentions that the neural network and the oscillator were initialized with different but close initial conditions. It is important to emphasize that using completely different initial conditions does not decrease the identification capacity of the neural network but increases the amount of time the network stabilizes the identification. As with any other dynamical system, initializing with values far from the resulting attractor will increase the transient state. Still, it will not affect the identification properties of the neural algorithm.
Although the neural network studied was robust enough to identify nine oscillators (those studied in this work plus those published in [
17]), its applicability is not universal. Tests were carried out with the Lorenz (MSE = [
,
,
];
= [
,
, 0]) and Rössler (MSE = [
,
,
];
= [1, 1,
]) systems (see
Figure 3), where satisfactory state identification could not be achieved, regardless of which state was fed back to the neural network. It is worth noting that this study was conducted in the same way as presented in this paper, with the same structure and values as those described here for the neural network.
Considering the results obtained and the methodology proposed in [
37], where a system of equations based on polynomials can be reconstructed by sampling time series to produce a dynamic behavior similar to that of the original system, it would be worthwhile to consider both the inclusion of this methodology in the identification of systems by neural networks and a possible performance evaluation by calculating indices such as MSE and
for the behaviors identified by the neural network as well as the behaviors generated by the polynomial-based system of equations. However, it should be noted that the computational time and complexity between the two methods are different, with RWFONN being more straightforward to implement.
RWFONN vs. RHONN
Since the RWFONN has a unique structure, we considered it valuable to compare the performance of the proposed neural network with a neural network that uses the same type of supervised training for system identification. The structure of a Recurrent High-Order Neural Network (RHONN) is similar to the structure presented by Jurado et al. (see Equation (2.1) in [
20]), adjusted with the same hyperparameters given for the RWFONN in
Table 3. Contrary to expectations, the results show that our neural network outperformed the RHONN in precision (highlighted in blue in
Table 8), reflected in lower MSE values for most of the tests performed. This observation suggests that despite the greater complexity of higher-order networks and their theoretical ability to capture complicated dynamics, the RWFONN studied here is more effective for this particular type of task.
5. Conclusions
In this paper, the applicability of the First-Order Neural Network proposed by [
17], with online training and a Morlet wavelet activation function, was tested for the identification of multiple chaotic systems under a fixed set of parameters in the neural algorithm. The state variables of seven chaotic oscillators were successfully identified using a fixed structure and their synaptic weights for the RWFONN. It should be emphasized that the dynamical systems studied in this work have no common structure in the vector field, which guarantees the effectiveness of the neural network and confirms that the approach discussed here is robust enough to identify the state variables of different dynamical systems adequately. The Mean Square Error (MSE) and the coefficient of determination (
) were used as measurement tools to quantify the performance of the neural network. In all cases, the errors provided were very close to zero, with r-squared values very close to one (see
Table 5 and
Table 7). The neural algorithm used provided a good identification of the analyzed chaotic systems in a very short time. It converged to identify the three state variables of each analyzed oscillator in less than a second.
However, when extending the test bench of the proposed neural network and its synaptic values, we were able to find two chaotic systems (the Lorenz system and the Rössler model) that the neural algorithm used could not identify correctly (see Equations (
8)–(
10)), with the same values as described in
Table 3. Nevertheless, the authors identified two questions that need to be addressed in future work: (i) What properties do the systems reported here have in common so that the neural network can adequately identify them? (ii) Is there a combination of values in the synaptic weights of the studied network that allows correct identification of the oscillators described in
Table 1 as well as the Lorenz and Rössler models?
The results obtained and reported here contribute to the study and development of neural algorithms for state identification that are robust enough for their application in multiple systems and allow a structure similar to a “plug-and-play”, where the parameters and network configurations remain static for each oscillator studied. Finally, this work contributes to designing neural controllers for chaotic dynamical systems by using an RWFONN, due to its simple structure and low computational cost.