A Recurrent Neural Network for Identifying Multiple Chaotic Systems

Abstract

This paper presents a First-Order Recurrent Neural Network activated by a wavelet function, in particular a Morlet wavelet, with a fixed set of parameters and capable of identifying multiple chaotic systems. By maintaining a fixed structure for the neural network and using the same activation function, the network can successfully identify the three state variables of several different chaotic systems, including the Chua, PWL-Rössler, Anishchenko–Astakhov, Álvarez-Curiel, Aizawa, and Rucklidge models. The performance of this approach was validated by numerical simulations in which the accuracy of the state estimation was evaluated using the Mean Square Error (MSE) and the coefficient of determination ($r^2$), which indicates how well the neural network identifies the behavior of the individual oscillators. In contrast to the methods found in the literature, where a neural network is optimized to identify a single system and its application to another model requires recalibration of the neural algorithm parameters, the proposed model uses a fixed set of parameters to efficiently identify seven chaotic systems. These results build on previously published work by the authors and advance the development of robust and generic neural network structures for the identification of multiple chaotic oscillators.

1. Introduction

Almost all types of phenomena can be described by differential equations that make it possible to capture the studied entity’s changes over time. In nature, these behaviors are usually nonlinear, leading to the appearance of hybrid systems in which linear components coexist with nonlinear elements, leading to more complex phenomena and interesting geometric patterns associated with chaotic behavior [1]. A vital feature of this type of dynamics is a high sensitivity to initial conditions, which implies a rapid divergence between states at a virtually identical starting point, leading to poor predictions [2,3]. However, various behaviors in nature (e.g., the weather), in the human body (neuronal transmission), in mechanical systems, and in chemical reactions, to name a few, exhibit chaotic properties. Due to their universality, they are a recurring theme in scientific research [4,5,6,7].

Machine learning-based methods are being increasingly explored for their accurate processing, cataloging, and classification of large amounts of information. This type of deep learning algorithm has its roots in the early 1980s in the work of John Hopfield, who presented a model capable of storing and retrieving patterns of information [8]. Since then, the research and the development of more robust and efficient algorithms have intensified, and they are currently used to solve ordinal or temporal problems, such as language translation, speech recognition, and image captioning, among others.

Since chaotic systems and their high sensitivity to initial conditions permeate almost all areas of nature, predicting and identifying their trajectories over time is not trivial; this is why different approaches have been used in designing and using neural algorithms, yielding extensive results [9]. As feature examples, in [10] the authors compared three techniques for predicting chaotic time series (artificial neural networks, adaptive neuro-fuzzy inference systems, and least squares support vector machines) to determine the best in the short-term estimation and prediction of temporal behaviors with different Lyapunov exponents. In [11] an echo state network to improve the prediction horizon of chaotic behaviors was discussed, where the system values were optimized using metaheuristics. In addition, an improvement, on the previously mentioned paper, in the prediction horizon and a reduction in the number of neurons used was reported in [12]. Finally, in [13], the authors used Recurrent Neural Networks (RNN) and Deep Recurrent Neural Networks (DRNN) to estimate the states of chaotic systems with optimal results, in terms of loss and accuracy.

An essential characteristic of these RNN algorithms is that their nucleus is based on two main fronts: On the one hand are the algorithms that are structured from pre-training based on epochs, as presented in [13,14]. These epochs relate to the number of times the data are processed through the algorithm to adjust the weight parameters and improve the performance and convergence to a network solution. On the other hand are the algorithms that perform their training and identification online while the system to be studied is evolving [15,16]. Therefore, the weight parameter estimation is adjusted online due to training processes, such as the filtered error algorithm [17] and the Extended Kalman Filter (EKF) [18].

In particular, regarding the online algorithms, Recurrent Wavelet First-Order Neural Networks (RWFONNs) have been mainly used to implement neural control of electrical machines and emulated energy storage systems, to identify and control electrical systems, robotic manipulators, and crewless aerial vehicles [19,20], and for the identification and control of multi-stable systems [21]. Magallón and coworkers, in turn, have presented an RWFONN that can identify systems based on the jerk equation (an Unstable Dissipative System of type I (UDS I) [22] and a memristive system without equilibrium points [23]), where the values and structure in the network are the same for both systems [17]. In this work, the authors identify a Mean Square Error between the time series that is practically zero. One of the main advantages of this technique is that the network can recognize complex trajectories as the systems evolve; this is possible because the training and validation processes are online, unlike most neural algorithms. In addition, the authors have recently presented a real-time identification of chaotic trajectories using an RWFONN and a Field-Programmable Analog Array (FPAA), which physically implements the chaotic dynamics of a third-order jerky system with reconfigurable analog electronics [24].

Considering that the development of algorithms based on neural networks seems to require a constant calculation of gains, the development of new identification structures, and the optimization of parameters, this work extends the results shown in [17] for the development of an online general-purpose neural network, addressing the question: Is it possible to have a global neural network that can identify different chaotic systems without changing the neural algorithm (topology and/or hyperparameters)? The identification presented here by the studied network has Mean Square Error values of practically zero in all the studied cases. This measure is complemented by the coefficient of determination ($r^2$), which indicates the neural network’s ability to reproduce the observed phenomenon. The results contribute to developing robust structures capable of identifying multiple chaotic systems with no common topology.

The rest of the article has the following structure: The description of the neural network used and the chaotic systems investigated can be found in Section 2. The methodology used and the results obtained are described in Section 3. The last part of the article deals with the discussions and preliminary conclusions.

2. Preliminaries

2.1. Recurrent Wavelet First-Order Neural Network

There are recent works in which artificial Recurrent Wavelet First-Order Neural Networks (RWFONN) have been used to identify and control different dynamical systems. This is due to their simple structure, good identification capability, and low computational cost [17,21]. The general structure of this kind of RWFONN is given by

$$\begin{array}{c}\hfill {\dot{y}}_j^i=-{\alpha }_j^i{y}_j^i+{\left(w_{jk}^i\right)}^{\top }{\varphi }_{jk}^i,\end{array}$$

(1)

where $y_j^i$ is the state of the i-th neuron, ${\alpha }_j^i>0$ (for $i=1,2,\cdots n$) is part of the underlying network architecture and is determined during the training process, $w_{jk}^i$ is the k-th adjustable synaptic weight connecting the j-th state to the i-th neuron, and ${\varphi }_{jk}^i$ is a Morlet wavelet activation function. The systems are identified online using an RWFONN, where the synaptic weights are adjusted using the filtered error algorithm.

2.2. Filtered Error Algorithm

The identification scheme starts from the differential equation that describes the unknown system (see Equation (2)); then, the identifier can be chosen as described by Equation (3):

$${\dot{\chi }}_j^i=-{\alpha }_j^i{\chi }_j^i+{{\left(w_{jk}^i\right)}^{*}}^{\top }{\varphi }_{jk}^i.$$

(2)

$${\dot{y}}_j^i=-{\alpha }_j^i{y}_j^i+{\left(w_{jk}^i\right)}^{\top }{\varphi }_{jk}^i.$$

(3)

Defining the identification error as ${\xi }_j^{\prime i}=y_j^i-{\chi }_j^i$ leads to

$$\begin{array}{cc}\hfill {\dot{\xi }}_j^{\prime i}=& {\dot{y}}_j^i-{\dot{\chi }}_j^i,\hfill \\ \hfill =& -{\alpha }_j^i{y}_j^i+{\left(w_{jk}^i\right)}^{\top }{\varphi }_{jk}^i-(-{\alpha }_j^i{\chi }_j^i+{{\left(w_{jk}^i\right)}^{*}}^{\top }{\varphi }_{jk}^i),\hfill \\ \hfill =& -{\alpha }_j^i{y}_j^i+{\left(w_{jk}^i\right)}^{\top }{\varphi }_{jk}^i+{\alpha }_j^i{\chi }_j^i-{{\left(w_{jk}^i\right)}^{*}}^{\top }{\varphi }_{jk}^i,\hfill \\ \hfill =& -{\alpha }_j^i(y_j^i-{\chi }_j^i)+(w_{jk}^{i\top }-w_{jk}^{i*\top }){\varphi }_{jk}^i,\hfill \end{array}$$

(4)

where ${\chi }_j^i$ denotes each of the state variables of the oscillators to be identified. The model of the identification error, Equation (4), can be rewritten as

$${\dot{\xi }}_j^{\prime i}=-{\alpha }_j^i{\xi }_j^i+{\tilde{w}}_j^i{\varphi }_{jk}^i,$$

(5)

where ${\tilde{w}}_j^i=w_j^i-w_j^{*i}$. The synaptic weights $w_j^i$ (for $i=1,2,\cdots ,n$) are adjusted according to the learning law defined in [25], also called “filtered error”, described by Equation (6):

$$\begin{array}{c}\hfill {\dot{w}}_j^i=-{\gamma }_j^i{\varphi }_{jk}^i{\xi }_j^{\prime i}.\end{array}$$

(6)

It is worth noting that the synaptic weights of the neural network for identification are bounded; further details are available in [17].

Theorem 1.

Consider the RWFONN model whose weights are adjustable according to Equation (6) for each $i=1,2,\cdots n$, so that:

• ${\mathit{\xi }}_j^{\prime i}$, ${\mathit{w}}_j^i$∈${\mathcal{L}}_{\infty }$ ($i.e.,{\xi }_j^{\prime i}$ and $w_j^i$ are uniformly bounded);
• ${lim}_{t\to \infty }{\xi }_i\left(t\right)=0.$

Proof. 

The proof of Theorem 1 is available in [25], and is also developed by the authors in [17]. □

Theorem 2.

Suppose that any system of

Table 1. Equations of the chaotic systems analyzed in this work. The attractors of each of the systems are shown in Figure 1.

System	Equations
Multiscroll	$\begin{array}{c}\dot{x}=y,\\ \dot{y}=z,\\ \dot{z}=-a[x+y+z-f\left(x\right)],\\ f\left(x\right)=\left\{\begin{array}{ccc}\hfill -2& \mathrm{if}& x<-1,\hfill \\ \hfill 0& \mathrm{if}& -1\le x<1,\hfill \\ \hfill 2& \mathrm{if}& x\ge 1.\hfill \end{array}\right.\end{array}$
Rucklidge	$\begin{array}{c}\dot{x}=-ax+by-yz,\\ \dot{y}=x,\\ \dot{z}=y^2-z.\end{array}$
A–A	$\begin{array}{c}\dot{x}=ax+y-xz,\\ \dot{y}=-x,\\ \dot{z}=-bz+bf\left(x\right)x^2,\\ f\left(x\right)=\left\{\begin{array}{ccc}\hfill 1& \mathrm{if}& x>0,\hfill \\ \hfill 0& \mathrm{if}& x\le 0.\hfill \end{array}\right.\end{array}$
A–C	$\begin{array}{c}\dot{x}=y,\\ \dot{y}=-x-2ay+bf\left(z\right),\\ \dot{z}=-cx,\\ f\left(z\right)=z(z^2-1).\end{array}$
Chua	$\begin{array}{c}\dot{x}=c[y-x-f\left(x\right)],\\ \dot{y}=x-y+z,\\ \dot{z}=-py,\\ f\left(x\right)=bx+\frac{1}{2}(a-b)\left(\right|x+1|-|x-1\left|\right).\end{array}$
Rössler-PWL	$\begin{array}{c}\dot{x}=-ax-by-z,\\ \dot{y}=x+cy-py,\\ \dot{z}=qf\left(x\right)-z,\\ f\left(x\right)=\left\{\begin{array}{ccc}\hfill 0& \mathrm{if}& x\le 3,\hfill \\ \hfill x-3& \mathrm{if}& x>3.\hfill \end{array}\right.\end{array}$
Aizawa	$\begin{array}{c}\dot{x}=x(z-b)-py,\\ \dot{y}=px+y(z-b),\\ \dot{z}=c+az-\frac{z^3}{3}-(x^2+y^2)(1+qz)+rz{x}^3.\end{array}$

and model (1) is initially at the same state $\mathbf{x}\left(0\right)=y\left(0\right)$; then, for any $ϵ>0$ and any finite $T>0$ an integer L and a matrix $\mathbf{w}\ast \in {\mathbb{R}}^{L\times n}$ exist, such that the state $y\left(t\right)$ of the RWFONN model (1) and the weight values $\mathbf{w}=\mathbf{w}\ast$ satisfy:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{0\le t\le T}{\sup }\left|\mathbf{x}\left(t\right)-\mathit{y}\left(t\right)\right|\le ϵ.\end{array}\end{array}$$

Then, using the Bellman–Gronwall Lemma [26] we ascertain that the identification error $\mathit{\xi }=\mathbf{x}\mathbf{-}\mathit{y}$ is bounded by:

$$\begin{array}{c}\hfill ∥\mathit{\xi }∥\le \frac{ϵ}{2}.\end{array}$$

(7)

Proof. 

See reference [27]. □

2.3. Chaotic Systems

The chaotic oscillators investigated in this work are as follows:

• Multiscroll: System described by a monoparametric family (Figure 1a,b), used to model diffusion processes [28,29].
• Rucklidge: This system models double convection processes where the motion is confined to coils resembling convection in an applied vertical magnetic field and a slightly rotating fluid layer [30] (see Figure 1c,d).
• A–C: The Álvarez–Curiel system is derived from classical control schemes with typical nonlinearities where chaotic behavior emerges [31] (shown in Figure 1e,f).
• A–A: The Anishchenko–Astakhov system (shown in Figure 1g,h) presents a modified oscillator with inertial nonlinearity that describes a radiophysical generator via three coupled equations and is considered a fundamental model of deterministic chaos [32].
• Chua: One of the classical attractors of chaos theory, composed of a Piecewise (PW) function [33] (Figure 1i,j).
• Rössler-PWL: A piecewise linear version of the system introduced by the German biochemistry Otto Rössler, presented in [34] (Figure 1k,l).
• Aizawa: A chaotic three-dimensional system, whose dynamics are described in [35,36] (Figure 1m,n).

The equations of the oscillators mentioned above are described in Table 1, and the values for the parameters used in this work are listed in

Table 2. Parameters of the chaotic systems described in Table 1.

System	Parameter
Multiscroll	$a=0.45$
Rucklidge	$a=2,b=6.7$
A–A	$a=1.2,b=0.5$
A–C	$a=0.5,b=1.4,c=1$
Chua	$a=-1.27,b=-0.68,c=10,p=14.87$
Rössler-PWL	$a=0.05,b=0.5,c=0.133,p=0.02,q=15,$
Aizawa	$a=0.95,b=0.7,c=0.6,p=3.5,q=0.25,r=0.1$

.

3. Methodology and Results

Considering the neural network described by Equation (3) and the filtered error model of Equation (6), this paper used the neural network structure (RWFONN) described by Equation (8), under the assumption that $i=j=k$ (for $i=1,2,3$):

$$\begin{array}{c}\dot{y_1}=-{\alpha }_1{y}_1+{\delta }_1{w}_1{\varphi }_1\left({\chi }_1\right)+y_2,\hfill \\ \dot{y_2}=-{\alpha }_2{y}_2+{\delta }_2{w}_2{\varphi }_2\left({\chi }_2\right)+y_3,\hfill \\ \dot{y_3}=-{\alpha }_3{y}_3+{\delta }_3{w}_3{\varphi }_3\left({\chi }_3\right)+u,\hfill \end{array}$$

(8)

where u is the same control input of the system to be identified. In this case, the functions $f\left(x\right)$, ${\alpha }_i$, and ${\delta }_i$, are parameters of the network described in

Table 3. Parameters of the neural network described by Equations (8)–(10) for the identification of chaotic systems.

Parameter	Value
h	$0.01$
${\alpha }_1$ = ${\alpha }_2$ = ${\alpha }_3$	5
${\delta }_1$ = ${\delta }_2$ = ${\delta }_3$	5
${\beta }_1$ = ${\beta }_2$ = ${\beta }_3$	100
${\lambda }_1$ = ${\lambda }_2$ = ${\lambda }_3$	$0.01$
${\gamma }_1$ = ${\gamma }_2$ = ${\gamma }_3$	5000

(see [17] for further details of the procedure for determining the neural network values), and ${\chi }_1=x$, ${\chi }_2=y,$ and ${\chi }_3=z$. In addition, derived from Equation (6), the algorithm for filtered errors is defined as follows:

$$\begin{array}{c}\dot{w_1}=-{\gamma }_1{\varphi }_1{\xi }_1,\hfill \\ \dot{w_2}=-{\gamma }_2{\varphi }_2{\xi }_2,\hfill \\ \dot{w_3}=-{\gamma }_3{\varphi }_3{\xi }_3,\mathrm{being}{\xi }_i={\chi }_i-y_i.\hfill \end{array}$$

(9)

Finally, the Morlet wavelet activation function implemented by the neuron has the following form:

$$\begin{array}{c}{\varphi }_1\left({\chi }_1\right)=exp\left(\frac{-{\chi }_1^2}{{\beta }_1}\right)cos\left({\lambda }_1{\chi }_1\right),\hfill \\ {\varphi }_2\left({\chi }_2\right)=exp\left(\frac{-{\chi }_2^2}{{\beta }_2}\right)cos\left({\lambda }_2{\chi }_2\right),\hfill \\ {\varphi }_3\left({\chi }_3\right)=exp\left(\frac{-{\chi }_3^2}{{\beta }_3}\right)cos\left({\lambda }_3{\chi }_3\right),\hfill \end{array}$$

(10)

where $\beta$ and $\lambda$ are expansion and dilation terms.

Remark 1.

In [17], it is stated that the input “u” should contain information of the system, marked as $u=f\left(x\right)$ in Equation (8), where $f\left(x\right)$ is a function of the system to be identified. However, we found that a more straightforward input can be used, since more of the benchmark systems utilized do not present an input function. To illustrate this, during the identification process of the Aizawa and Rucklidge models (see Table 3), we considered $u=x$. The control input is $u=f\left(x\right)$ to identify the remaining systems. Although the use of $u=y$ or even $u=z$ can be studied, such cases are not considered in the rest of this work.

3.1. Metrics Used for the Identification Quantification

The degree of identification between the variables of the neural network ($y_1,y_2,y_3$) and the state variables of the chaotic oscillators ($x,y,z$) was performed using the following two statistical metrics:

• The Mean Square Error (MSE): Also known as Mean Squared Deviation (MSD), it measures the average of the squares of the errors between the estimated values and the actual values of any phenomenon. MSE is a risk function that corresponds to the expected value of the squared error loss and is defined as:

  $$\begin{array}{c}MSE=\sqrt{{(x-y_1)}^2+{(y-y_2)}^2+{(z-y_3)}^2}.\end{array}$$

  (11)
• $r^2$ (r-squared): This value quantifies the amount of information that a model can reproduce from the original system, i.e., it indicates how well the neuron reproduces the behavior of the analyzed oscillator and is defined by:

  $$\begin{array}{c}{r}^2={\left({\displaystyle \frac{{\sigma }_{xy}}{{\sigma }_x{\sigma }_y}}\right)}^2,\end{array}$$

  (12)

  where ${\sigma }_{xy}$ is the covariance between the original data ($x,y,z$) and the information obtained by the neural system ($y_1,y_2,y_3$), ${\sigma }_x$ is the standard deviation of the time series from the chaotic system, and ${\sigma }_y$ is the standard deviation of the temporal behavior of the neuron. The closer the value of $r^2$ is to 1, the greater the amount of information the neuron reproduces from the analyzed oscillator.

3.2. Results

The identification of the chaotic systems was performed as follows:

• The oscillator to be identified was “connected” to the RWFONN considering ${\chi }_1=x$, ${\chi }_2=y,$ and ${\chi }_3=z$, as it is described in Equation (4) for the same structure and values.
• The neural network identified the three state variables of the oscillator.
• A time series was obtained from the neural network that was the same size as that injected into the system by each state variable.
• The MSE and $r^2$ were calculated for each oscillator, comparing a time series of each state variable of the oscillator with those estimated by the neural network. For the comparison, the transient state of the neural network was not removed.

Remark 2.

The expected values in the identification metrics were MSE $\simeq 0$ and $r^2\simeq 1$; this guaranteed that the neuron could correctly identify the states of the connected chaotic system.

All the results in this paper were calculated with a fourth-order Runge–Kutta integrator (RK4) implemented in MatLab 2024 and a step size of $\tau =0.01$, with a total observation time of 1500 s, which implied a total of 150,000 samples in each time series.

The initial conditions from the basin of attraction of each chaotic system were used. The neural network and the filtered error system were initialized with values similar but not identical to the oscillator under study.

Figure 2 shows a simulation time window for the identification of each of the analyzed systems. The x-variable time series of the oscillator is shown in blue, while the $y_1$-series estimated by the neural network is shown in red. To clarify: The variable $y_1$ of the neural network estimated the variable x of the oscillator connected to the RWFONN. Different initial conditions were used to initialize the oscillator and the neural network; see

Table 4. Initial conditions used for the identification of the chaotic systems.

System	Initial Conditions
	$\mathit{x}$	${\mathit{y}}_{\mathit{1}}$	$\mathit{y}$	${\mathit{y}}_{\mathit{2}}$	$\mathit{z}$	${\mathit{y}}_{\mathit{3}}$
Multiscroll	$0.2$	$0.1$	$0.1$	$-0.2$	$-0.1$	$-0.1$
Rucklidge	$0.2$	$0.1$	$0.1$	$-0.2$	$-0.1$	$-0.1$
A–A	$-0.1$	$0.1$	$0.1$	$0.2$	$-0.1$	$-0.1$
A–C	$0.1$	$0.1$	$0.1$	$0.2$	$-0.1$	$-0.1$
Chua	$0.1$	$0.1$	$0.1$	$0.9$	$-0.1$	$-0.1$
Rössler-PWL	$2.5$	$-0.1$	$-0.1$	$2.0$	$0.1$	$0.1$
Aizawa	$0.1$	$0.1$	$0.1$	$-0.2$	$-0.1$	$-0.1$

. Theorem 2 states that the filtered error bounds are calculated when the systems are initialized with identical initial conditions. However, the performance tests were performed with different initial conditions in the oscillator and neural network to ensure correct identification (without bias) and robustness to disturbances and noise. Using identical initial conditions would only shorten the identification time, resulting in a shorter transient state. The same figure shows how the dynamics of the neural network stabilized in less than a second and perfectly identified the states of the oscillator. The results based on the MSE value for identifying the oscillators described in Table 2 are shown in

Table 5. Values obtained in the MSE for identifying the chaotic systems described in Table 1 and Table 2.

System	MSE
	$\mathit{x}-{\mathit{y}}_{\mathit{1}}$	$\mathit{y}-{\mathit{y}}_{\mathit{2}}$	$\mathit{z}-{\mathit{y}}_{\mathit{3}}$
Multiscroll	$6.157\times {10}^{-7}$	$4.7\times {10}^{-9}$	$9.02\times {10}^{-8}$
Rucklidge	0	0	$3\times {10}^{-3}$
A–A	$5.932\times {10}^{-7}$	$1.9\times {10}^{-8}$	$8.3\times {10}^{-9}$
A–C	$3.736\times {10}^{-5}$	$1\times {10}^{-7}$	$5.825\times {10}^{-5}$
Chua	$4.48\times {10}^{-7}$	$2.8\times {10}^{-8}$	$1.008\times {10}^{-6}$
Rössler-PWL	$6.756\times {10}^{-7}$	$1.732\times {10}^{-7}$	$1.52\times {10}^{-8}$
Aizawa	$8.982\times {10}^{-7}$	$2.606\times {10}^{-7}$	$1.95\times {10}^{-8}$

.

Table 6. Convergence times of neuronal identification of chaotic systems.

System	Convergence Times
	$\mathit{x}-{\mathit{y}}_{\mathit{1}}$	$\mathit{y}-{\mathit{y}}_{\mathit{2}}$	$\mathit{z}-{\mathit{y}}_{\mathit{3}}$
Multiscroll	$0.4$ s	$0.2$ s	$0.3$ s
Rucklidge	$0.3$ s	$0.25$ s	$0.4$ s
A–A	$0.45$ s	$0.35$ s	$0.6$ s
A–C	$0.55$ s	$0.4$ s	$0.6$ s
Chua	$0.25$ s	$0.35$ s	$0.4$ s
Rössler-PWL	$0.55$ s	$0.2$ s	$0.25$ s
Aizawa	$0.55$ s	$0.4$ s	$0.5$ s

shows the convergence times of the neural identification of each variable of the chaotic systems. On the other hand, the values of the determination coefficients are listed in

Table 7. Values obtained in $r^2$ for identifying the chaotic systems described in Table 1 and Table 2.

System	${\mathit{r}}^2$
	$\mathit{x}-{\mathit{y}}_{\mathit{1}}$	$\mathit{y}-{\mathit{y}}_{\mathit{2}}$	$\mathit{z}-{\mathit{y}}_{\mathit{3}}$
Multiscroll	1	1	1
Rucklidge	1	1	$0.9997$
A–A	1	1	1
A–C	1	1	1
Chua	1	1	1
Rössler-PWL	1	1	1
Aizawa	1	1	1

.

4. Discussion

The presented online Recurrent Neural Network could identify all the proposed chaotic systems. First, values in the Mean Square Error very close to zero were obtained (as Table 5 depicts). In the case of the Rucklidge oscillator—the one that presented “the worst” values, with a value in the MSE for the state variable z of $3\times {10}^{-3}$—it was the only one that presented values of absolute zero in the identification of the other two state variables. A similar case occurred when the identification properties of the neural network were evaluated under the determination coefficient (Table 7). All the oscillators, except for the Rucklidge case, had values equal to 1, indicating that the neural network used can fully reproduce the behavior of each state variable of the oscillators studied. Although the Rucklidge system was not perfectly identified, the values obtained met our criteria as they were close enough to the objective value (MSE ≃ 0 and $r^2\simeq 1$ for $\Delta =0.001$), which is why they were considered a satisfactory identification of the system.

On the other hand, the time the neural network requires to identify the chaotic system unambiguously was very short. A correct identification was possible in less than one second in all cases, as shown in Table 6 and Figure 2. Remark 2 mentions that the neural network and the oscillator were initialized with different but close initial conditions. It is important to emphasize that using completely different initial conditions does not decrease the identification capacity of the neural network but increases the amount of time the network stabilizes the identification. As with any other dynamical system, initializing with values far from the resulting attractor will increase the transient state. Still, it will not affect the identification properties of the neural algorithm.

Although the neural network studied was robust enough to identify nine oscillators (those studied in this work plus those published in [17]), its applicability is not universal. Tests were carried out with the Lorenz (MSE = [$0.4494$, $3.3921$, $325.4273$]; $r^2$ = [$0.9887$, $0.9309$, 0]) and Rössler (MSE = [$0.0001$, $0.0002$, $1.7080$]; $r^2$ = [1, 1, $0.7648$]) systems (see Figure 3), where satisfactory state identification could not be achieved, regardless of which state was fed back to the neural network. It is worth noting that this study was conducted in the same way as presented in this paper, with the same structure and values as those described here for the neural network.

Considering the results obtained and the methodology proposed in [37], where a system of equations based on polynomials can be reconstructed by sampling time series to produce a dynamic behavior similar to that of the original system, it would be worthwhile to consider both the inclusion of this methodology in the identification of systems by neural networks and a possible performance evaluation by calculating indices such as MSE and $r^2$ for the behaviors identified by the neural network as well as the behaviors generated by the polynomial-based system of equations. However, it should be noted that the computational time and complexity between the two methods are different, with RWFONN being more straightforward to implement.

RWFONN vs. RHONN

Since the RWFONN has a unique structure, we considered it valuable to compare the performance of the proposed neural network with a neural network that uses the same type of supervised training for system identification. The structure of a Recurrent High-Order Neural Network (RHONN) is similar to the structure presented by Jurado et al. (see Equation (2.1) in [20]), adjusted with the same hyperparameters given for the RWFONN in Table 3. Contrary to expectations, the results show that our neural network outperformed the RHONN in precision (highlighted in blue in

Table 8. MSE values for the identification of the chaotic systems described in Table 1 using the RWFONN from this work described by the Equations (8)–(10) and the Recurrent High-Order Neural Network (RHONN) presented in [20].

System	MSE-RWFONN			MSE-RHONN
	$\mathit{x}-{\mathit{y}}_{\mathit{1}}$	$\mathit{y}-{\mathit{y}}_{\mathit{2}}$	$\mathit{z}-{\mathit{y}}_{\mathit{3}}$	$\mathit{x}-{\mathit{y}}_{\mathit{1}}$	$\mathit{y}-{\mathit{y}}_{\mathit{2}}$	$\mathit{z}-{\mathit{y}}_{\mathit{3}}$
Multiscroll	$6.157\times {10}^{-7}$	$4.7\times {10}^{-9}$	$9.02\times {10}^{-8}$	$8.177\times {10}^{-8}$	$8.363\times {10}^{-6}$	$0.0041$
Rucklidge	0	0	$3\times {10}^{-3}$	$0.0068$	$0.0061$	$0.2355$
A–A	$5.932\times {10}^{-7}$	$1.9\times {10}^{-8}$	$8.3\times {10}^{-9}$	$9.4250\times {10}^{-7}$	$0.0021$	$0.0104$
A–C	$3.736\times {10}^{-5}$	$1\times {10}^{-7}$	$5.825\times {10}^{-5}$	$9.6063\times {10}^{-7}$	$0.0011$	$0.0148$
Chua	$4.48\times {10}^{-7}$	$2.8\times {10}^{-8}$	$1.008\times {10}^{-6}$	$8.4028\times {10}^{-4}$	$0.0098$	$0.0108$
Rössler-PWL	$6.756\times {10}^{-7}$	$1.732\times {10}^{-7}$	$1.52\times {10}^{-8}$	$0.0060$	$7.8632\times {10}^{-4}$	$1.8730\times {10}^{-4}$
Aizawa	$8.982\times {10}^{-7}$	$2.606\times {10}^{-7}$	$1.95\times {10}^{-8}$	$0.0066$	$0.0053$	$0.0436$

), reflected in lower MSE values for most of the tests performed. This observation suggests that despite the greater complexity of higher-order networks and their theoretical ability to capture complicated dynamics, the RWFONN studied here is more effective for this particular type of task.

5. Conclusions

In this paper, the applicability of the First-Order Neural Network proposed by [17], with online training and a Morlet wavelet activation function, was tested for the identification of multiple chaotic systems under a fixed set of parameters in the neural algorithm. The state variables of seven chaotic oscillators were successfully identified using a fixed structure and their synaptic weights for the RWFONN. It should be emphasized that the dynamical systems studied in this work have no common structure in the vector field, which guarantees the effectiveness of the neural network and confirms that the approach discussed here is robust enough to identify the state variables of different dynamical systems adequately. The Mean Square Error (MSE) and the coefficient of determination ($r^2$) were used as measurement tools to quantify the performance of the neural network. In all cases, the errors provided were very close to zero, with r-squared values very close to one (see Table 5 and Table 7). The neural algorithm used provided a good identification of the analyzed chaotic systems in a very short time. It converged to identify the three state variables of each analyzed oscillator in less than a second.

However, when extending the test bench of the proposed neural network and its synaptic values, we were able to find two chaotic systems (the Lorenz system and the Rössler model) that the neural algorithm used could not identify correctly (see Equations (8)–(10)), with the same values as described in Table 3. Nevertheless, the authors identified two questions that need to be addressed in future work: (i) What properties do the systems reported here have in common so that the neural network can adequately identify them? (ii) Is there a combination of values in the synaptic weights of the studied network that allows correct identification of the oscillators described in Table 1 as well as the Lorenz and Rössler models?

The results obtained and reported here contribute to the study and development of neural algorithms for state identification that are robust enough for their application in multiple systems and allow a structure similar to a “plug-and-play”, where the parameters and network configurations remain static for each oscillator studied. Finally, this work contributes to designing neural controllers for chaotic dynamical systems by using an RWFONN, due to its simple structure and low computational cost.