Sustainable Energy Technologies and Assessments 41 (2020) 100802

Contents lists available at ScienceDirect

Sustainable Energy Technologies and Assessments

journal homepage: www.elsevier.com/locate/seta

Original article

An intelligent hybrid model of neuro Wavelet, time series and Recurrent T

Kalman Filter for wind speed forecasting
Hamed H.H. Aly
⁎

Electrical and Computer Engineering Department, Dalhousie University and Electrical Power and Machines Engineering, Zagazig University, Egypt
Mathematics and Statistics Department, Acadia University, Canada

ARTICLE INFO ABSTRACT

Keywords: Wind speed Forecasting is the first step to integrate wind power into the main grid. It is important to improve the
Wind speed forecasting accuracy of wind speed forecasting to improve the load management side and the renewable energy integration.
ANN Due to the chaotic in the wind speed fluctuation the wind speed data forecasting is difficult. Many models are
TS proposed in the literature for wind speed forecasting. This paper is proposing accurate hybrid models for wind
WNN
speed forecasting to improve the overall system accuracy. These hybrid models involve various combinations of
RKF
Wavelet and Artificial Neural Network (WNN and ANN), Time Series (TS) and Recurrent Kalman Filter (RKF).
Three main hybrid models are proposed and tested. From those three models the best model with the highest
performance is the hybrid of WNN, RKF, TS. The order of the techniques used in the hybrid models is very
important. Different combinations with different orders are tested in this stage. Different models are tested with
different techniques order. The proposed work is validated by using different unseen dataset with the proposed
models and prove their effectiveness. All proposed models are accurate, but the best model is a hybrid of WNN,
TS and RKF in sequence.

Introduction was proposed and used to obtain optimal estimation of output power
and wind speed. It showed a new method used for estimating wind
Wind speed output power forecasting is depending on the wind speed and hourly solar using the frequency distribution in shorter time
speed forecasting. The accuracy of the forecasting models used for wind frames. The proposed model was able to generate annual forecasts of
speed is very important to improve the integration of the wind power solar and wind speed [8]. Probabilistic forecasting using two models to
into the grid and the overall system quality. Many works done in the generate the forecasting techniques that are combined by the forest
literature for wind speed forecasting but most of the work done was were studied. The first model was assembled based on probabilistic
depending on using an artificial technique for the wind speed and rarely forecasting using a bootstrapping approach to obtain the prediction for
used hybrid models [1–5]. In this paper, we propose different hybrid a given point that implements the cumulative distribution function. The
models to improve the overall system accuracy; hence improve the second was an analog assembled probabilistic forecasting model that
integration and increase the overall system stability and quality. generated the probabilistic forecasting using a procedure to anticipate a
Different models were proposed in the literature for wind speed point of comparison to obtain a similar expectation [9]. A proposed
forecasting either a hybrid or non-hybrid model. An efficient fore- learning hybrid model for forecasting based on meteorological data was
casting optimization scheme is proposed that shows a depended peri- proposed and showed a linear model called time varying multiple linear
odic vector auto-regressive model by using wind and solar measure- model to obtain dynamic property of data. Genetic algorithm back
ments. Different comparison schemes between existing deterministic propagation neural network was used to understand the nonlinear re-
and unit commitment methods using a technology-clustered interval lationships between different types of data [10,11]. Three models for
technique was proposed [6]. Time series method was used for wind and wind speed forecasting including hybrid model of ARIMA and ANN
solar forecasting based on a regression model to analyze the impact of were used. Heteroscedastic spline regression model and robust spline
renewable energy forecasting error. The wind output power error regression model were used [12]. A group method of data handling
forecasting had more impact than solar forecasting errors [7]. Weibull neural network for short-term forecasting of wind speed and wind
probability for wind speed modeling and wind turbine output power power output to construct the relation between input and output

⁎
Address: Electrical and Computer Engineering Department, Dalhousie University and Electrical Power and Machines Engineering, Zagazig University, Egypt.
E-mail address: hamed.aly@dal.ca.

https://doi.org/10.1016/j.seta.2020.100802
Received 29 January 2020; Received in revised form 18 May 2020; Accepted 25 July 2020
2213-1388/ © 2020 Elsevier Ltd. All rights reserved.
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

variables was proposed [13]. A smart method for wind power predic- Table 1
tion using support vector technique was modeled. This wind power ANN and WNN with different number of hidden layers and neurons.
model is used to forecast for the future at three-hour intervals in steps of Neurons no Hidden layers ANN WNN
fifteen minutes using three wind power generation systems. It was no
considered a practical test with better forecasting than other methods R-Squared nRMSE R-Squared nRMSE
[3]. Some work studied wind power forecasting and hybrid forecast
10 1 0.96985 0.0987 0.9698985 0.0987049
engine employing methodology that measured actual wind signals 10 2 0.97185 0.098115 0.9718986 0.0981199
using a feature selection filter process. They used feature selection and 10 3 0.97385 0.09753 0.9738987 0.0975349
hybrid forecast engines bundled with honeybee mating optimization 10 4 0.97585 0.096945 0.9758988 0.0969499
method. In order to collect pertinent data, filter and function-based 20 1 0.96185 0.10104 0.9618981 0.1010451
20 2 0.96385 0.100455 0.9638982 0.10046
clustering models were amalgamated to form a hybrid method which
20 3 0.96585 0.09987 0.9658983 0.099875
brought about a more accurate and precise calculation of the wind 20 4 0.96785 0.099285 0.9678984 0.09929
signal. After exhaustive testing and comparisons to several established 30 1 0.95385 0.10338 0.9538977 0.1033852
models, it was discovered that various weather-related factors, such as 30 2 0.95585 0.102795 0.9558978 0.1028001
30 3 0.95785 0.10221 0.9578979 0.1022151
wind speed and temperature, impact wind energy optimal performance.
30 4 0.95985 0.101625 0.959898 0.1016301
However, it was concluded that wind power is a valid and useful form 40 1 0.9913 0.09957 0.9913496 0.099575
of energy [14]. Another work presents the Gaussian Process (GP) that 40 2 0.9997 0.09022 0.99975 0.0902245
enabled the forecasting of the unpredictability of wind velocity based 40 3 0.9978 0.09667 0.9978499 0.0966748
on actual wind speed statistics. Because the GP method is correlated 40 4 0.9986 0.09547 0.9986499 0.0954748
50 1 0.98585 0.09402 0.9858993 0.0940247
with the kernel machine approach and Bayesian estimation, a more
50 2 0.98785 0.093435 0.9878994 0.0934397
measured result was arrived at leading to fewer errors. In fact, this 50 3 0.98985 0.09285 0.9898995 0.0928546
method gave a better understanding and assessment of wind speed 50 4 0.99185 0.092265 0.9918996 0.0922696
variances [15]. 60 1 0.97785 0.09168 0.9778989 0.0916846
Most of the new forecasting models are based on the combination of 60 2 0.97985 0.091095 0.979899 0.0910996
60 3 0.98185 0.09051 0.9818991 0.0905145
different approaches. This combination will include different statistical 60 4 0.98385 0.089925 0.9838992 0.0899295
features for different approaches in the proposed model. This combi-
nation will achieve the optimal accuracy which will minimize the error
in the forecasted model [16]. Different models were proposed in the Table 2
literature to predict the speed of the short-term wind. Autoregressive MAPE and nRMSE for different models for wind speed forecasting.
integrated moving average (ARIMA), artificial neural network (ANN)
Model Models trained data Models for untrained data
and models of the support-vector machine (SVM) were used and based
on the results the hybrid ARIMA-ANN and ARIMA-SVM gave the best MAPE nRMSE MAPE nRMSE
results [17]. A combination of ensemble empirical mode decomposition
3 WNN_ANN_TS 3.3492 0.07565 3.76596 0.098345
(EEMD) and Genetic Algorithm based of Back Propagation neural net-
2 WNN_ANN_RKF 3.2991 0.07353 3.69043 0.085989
work was proposed [18]. Visual Molecular Dynamics (VMD) was used 1 WNN_TS_RKF 3.1001 0.05873 3.35403 0.079079
to decompose wind speed into three sections: nonlinear, linear and the
noise section. Different models are proposed based on the character-
istics of different sections. Principal component analysis -radial basis description of physical phenomena in the atmosphere, geographical
function (PRBF) model was used for the nonlinear section, ARMA with conditions and is called definite method. This method is considered as
Markov chain Monte Carlo (MCMC) methods was used for the linear one of the oldest and classic methods [25]. A hybrid method which is a
section, and probability distribution was used for the noise part. Then hybrid of Variational Mode Decomposition (VMD) [26] and Multi-
the three sections were combined to create a hybrid model of VMD- Kernel Regularized Pseudo Inverse Neural Network (MKRPINN) [27]
PRBF-ARMA-MCMC [19]. was used to predict the speed of the short-term wind speed. Two hybrid
Long short-term memory network (LSTM) algorithm and the models of autoregressive integrated moving average (ARIMA), artificial
Gaussian mixture model (GMM) was proposed in the literature to study neural network (ANN) and ARIMA and support-vector machine (SVM)
the error distribution characteristics of short-term wind power fore- were proposed and compared. The hybrid of ARIMA-ANN and ARIMA-
casting. Radial basis function (RBF), wavelet, deep belief network SVM proofs their effectiveness compared to ARIMA, ANN and SVM
(DBN), back propagation neural networks (BPNN), and Elman neural [28]. A hybrid of ensemble empirical mode decomposition (EEMD) and
network (ELMAN) were used to forecast the wind power. The LSTM GA-BP neural network, was introduced as a new forecasting method for
method gave higher accuracy and faster convergence and GMM method wind speed [29,30].
gave better performance and evaluation [20]. Six different hybrid The literature work done on wind speed forecasting showed that
models for harmonic tidal currents forecasting based on ANN, least hybrid approaches performed well compared to individual forecasting
square methods and WNN were used. The author proved the effec- techniques, but the problem is which combination should we use. Most
tiveness of using hybrid intelligent modes like ANN and WNN. Different of the recent papers are based on using hybrid models for forecasting to
combinations of Kalman filtering (KF), Wavelet and artificial neural improve the accuracy of the overall system. Most researchers are using
network for load forecasting were used [21–23]. different approaches based on the characteristic of the forecasted
There are mainly three different categories of forecasting: statistical phenomena. For wind speed the best approaches are ANN, WNN, RKF
methods, physical (climate) methods and hybrid forecasting methods and TS as the wind speed is nonlinear and it has some noise. ANN and
[24]. The most used methods are hybrid methods as they are able to WNN are excellent approaches in dealing with nonlinear function as
deal with the nonlinearity of the data like wind speed. Statistical well as TS and RKF are excellent approaches in dealing with the noise.
methods are not preferred specially if the data is nonlinear. Climate From that point we tried to use different combinations of ANN, WNN,
methods need more data. The combined methods are the more accurate TS and RKF. This motivates us to use hybrid combinations of WNN-
methods for wind speed forecasting. Combined artificial intelligent ANN-TS,WNN-ANN-RFK and WNN-TS-RKF and train the hybrid model
methods are outperform as they can deal with the nonlinear oscillations that has all features of different used approaches to minimize the error
of wind speed, but the duration of training is longer, and it is more and increase the accuracy to reach to the optimal accurate model. Due
complex. The physical method depends on using mathematical

2
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 1. The proposed hybrid model construction of WNN, ANN and TS.

Fig. 2. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
WNN, ANN and TS.

Fig. 3. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid model
WNN, ANN and TS.

Fig. 4. The proposed hybrid model construction of WNN, ANN and RKF.

to uncertainties and fluctuation of the data; the models are not accurate proposed models. These models are designed such that the forecasting
especially if we are dealing with the new generations of renewable will be used for 24 h a head. Mean absolute percentage error (MAPE) is
energy and smart grid. This motivates hybrid models to take the ad- used in this paper as a comparison method for different models.
vantages of all of these models. Hybrid models of WNN, ANN, TS and The paper is organized as follows. In Section “ANN, WNN, FS and
RKF are used in this work to add many desirable features to the RKF Approaches and Measuring Methods” all used techniques are

3
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 5. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
WNN, ANN and RKF.

Fig. 6. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid model
WNN, ANN and RKF.

Fig. 7. The proposed hybrid model constrcution of WNN, TS and RKF.

introduced. In Section “Models Parameters Selection” the model para- 4. To modify the AI parameters based of the optimal size of individual
meters selections are introduced. In Section “Wind Speed Forecasting approaches to achieve the optimal accuracy for the hybrid models
Models” the proposed network construction is introduced. Section which will lead to the system convergence and increase the accuracy
“Order of the Hybrid models selection” describes different orders of the of the network. This will reduce the training duration.
used techniques for the hybrid models as well as their simulation and
the model testing used for the proposed work. The aim of this work is ANN, WNN, FS and RKF approaches and measuring methods
to:
Artificial neural networks
1. To develop a new combination of hybrid intelligent (AI) approaches
with RKF and TS for the preprocessed data. Back propagation is a supervised training approach and is used for
2. To predict the wind speed as well as employing a proposed accurate improving the error derivative of the weights and biases. The Radial
hybrid model. Basis Function (RBF) network is practical application model. Linear
3. To obtain an optimal size of the induvial approaches for the best transfer functions are used for layers and nonlinear transfer functions
performance. (normally Gaussian) are used for hidden layers. RBF requires more

4
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 8. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
WNN, TS and RKF.

Fig. 9. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid model
WNN, TS and RKF.

neurons compared to BP but it needs less time for the training. The The transformation performed are local and as a result or that their
weights of the hidden layer are depending on clustering techniques training is much faster. Radial basis networks may require more neu-
[21,22,31,32]. BP is a three-layer feed-forward neural network, which rons than standard feed forward networks, but often they can be de-
includes an input layer, a hidden layer/s and an output layer with linear signed to take a fraction of time it takes to train standard feed forward
neurons. The backpropagation algorithm consists of two phases; the networks [31–35]. Both BB and RBF are used in this work, but RBF
forward phase where the activations are propagated from the input to proves its strength in dealing with the clustered data.
the output layer, and the backward phase, where the error between the
observed actual and the requested nominal value in the output layer is Wavelet neural networks
propagated backwards in order to modify the weights and bias values.
BP algorithm needs many repetitions to converge; however, there is WNN has the same ability as of ANN with different activation
always the possibility to get stuck in a local minimum, often due to false functions in the hidden layer. Due to the localized wavelet activation
weight dimension choice [31–35]. RBF utilizes Cover’s theorem which functions WNN has more compact techniques and learning speed. The
states that “A non-linearly separable problem is highly separable in output is represented by sum of weighted wavelets in a similar way like
high dimensional space than it is in low dimensional space.” Nodes in ANN. wjk is the weight between the hidden unit j and input unit k. wij is
hidden layer perform the radial basis transformation which is basically the weight between the output and hidden and hidden unit j. The sum
increasing the dimensions of input feature vector and by doing that of weighted inputs to the jth hidden neuron, xk (n) is defined as thek th
defining optimal receptors and spread of the radial basis function. Input input and is represented by f j (n) [35–37].
units distribute the values to the hidden layer units uniformly, without
k=m
multiplying them with weights. Hidden units are known as RBF units f j (n ) = wjk (n) x k (n ) (6)
k=0
because their transfer function is a monotonous radial basis function.
The output of each hidden neuron is defined by

5
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Table 3 auto-regression algorithm.

MAPE and nRMS of all possible models for wind speed forecasting.
Model Models trained data Models for untrained data Kalman filtering

MAPE nRMSE MAPE nRMSE

Kalman filtering is known as linear quadratic estimation. It is an
2 WNN_TS_RKF 3.1001 0.05873 3.35403 0.079079 algorithm that uses set of data containing some statistical noise and
1 WNN_RKF_TS 3.0202 0.05234 3.25201 0.069079 other inaccuracies and estimates some variables to represent the trend
6 TS_RKF_WNN 3.2919 0.07229 3.66211 0.084907 of these dataset by using some probability distribution over the vari-
5 TS_WNN_RKF 3.2698 0.0719 3.66009 0.081079 ables.
4 RKF_TS_WNN 3.2599 0.07015 3.55807 0.082079
3 RKF_WNN_TS 3.18 0.06112 3.36605 0.080079
The model is defined as
16 WNN_ANN_RKF 3.2991 0.07353 3.69043 0.085989 xk = Fxk + B uk + wk (16)
1 1 1
18 WNN_ANN_TS 3.3492 0.07565 3.76596 0.098345
17 WNN_RKF_ANN 3.2992 0.07354 3.69053 0.085999 where F is the state transition matrix applied to the previous state
15 WNN_TS_ANN 3.29283 0.073519 3.66288 0.08552
vector xk-1, B is the control-input matrix applied to the control vector uk-
24 ANN_WNN_RNN 3.413438 0.085139 3.836992 0.107638
22 ANN_WNN_TS 3.35231 0.077338 3.769997 0.099614 1, and wk-1 is the process noise vector that is assumed to be zero-mean
23 ANN_RKF_WNN 3.391481 0.082725 3.812927 0.105112 Gaussian. Kalman filter is used to estimate xk. Kalman filter has two
19 ANN_TS_WNN 3.3494 0.07665 3.76616 0.098445 stages; the prediction and the updating. The Prediction is based on the
20 ANN_TS_RKF 3.3496 0.0768 3.76636 0.098615 state estimate xk – = F xk 1 + Buk-1 and the updating is using the re-
21 ANN_RKF_TS 3.350135 0.077108 3.767513 0.098865
14 RKF_ANN_WNN 3.29271 0.073516 3.66269 0.08543
sidual y~k = Zk–H xk where is z is the measurement vector and H is the
12 RKF_ANN_TS 3.29248 0.072606 3.66259 0.085322 measurement matrix [38–40].
13 RKF_WNN_ANN 3.29258 0.073506 3.66269 0.08542
11 RKF_TS_ANN 3.29238 0.072516 3.66247 0.085227
Measuring methods
8 TS_ANN_RKF 3.29193 0.072315 3.662141 0.085087
10 TS_ANN_WNN 3.29228 0.072426 3.66237 0.085137
7 TS_RKF_ANN 3.29191 0.0723 3.66212 0.084917 MAPE is defined as the absolute average difference between the
9 TS_WNN_ANN 3.29205 0.072326 3.66224 0.085037 actual and predicted values as shown in Eq. (1) [21,22].

1 N xi yi
MAPE = 100
a, b (f j (n)) = [(f j (n) bj (n))/aj (n)] (7) N 1 yi (1)
where is the wavelet function, aj (n) is defined as the scaling, and where xi is the predicted value, yi is the actual value, and N is the
bj (n) is the translation coefficients of the wavelet function in hidden number of observations.
neuron. The input f (n) and output y (n) of the output neuron is de-
scribed by the following equations n( xi yi ) ( yi )( xi )
r=
k=m (n x i2 ( x i )2)(n yi 2 ( yi ) 2) (2)
f (n ) = wij (n) a, b (f j (n)) (8)
k=0
where r is describing the strength of the relation between the measured
y (n ) = [f (n)] (9) yi and predicted x i data and is defined in Eq. (2) as [21,22]. The most
commonly used factor is normalized root mean square error (nRMSE)
which is defined in Eq. (3) and is used also in this paper [21–23].
Time series
1 N
N 1
(x i yi )2
Time series are formed by discrete measurements of a specific nRMSE =
y (3)
quantity at successive time instants. One of the most powerful methods
is allowing analyzing time series and make predictions, is the nonlinear where y is mean of the actual data.

Fig. 10. The relatioship between the actual and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using hybrid model of WNN,
RKF and TS.

6
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 11. The relatioship between the actual and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using hybrid model of
WNN, RKF and TS.

Fig. 12. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
TS, RKF and WNN.

Models parameters selection Wind speed forecasting models

In this section the model parameters are selected based on many In this section we use three different models to choose the best
runs to optimize the model performance and increase its accuracy for combination of different techniques. Three hybrid models are tested at
the individual algorithms using different numbers of neurons and the beginning to choose the best combination based on the smallest
hidden layers. The best model is chosen based on the lowest normalized error. These models are hybrid models of WNN ANN TS, WNN ANN
root mean square error. By doing that we tried to find the best number RKF, and WNN TS RKF. This combination will include different statis-
of neurons and hidden layers for the minimum nRMSE value. The best tical features and characteristics for different approaches in the pro-
parameters will be used for all hybrid models. Table 1 shows nRMSE posed model. This combination will achieve the optimal accuracy to
values for different runs using either ANN or WNN with different achieve the minimum error in the forecasted model. Using ANN, TS,
number of neurons and hidden layers to choose the best size based on RKF or WNN alone is not recommended as each approach itself has
the minimum nRMSE. From Table 1; the best ANN model is the model specific characteristics. Using the hybrid models will gain all char-
that has 40 neurons and 2 hidden layers and the best WNN model is the acteristics of the different used approaches for example RKF works very
model that has 40 neurons and 2 hidden layers as well for minimum well with the noise, ANN and WNN work very well with the nonlinear
nRMSE. Based on these tests the hybrid models are adjusted and trained data and TS works well with the periodic data. The hybrid models will
and the results for each model is shown in Table 2 as well as their ranks gain all characteristics of the used approach and this will facilitate their
are written before each model. role to achieve the best performance. The data that we have for wind
and residual has noise, it is nonlinear data and periodic so using the
combination of different approaches will accelerate the training time
and increase the accuracy.

7
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 13. The relatioship between the actual and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using hybrid model of TS,
WNN and RKF.

Fig. 14. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
TS WNN and RKF.

Fig. 15. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid
model TS WNN and RKF.

8
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 16. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
RKF TS and WNN.

Fig. 17. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid
model RKF TS and WNN.

Fig. 18. The relationship between the actual data and forecasted data for hourly wind speed forecasting of trained data in Halifax/Nova Scotia using a hybrid model
RKF, WNN and TS.

9
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

Fig. 19. The relationship between the actual data and forecasted data for hourly wind speed forecasting of untrained data in Halifax/Nova Scotia using a hybrid
model RKF, WNN and TS.

Hybrid model of WNN, ANN and TS Order of the hybrid models selection

In this model the data is trained using WNN first; then the difference In this section all different orders for the best sequence of the
between the forecasted and trained wind speed data is calculated (re- techniques are tested to choose the best sequence of different techni-
siduals 1). After that the residuals data is used along with the time to ques. Twenty-four different combinations of hybrid models of WNN,
train the second stage (ANN). The residuals from the second stage ANN, TS and RKF with different orders are tested. All results for all
(residuals 2) which is the difference between residuals 1 and the fore- combinations are shown in Table 3. In this section only the best six
casted residuals is calculated. Residuals 2 data is fed to the third stage models are simulated but the results for all different models are shown
which is TS in this model. The output from TS, WNN and ANN is added in Table 3. Clustering is considered as the grouping of similar data. This
to calculate the forecasted wind speed as seen in Fig. 1. The results of could be done by dividing the data into identified number of classes
the proposed model for trained and untrained data is shown in Figs. 2 according to a certain criterion, and of assigning the membership of the
and 3. Table 2 shows the MAPE and nRMSE for the trained and un- patterns in these classes. The number of classes is playing a crucial rule
trained data for that model. in dealing with the data. In this model we use ANN and WNN to train
the row data. In ANN and WNN there are a number of different ar-
chitectures which are designed for clustering. One of the most common
Hybrid model of WNN, ANN and RKF used methods for clustering is self organizing maps (SOM). SOM has a
set of neurons connected to form a certain topological grid which has
In this model the data is trained using WNN first then the difference the ability to adapt weights to the pattern and the weights of its
between the forecasted and trained wind speed is calculated (residuals neighbourhood when the neuron with closest weight vector is con-
1). After that the residuals data is used along with the time to train the sidered. This way is helping in finding clustered data using SOM. An-
second stage (ANN). The residuals from the second stage (residuals 2) other approach is Adaptive Resonance Theory (ART) which is devel-
which is the difference between residuals 1 and the forecasted residuals oped by Carpenter and Grossberg (1987, 1988). This network has the
data is calculated. Residuals 2 data is fed to the third stage which is RKF ability to learn clusters in an unsupervised mode in a controlled dis-
in this model. The output from RKF, WNN and ANN is added to cal- covery way of clusters. Two layers are characterized in this method:
culate the forecasted wind speed as seen in Fig. 4. The results of the comparison field and recognition field.
proposed model for trained and untrained data is shown in Figs. 5 and
6. Table 2 shows the MAPE and nRSME for the trained and untrained
Hybrid model of WNN, TS and RKF
data for that model.
In this model a combination of WNN, TS and RKF is tested as seen in
Figs. 8 and 9 for the trained and untrained data. In this model the data
Hybrid model of WNN, TS and RKF
is trained using WNN first then the difference between the forecasted
and trained wind speed is calculated (residuals 1). After that the re-
In this model the data is trained using WNN first then the difference
siduals data is used along with the time to train the second stage (TS).
between the forecasted and trained wind speed is calculated (residuals
The residuals from the second stage (residuals 2) which is the difference
1). After that the residuals data is used along with the time to train the
between residuals 1 and the forecasted residuals data is calculated.
second stage (TS). The residuals from the second stage (residuals 2)
Residuals 2 data is fed to the third stage which is RKF in this model. The
which is the difference between residuals 1 and the forecasted residuals
output from RKF, WNN and TS is added to calculate the forecasted wind
is calculated. Residuals 2 data is fed to the third stage which is RKF in
speed. Table 3 shows the MAPE and nRSME for the trained and un-
this model. The output from RKF, WNN and TS is added to calculate the
trained data for that model. This model has MAPE and nRMSE of
forecasted wind speed as seen in Fig. 7. The results of the proposed
3.1001 and of 0.05873 for the trained data and 3.25201 and is con-
model for trained and untrained data is shown in Figs. 8 and 9. From
sidered as a good model.
Table 2 the MAPE and nRMSE are 3.1001 and 0.05873 for the trained
data and 3.35403 and 0.079079 for the untrained data respectively
which is considered as the smallest error compared to the previous two Hybrid model of WNN, RKF and TS
models. From Table 3 and all of the previous figures the best combi-
nation is the combination of WNN, TS, and RKF. In this model a combination of WNN, RKF and TS is tested, and the

10
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

results is shown in Table 3. This model is based on training the system untrained data. In this model RKF is fed by the training data as a first
using WNN as a first step. The results from that step is used for calcu- stage (first sub model). The input to that model is the time in hours and
lating the residuals for the second step (residuals 1). Residuals 1 data is the output is the wind speed in meters per seconds. Residuals 1 data is
fed to the second stage to train the second stage (RKF). In this stage calculated based on that sub model. Residuals 1 data is then fed to the
residuals 2 data is calculated based on the difference between residuals second sub model (WNN) for training. Residuals 2 data is calculated
1 and the forecasted residuals. In the third stage Residuals 2 data is fed based on the second sub model. Residuals 2 data is used to the train the
to the sub model which is TS. The output from RKF, WNN and TS is third sub model (TS). The output from RKF, WNN and TS is added to
added to calculate the forecasted wind speed. The forecasted data is calculate the forecasted wind speed. This model has MAPE and nRMSE
shown in Figs. 10 and 11 for the trained and untrained data. This model of 3.2919 and of 0.07229 for the trained data and 3.66211 and
has MAPE and nRMSE of 3.0202 and of 0.05234 for the trained data 0.0084907 for the untrained data which is considered better than the
and 3.25201 and 0.069079 for the untrained data and is considered as previous three models but still the first two models are better. All
excellent model as its error is very small compare to the previous models are ranked based on their MAPE and nRMSE. The best sequence
model. of techniques is WNN, RKF and TS. This sequence is ranked as the best
model then a hybrid of WNN, TS and RKF sequence is the second best as
Hybrid model of TS, RKF and WNN shown in Table 3. The sequence of TS, RKF and WNN is the worst one.

This model has three sub models (TS, KRF and WNN) to form the
hybrid model of different combinations. The data is fed first to the first Conclusions
sub model (TS) to train it for generating the forecasted data from the
first stage. The output of the first sub model is used to calculate re- This paper proposes novel hybrid models for wind speed forecasting
siduals 1. Residuals 1 data is fed to the second sub model (RKF) then the using different combinations of intelligent models. Hybrid models of
results of that model are used to calculate residuals 2 data that will be WNN, ANN, TS and RKF are applied to simulate and forecast the wind
used to train the third sub model (WNN). The output from all sub speed. Twenty-four models are proposed and ranked from the highest
models are added to generate the forecasted wind speed. The forecasted accurate one to the lowest accurate one based on MAPE and nRMSE
data is shown in Figs. 12 and 13 for the trained and untrained data. This results. All models are giving promising results. All models are tested
model has MAPE and nRMSE of 3.2919 and of 0.07229 for the trained and validated using different dataset. The techniques combinations and
data and 3.66211 and 0.0084907 for the untrained data which is higher order in the hybrid models are very important in determining which
than the previous two models. one is the best. The different combinations of models are ranked as
shown in Table 2. The worst combination model is the hybrid of WNN,
Hybrid model of TS, WNN and RKF ANN and TS. The combination of WNN, TS, and RKF is ranked at the
top. The order of the techniques used in the hybrid models is playing a
In this model a combination of TS, WNN and RKF in order is tested crucial role for ranking. Based on Table 3 the best sequence of the
and the MAPE and nRSME are calculated as seen in Table 3. This model techniques used for the proposed models is the hybrid of WNN, RKF and
is based on three different sub models (TS, WNN and RKF). Residuals TS in order.
data 1 is calculated based on the first sub model; then residuals 1 data is The new contribution in this paper is the hybrid models of different
used to train the second sub model to calculate residuals 2 data that will techniques based on Artificial Intelligent as well as the feedback signals
be used to train the third sub model. The output from RKF, WNN and TS from the residuals to improve the overall system accuracy for a pre-
is added to calculate the forecasted wind speed. The forecasted data is processed data. Different algorithms with different learning rates, dif-
shown in Figs. 14 and 15 for the trained and untrained data. The ferent training functions and epochs are tested based on the pre-
forecasted data is shown in Figs. 14 and 15 for the trained and un- processed data. A table of comparison is added to show the importance
trained data. This model has MAPE and nRMSE of 3.2698 and of 0.0719 of the proposed work compared to the previous work done in that area
for the trained data and 3.66009 and 0.0081079 for the untrained data by using the previous models with the same data used in this work. The
which is better than the previous model but the first two models are proposed models outperform all the previous used approaches. This
better than this model. study is very important as wind power generated is highly dependent
on wind speed which is intermittent and variable in nature which
Hybrid of RKF, TS and WNN would lead to problems like voltage fluctuations, power quality and
stability issues. Accurate prediction enables better preparation of power
In this model a combination of RKF, TS and WNN in order is tested system operation to manage these issues in wind power turbines output.
and the MAPE and nRSME are calculated as seen in Table 3. In this sub The inaccurate information will lead to instability in the grid. This will
model the data is trained using RKF first. The output of that sub model cause voltage fluctuations and transformers maloperations and power
is used to calculate residuals 1 that will be used to train the second sub outage. This would cause huge loss of money. Thus, more accurate
model (TS). The output of that model is used to calculate residuals 2 prediction model of wind speed is very essential part in the power
that will be used to train the third sub model (WNN). The overall model system grid.
is used to calculate the forecasted wind speed. The forecasted data is
shown in Figs. 12 and 13 for the trained and untrained data. The
forecasted data is shown in Figs. 16 and 17 for the trained and un- Declaration of Competing Interest
trained data. This model has MAPE and nRMSE of 3.2599 and of
0.07015 for the trained data and 3.55807 and 0.0082079 for the un- The authors declare that they have no known competing financial
trained data which is better than the previous model but the first two interests or personal relationships that could have appeared to influ-
models are still better than this model. ence the work reported in this paper.

Hybrid model of RKF, WNN and TS

Appendix A. Supplementary data
In this model a combination of RKF, WNN and TS in order is used to
create three different sub models to generate the overall model. The Supplementary data to this article can be found online at https://
forecasted data is shown in Figs. 18 and 19 for the trained and doi.org/10.1016/j.seta.2020.100802.

11
H.H.H. Aly Sustainable Energy Technologies and Assessments 41 (2020) 100802

References ANN, WNN and KF based on clustering techniques for smart grid. Int J Electric
Power Syst Res 2020.
[23] Senjyu T, Takara H, Uezato K, Funabashi T. One-Hour-Ahead load forecasting using
[1] Aly HH Hamed, El-Hawary ME. The current status of wind and tidal in-stream neural network. IEEE Trans Power Syst 2002;17:113–8.
electric energy resources. Am J Electr Power Energy Syst 2013;2(2):23–40. [24] Yang W, Wang J, Lu H, Niu T, Du P. Hybrid wind energy forecasting and analysis
[2] Hamed Aly. Forecasting, Modeling and Control of Tidal Currents Electrical Energy system based on divide and conquer scheme: a case study in China. J Cleaner Prod
Systems, Ph.D. dissertation, Department of Electrical and Computer Engineering, 2019.
Dalhousie University, Halifax, N.S.; 2012. [25] Zhang W, Maleki A, Rosen M. A heuristic-based approach for optimizing a small
[3] Aly Hamed H, El-Hawary ME. An Overview of Off-shore Wind Electrical Energy independent solar and wind hybrid power scheme incorporating load forecasting. J
Systems. In: 23rd Annual Canadian IEEE Conference on Electrical and Computer Cleaner Prod 2019.
Engineering, Calgary, Alberta, Canada, May 2-5, 2010. [26] Dupre A, Drobinski P, Alonzo B, Badosa J, Briard C, Plougonven R. Sub-hourly
[4] Aly HH Hamed, El-Hawary ME. A proposed ANN and FLSM hybrid model for tidal forecasting of wind speed and wind energy. J Renewable Energy 2020.
current magnitude and direction forecasting. IEEE J Oceanic Eng 2014;39(1). [27] Sun W, Huang C. A carbon price prediction model based on secondary decom-
[5] Aly HH Hamed, El-Hawary ME. A proposed algorithms for tidal in-stream speed position algorithm and optimized back propagation neural network. J Cleaner Prod
model. Am J Energy Eng. 2013;1(1):1–10. 2020.
[6] Cordova Samuel, Rudnick Hugh, lvaro Lorca A, Martınez Vıctor. An efficient fore- [28] Naik J, Dash S, Dash P, Bisoi R. Short term wind power forecasting using hybrid
casting – optimization scheme for the intra-day unit commitment process under variational mode decomposition and multi-kernel regularized pseudo inverse
significant wind and solar power. IEEE Trans Sustainable Energy 2018. neural network. J Renewable Energy 2018.
[7] Florian Ziel. Modeling the impact of wind and solar power forecasting errors on [29] Shi J, Guo J, Zheng S. Evaluation of hybrid forecasting approaches for wind speed
intraday electricity prices. In: 14th International Conference on the European and power generation time series. Renew Sustain Energy Rev 2012.
Energy Market (EEM), Dresden, Germany; 2017. [30] Wang S, Zhang N, Wu L, Wang Y. Wind speed forecasting based on the hybrid
[8] Kofi E. Hagan, Olufemi O. Oyebanjo, Tarek Medalel Masaud, Rajab Challo. A ensemble empirical mode decomposition and GA-BP neural network method. J
Probabilistic Forecasting Model for Accurate Estimation of PV Solar and Wind Renewable Energy 2019.
Power Generation. In: IEEE Power and Energy Conference at Illinois (PECI), 2016, [31] Quilumba Franklin L, Lee Wei-Jen, Huang Heng, Wang David Y, Szabados Robert L.
Urbana, IL, USA. Using smart meter data to improve the accuracy of intraday load forecasting con-
[9] Mohamed Abuella, Student Member, IEEE, Badrul Chowdhury, Senior Member, sidering customer behavior similarities. IEEE Trans Smart Grid March 2015;6(2).
IEEE Hourly Probabilistic Forecasting of Solar Power North American Power [32] Aly Hamed. Offshore wind and tidal current energy resources: forecasting, mod-
Symposium (NAPS); 2017. eling, and control. LAMBERT Academic Publishing; 2013.
[10] Wang Yu, Shen Yinxing, Mao Shiwen, Cao Guanqun, Nelms Robert M. Adaptive [33] Xie BT, Yu H, Wilamowski B. Comparison between traditional neural networks and
learning hybrid model for solar intensity forecasting. IEEE Trans. Ind. Inf. 2018. radial basis function networks. In: 2011 IEEE International Symposium on
[11] Krishnaveny R. Nair, Vanitha V. Forecasting of wind speed using ANN, ARIMA and Industrial Electronics, Gdansk, 2011, p. 1194–9.
Hybrid Models. In: International Conference on Intelligent Computing, [34] MacQueen JB. Some methods for classification and analysis of multivariate ob-
Instrumentation and Control Technologies (ICICICT); 2017. servations. Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and
[12] Yun Wang, Qinghua Hu, Dipti Srinivasan, Zheng Wang. Wind power curve mod- Probability. Berkeley: University of California Press; 1976.
eling and wind power forecasting with inconsistent data, J IEEE Trans Sustainable [35] Rokach L. A survey of clustering algorithms. In: Maimon O, Rokach L, editors. Data
Energy; 2018. Mining and Knowledge Discovery Handbook. Boston, MA: Springer; 2009.
[13] Saida Makhloufi, Gobind Gopalakrishna Pillai. Wind Speed and Wind Power [36] Haykin Simon, Haykin S. Neural networks: a comprehensive foundation. 2nd ed.
Forecasting Using Wavelet Denoising-GMDH Neural Network. In: The 5th New Jersey: Prentice-Hall; 1999.
International Conference on Electrical Engineering – Boumerdes (ICEE-B); 2017. [37] Moody J, Darken CJ. Fast learning in networks of locally-tuned processing units.
[14] Mehdi Bagheri, Oveis Abedinia, Mohammad Salay Naderi, A Novel Wind Power Neural Comput 1989;1(2):281–94.
Forecasting based Feature Selection and Hybrid Forecast Engine Bundled with [38] Zainuddin Z, Pauline O. Improved wavelet neural networks for early cancer diag-
Honey Bee Mating Optimization. In: IEEE Industrial and Commercial Power nosis using clustering algorithms. Int J Inf Mathem Sci 2010;6(1):30–6.
Systems Europe (EEEIC / I&CPS Europe); 2018. [39] Simon D. Optimal state estimation: Kalman, H infinity, and nonlinear approaches.
[15] Hiroyuki Mori, Eitaro Kurata. Application of Gaussian Process to wind Speed Oxford: John Wiley & Sons; 2006.
Forecasting for Wind Power Generation. In: IEEE International Conference on [40] Fred C. Schweppe. Uncertain Dynamic Systems, Englewood Cliffs, N.J., Prentice-
Sustainable Energy Technologies; 2008. Hall; 1973.
[16] T. Heppelmann, A. Steiner, S. Vogt, Application of numerical weather prediction in
wind power forecasting: assessment of the diurnal cycle, J Metorologische
Zeitschrift, 26 (3), 2017. Hamed H.H. Aly received the B.Eng. and M.ASc. degrees in electrical engineering with
[17] Shi J, Guo J, Zheng S. Evaluation of hybrid forecasting approaches for wind speed distinction, in 1999 and 2005, respectively, from Zagazig University, Egypt and the Ph.D.
and power generation time series. J Renew Sustain Energy Rev degree from Dalhousie University, Canada, in 2013. He was working as Postdoctoral
2012;16(5):3471–80. Research Associate for one year and as a lecturer for four years at Dalhousie University.
[18] Wang S, Zhang N, Wu L, Wang Y. Wind speed forecasting based on the hybrid He worked as a lecturer at Zagazig University for eight years. His research interests in-
ensemble empirical mode decomposition and GA-BP neural network method. J clude: Power System Dynamics and Stability; Power Electronics; Electric Machines, Power
Renewable Energy 2016;94:629–36. Quality Issues; Applications of Artificial Intelligence in Power Systems; Smart Grid; and
[19] Zhang YG, Zhao Y, Kong CH. Chen BA new prediction method based on VMD-PRBF- the Integration of Renewable Energy into the Electrical Grid. He taught in different
ARMA-E model considering wind speed characteristic. Energy Convers Manage universities in Canada as well as in the Canadian Forces Naval Engineering School. He
2020;203:1–18. Paper ID: 112254. pioneered many computational and artificial intelligence solutions to problems in eco-
[20] Zhang Jinhua, Yan Jie, Infield David, Liu Yong Qian, Lien Fue-sang. Short-term nomic/environmental operation of power systems. He has written one textbook, three
forecasting and uncertainty analysis of wind turbine power based on long short- chapters books and more than 50 refereed journal and conference articles. He has con-
term memory network and Gaussian mixture model. J Appl Energy sulted and taught for more than 16 years. He is a senior member of the Institute of
2019;241:229–44. Electrical and Electronics Engineers (IEEE). Dr. Aly is a faculty member at Acadia
[21] Aly HH Hamed. A novel approach for tidal currents harmonic constitutions fore- University since 2013. Dr. Aly is currently an Assistant Professor at Faculty of
casting hybrid models based on clustering techniques for smart grid. Int J Engineering, Zagazig University and an adjunct Assistant Professor at Dalhousie
Renewable Energy 2020;147(Part 1):1554–64. University.
[22] Aly HH Hamed. A proposed intelligent short-term load forecasting hybrid models of

12