Electrical Power and Energy Systems 61 (2014) 399–409

Contents lists available at ScienceDirect

Electrical Power and Energy Systems

journal homepage: www.elsevier.com/locate/ijepes

A model predictive control framework for reliable microgrid energy

management
Ionela Prodan a,⇑, Enrico Zio a,b
a
Chair on Systems Science and the Energetic Challenge, European Foundation for New Energy – EDF, Ecole Centrale Paris, Grande Voie des Vignes 92295 and Supélec,
Gif sur Yvette 91192, France
b
Department of Energy, Politecnico di Milano, Milan, Italy

a r t i c l e i n f o a b s t r a c t

Article history: This paper proposes a framework for reliable microgrid energy management based on receding horizon
Received 20 September 2013 control. A microgrid is considered for exemplification, connected to an external grid via a transformer and
Received in revised form 5 March 2014 containing a local consumer, a renewable generator (wind turbine) and a storage facility (battery).
Accepted 18 March 2014
Optimal scheduling of battery is sought for minimizing costs. To this aim, a predictive control framework
Available online 22 April 2014
is proposed, which allows taking into consideration cost values, power consumption and generation
profiles, and specific constraints. Uncertainty due to variations in the generator model parameters is
Keywords:
taken into account. The efficiency of the proposed approach is validated through simulation results
Microgrid energy management
Uncertainty
and comparisons using real numerical data for a test system often considered in bulk power system
Constrained Model Predictive Control (MPC) reliability evaluation studies. The obtained results show that predictive control is a viable approach for
Mixed-Integer Linear Programming (MILP) providing optimal energy management solutions accounting for costs, profiles and constraints.
Fault Tolerant Control (FTC) Ó 2014 Elsevier Ltd. All rights reserved.

Introduction wind or solar energy equipment, on-site storage, etc.), which

may change (stochastically) due to degradation, failure, aging
Intelligent electrical grids with renewable energy sources have and so on.
attracted increasing public attention in recent years. Green (solar Various approaches for microgrid energy management are
and wind in particular) energy production is supposed to increase reported in the literature. Proposed solution techniques include
significantly in the coming years, since the traditional energy heuristics [9,26], mathematical programming [11,5] and priority
supplies of Earth are finite and suffer from a ‘‘diminishing returns’’ rules [36]. The authors in [15,21,20,37] propose agent-based
curse. This requires a ‘‘smartgrid’’ system capable of dealing with modeling and simulation to analyze the interactions between
distributed production/intermittent variations of output and individual intelligent decision-makers in microgrid management.
optimal scheduling of demand [1,24,40,35]. Energy management solutions for hybrid renewable energy gener-
Microgrids can be key solutions for integrating renewable and ation has been proposed in [4,16]. In these works, the long-term
distributed energy resources, as well as distributed energy-storage goals are focused on the efficient use of electricity within microgr-
systems [19,3,7]. ids, e.g., the planning of battery scheduling to store the electricity
The present paper introduces a model predictive control frame- locally generated from renewable sources and reuse it during
work for optimal management of energy production in a microgrid periods of high electricity demand. It is important to mention
system with renewable and distributed supply sources. For this that in these works the decision framework is developed under
type of system, it is no longer possible to control each subsystem deterministic conditions, e.g., those of a typical day in summer,
in isolation, because of the functional dynamic interactions among and without taking into account explicitly the dynamics of the
all subsystems [27,23,4]. Moreover, it is necessary to take into individual components of the microgrids.
account not only exogenous factors (e.g. change in consumer load, To overcome these limitations and extend the above frame-
wind speed, price profile, etc.) but also the internal (state) dynam- works, we propose here to use Model Predictive Control (MPC)
ics and the structural properties of the individual components (as for handling control and state constraints while offering good per-
formance specifications (see, for instance, [33,34] for basic notions
⇑ Corresponding author. about MPC). Typically, in MPC, the objective (or cost) function to be
E-mail addresses: ionela.prodan@supelec.fr, ionela.prodan@ieee.org (I. Prodan), optimized penalizes deviations of the states and inputs from their
enrico.zio@ecp.fr (E. Zio). reference values, while explicitly enforcing the constraints. Due to

http://dx.doi.org/10.1016/j.ijepes.2014.03.017
0142-0615/Ó 2014 Elsevier Ltd. All rights reserved.
400 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409

its versatility, MPC has had a successful record in industrial appli-

cations; to mention just a few recent ones: refrigeration systems
[14], power production plants [8], transportation networks [27],
and microgrid networks [12]. The authors in [38] propose a look-
ahead predictive control algorithm to solve the economic dispatch
problem with large presence of intermittent resources. In [28] a
model predictive controller is applied for controlling the energy
flows inside a household system equipped with a ‘‘micro’’
combined heat and power unit. In addition, the household can
buy and sell electricity from/to the energy supplier; heat and
electricity can be stored in specific storage devices. In [39], MPC
is used for energy scheduling on a hydrogen-based microgrid
without batteries.
In the present paper, we propose a solution approach for the
Fig. 1. Microgrid architecture.
reliable energy management of a microgrid system based on a pre-
dictive control framework. To the best of the authors’ knowledge,
there does not exist a similar work in the literature. For exemplifi-
cation purposes, we consider a microgrid scheme taken from [22], System and model description
which is connected to an external grid via a transformer and con-
tains a local consumer, a renewable generator (wind turbine) and a This section presents the microgrid system, the dynamic models
storage facility (battery). The underlying management setting is which describe its components, the profiles considered for the con-
one of multi-criteria decision-making for battery scheduling, with strained optimization problem and the cost function formulation.
the objectives of: Consider the microgrid of Fig. 1, which includes a local con-
sumer (e.g., large cooling houses), a renewable generator (e.g.,
 increasing the utilization rate of the battery during high elec- wind turbine) and a storage facility (e.g., battery). The microgrid
tricity demand (i.e., decrease of the electricity purchase from is connected to the external grid via a transformer. The goal is to
the external grid) and plan the battery schedule in order to achieve the consumer objec-
 increasing the utilization rate of the wind turbine for local use tives of increasing the utilization rate of the battery during high
(with a consequence increase of the consumer independence electricity demand (i.e., decrease the electricity purchase from
from the external grid). the external grid) and the utilization rate of the generator for local
use (i.e., increase the consumer independence from the external
The consumers’ load, wind generator and price profiles are pro- grid).
vided. Then, a constrained multi-objective optimization problem is The interactions between the independent components of the
defined and solved in an optimal way to follow the predicted pro- microgrid are most important for accomplishing the consumer
files, for reliable energy management. objectives. As shown in Fig. 1 there are various links between the
With respect to previous works [21,32,22], we propose also a components of the microgrid, which determine the energy flow:
more realistic model of the battery. In particular, we consider leak-
age current and a switched model with ‘‘charge’’ and ‘‘discharge’’  The electrical power transmitted by the renewable generator to
functioning modes. the battery at time step t is represented by gðtÞ 2 R (note that
The remainder of the paper is organized as follows. Section the electrical power actually taken by the battery from the
‘System and model description’ presents the components of the renewable generator can be less and is denoted by g b ðtÞ 2 R)1.
microgrid system and Section ‘Optimization-based control for bat-  The electrical power transmitted by the battery to the consumer
tery scheduling’ describes the predictive control model for battery at time step t is represented by bc ðtÞ 2 R.
scheduling within the microgrid. Further remarks on the inclusion  In order to maximize the utilization of the battery within the
of a cost function with time-varying weights for better dealing microgrid, the consumer takes electrical power from the local
with uncertainties are presented in Section ‘Further remarks and renewable generator through the battery. The battery can be
results’ together with a fault tolerance scheme. Simulation results charged from the external grid but can also discharge electricity
are given in Section ‘Simulation results and comparison’ and some to the external grid: the electrical power transmitted by the
conclusions are finally drawn in Section ‘Conclusions’. battery to the external grid at time step t is denoted by
be ðtÞ 2 R.
Notation  It is possible to sell electricity to the external grid when the
level of charge in the battery is deemed sufficient for covering
The following notations will be used throughout the paper: local needs. The electrical power transmitted by the external
grid to the battery at time step t is bb ðtÞ 2 R and the electrical
– Let xðt þ s j tÞ denote the value of x at discrete time instant power transmitted by the external grid to the consumer at time
t þ s, predicted upon the information available at time instant t is represented by pðtÞ 2 R. Here, the transformer provides elec-
t 2 N with s > 0. trical power from the external grid as well as information about
– The length of the prediction horizon is denoted by N p and the the electrical market price, which plays an important role as the
time step is denoted by s. consumer can decide to take energy when the price is low.
– Throught the paper, gðtÞ [W], g b ðtÞ [W], bb ðtÞ [W], be ðtÞ [W], Therefore, the consumer has also the possibility to take electri-
bc ðtÞ [W], pðtÞ [W], are electrical power quantities, bðtÞ [W h] cal power from the external grid when the renewable resource
denotes the amount of energy stored at time step t; dðtÞ [W] is not available (or sufficient).
denotes the consumer load, eðtÞ [Euro/kW h] the market price
per hour and v ðtÞ [m/s] denotes the wind speed. 1
By adding this power signal we take into account the possibility for the battery to
– Let smax denote the maximum number of steps for which the accept less power than the maximal electrical power given by the generator, as it is
optimization problem is solved. the case in realistic implementations.
 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409 401

Remark 1. Note that providing the generator output to the battery wind turbine. Uncertainty in the power curve parameters (3) can
to eventually reach the consumer leads to frequent charge/dis- be considered in the ranges v ci 6 v ci 6 v ci ; v r 6 v r 6 v r ; v co 6
charge cycles. Depending on the physical storage mechanism v co 6 v co and Pr 6 Pr 6 P r , where v ci ; v ci ; v r ; v r ; v co ; v co ; P r ; Pr 2 R define
(the ‘‘battery’’), this could result in wear and tear. To address this a collection of power curve functions which form an uncertainty range as
issue, one could eliminate the use of the battery during normal illustrated in Fig. 2(b).
functioning by linking the generator directly to the user and keep
the battery as an auxiliary component in the grid, or to impose Constraints
an explicit, direct cost on the number of charge/discharge cycles.
These investigations are subject of our future work and will not As illustrated in Fig. 1, the consumer can take electricity
be further investigated here. Also, note that the ‘‘battery’’ can be from two sources, i.e., the battery and the external grid. There-
any mechanism which stores and discharges electrical energy in fore, for a reliable management of the energy system it is nec-
a controlled manner (flywheels, pump + turbine + lake systems, essary to ensure that at time t the electricity purchased by the
etc.). For these systems the charge/discharge effect on the lifetime consumer from these two sources satisfies the following
and costs can be significantly lower. condition:
Considering the above interactions among the components of
the grid, the aim is to control the battery to satisfy all consumers bc ðtÞ þ pðtÞ ¼ dðtÞ: ð4Þ
objectives. The energy stored in the battery at time t is used when the local
production is not sufficient to meet demand. On the other hand,
Dynamic models of the microgrid components energy is stored when the power supplied by the renewable gener-
ator exceeds the consumer demand. Subsequently, the amount of
In the proposed microgrid setting, the only component that we energy stored in the battery needs to remain between some
assume to be directly controllable is the battery (through the rates bounds; hence, the following condition is imposed:
of charge and discharge). For describing the battery operation2 (in
the charge and discharge modes), we introduce the following Bmin 6 bðtÞ 6 Bmax ; ð5Þ
dynamic equations: where Bmin ; Bmax 2 R. The maximum battery capacity, Bmax , is equal
 to the rated capacity. The minimum capacity, Bmin , is determined
ð1  sÞbðtÞ þ ðg b ðtÞ þ bb ðtÞÞDt; ! charge
bðt þ 1Þ ¼ ð1Þ from the Depth of Discharge (DoD) which is used to describe how
ð1  sÞbðtÞ  ðbc ðtÞ þ be ðtÞÞDt; ! discharge
deeply the battery is discharged [6]:
where bðtÞ 2 R represents the amount of energy stored at time step
Bmin ¼ ð1  DoDÞ  Bmax : ð6Þ
t; bb ðtÞ 2 R; gðtÞ 2 R; bc ðtÞ 2 R; be ðtÞ 2 R are the electrical power
quantities previously described3 (see also Fig. 1) and s denotes the Furthermore, the rate of battery charge/discharge at time t
hourly self-discharge decay and is equal to 104 [25]. Henceforth, needs also to remain between some bounds as described by the fol-
we consider discrete dynamics with a sampling time Dt of one hour. lowing condition:
Dmin 6 DbðtÞ 6 Dmax ; ð7Þ
Remark 2. This switching between charge and discharge derives
from a physical limitation of the battery: in a real implementation where Dmin ; Dmax 2 R and DbðtÞ ¼ bðtÞ  bðt  1Þ represents the bat-
it is not possible to execute both operations simultaneously. Hence, tery charge variation.
the battery dynamics alternates between these two modes (see, for Next, we consider the limitations on the generator power taken
instance [6,2] which consider a similar dynamic model for the by the battery (as previously mentioned, it might be that the bat-
battery). tery cannot take the entire output provided by the generator
Consider also the generator system, whose dynamics can be because the battery is already full, or charge bound is reached,
described as: etc.):

gðt þ 1Þ ¼ f ðgðtÞ; v ðtÞÞ; ð2Þ 0 6 g b ðtÞ 6 gðtÞ: ð8Þ

where f ð; Þ : R ! R is the dynamical function, gðtÞ 2 R the electri-

cal power given by the generator and v ðtÞ 2 R the wind speed at Remark 3. Note that g b ðtÞ denotes the actual power drawn from
time step t (here we assume that the wind velocity remains con- the generator. While the generator can provide at time instant t a
stant along a sampling interval). In practice, f ð; Þ is nonlinear, that quantity gðtÞ of electrical power it might be that this cannot be
is, the output of the generator depends nonlinearly on the wind absorbed by the battery. The reasons for this fact are that the
speed and can be roughly partitioned into three regimes of func- battery is either in discharge mode and cannot accept any charging
input (and hence g b ðtÞ ¼ 0) or that the battery is close to being
tioning (starting-up, nominal functioning and dangerous wind lev-
els) as described by the following equation [17,30] and also shown fully loaded.4
in Fig. 2(a): Finally, constraints on the amount of electricity transmitted to
8 and from the battery as well as to the consumer can be imposed
>
> 0; if v ðtÞ < v ci ; (basically, these limitations stand from physical limits on the
>
>
< P  v ðtÞv ci  Dt; if v 6 v ðtÞ < v ; energy transfer: e.g., the transformer which links the external grid
r v r v ci ci r
gðtÞ ¼ ð3Þ with the microgrid may not send more power than an a priori
>
> Pr  Dt; if v r 6 v ðtÞ < v co ;
>
> given amount):
:
0; if v ðtÞ > v co ;
U min 6 uðtÞ 6 U max ; ð9Þ
where v ci [m/s], v r [m/s] and v co [m/s] are the cut-in, rated and cut-
off wind speeds, respectively, and Pr [W] is the rated power of the

2 4
We consider an important and realistic physical limitation that the battery can Note that we have not considered how the difference between total and used
only charge or discharge at a given moment of time. generator power output is dealt with. For a wind turbine we may try to slow down
3
Note that g b ðtÞ and be ðtÞ are the ‘‘charge’’ inputs of the battery and bc ðtÞ and be ðtÞ the generator block such that only the desired quantity of power is produced or,
are the ‘‘discharge’’ outputs. dissipate it through a resistor.
402 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409

Fig. 2. Reference power curve for the wind turbine as a function of the wind speed. (a) Nominal power generator curve. (b) Uncertainty range of power generator curve.

T T T T
where U min ; U max 2 R4 and uðtÞ ¼ ½pT ðtÞ be ðtÞ bc ðtÞ bb ðtÞ repre-
sents the vector of control signals.

Cost function

In this section the cost function of the optimization problem is

described.
Denote by eðtÞ 2 R the electricity market price at time step t.
Then, the total energy cost at time t is:
CðtÞ ¼ eðtÞ  Dt½bb ðtÞ þ pðtÞ  be ðtÞ; ð10Þ
where a small value of CðtÞ means small energy drawn from the
external grid.
The reference profiles used throught the paper are described in
the next section.
Fig. 3. Consumer load profile.

Reference profiles
the numerical data taken from [10]. As in the previous case, we
All the elements of the electrical system are characterized by illustrate both the original one year grid and the magnified grid.
certain profiles of reference. Arguably, the most important is the Replacing the wind profile in (3) we obtain the power generator
profile characterizing the consumer load which we denote as profile as depicted in Fig. 5(a). Note that when considering uncer-
dðtÞ. Taking into account the weekly, daily and hourly variability tainty in the parameters of the power curve function (3) (see also
it is possible to predict a reference load. Therefore, in order to pre- Fig. 2b), we obtain an uncertainty range profile for the power gen-
dict the reference load of the consumer, we consider a top-down erator as in Fig. 5b.
approach based on available statistical measurements of electricity Lastly, we need to provide the evolution of the prices eðtÞ on the
consumption (the real numerical data of a reliability test system electricity market. Again, we use existing historical data of market
considered in this paper are found in [10]). prices [10]: Fig. 6 depicts the evolution of the price on the electric-
Considering r peak
w (%) the weakly peak of power demand in per- ity market in a day (we only show the price evolution for a day
cent of annual peak, r peak
d (%) the daily peak of power demand in because in this simplified model we assume that each day has
percent of weekly peak, rpeak
h (%) the hourly peak of power demand the same profile).
peak
in percent of daily peak and d [W] the total electrical power
demand over a year, then the consumer load satisfies the following
relation:

r peak ðlÞ rpeak ðjÞ rpeak

w ðiÞ peak
dðtÞ ¼ P24h peak  P7 d peak  P52 peak  d  Dt; ð11Þ
l¼1 r h ðlÞ j¼1 r d ðjÞ i¼1 r w ðiÞ

where l ¼ 1 : 24 are the hours indices in a day, j ¼ 1 : 7, the days

indices in a week and i ¼ 1 : 52, the week indices in a year. Note
that the consumer load profile takes into account seasonal numer-
ical data; thus, it can be used to adapt to any system peaking season
that one may desire to model.
Fig. 3 depicts the reference load of the consumer as described in
(11). Note that in the Figure we provide the plot for one year load
profile and also a magnified plot for 150 h.
Furthermore, we also need to specify a profile for the generator
power output. Note that this output depends directly on the wind
profile, which has to be estimated from meteorological data and
information. Fig. 4 depicts the considered wind speed profile with Fig. 4. Wind speed profile.
 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409 403

Fig. 5. Power generator profile. (a) Generator power over one year. (b) Generator power profile for 150 h interval under uncertainty.

over a finite horizon of N p discrete steps of state dynamics. Here

V f ðÞ : Rn ! R represents the terminal cost function, V n ðÞ : Rn 
Rm ! R, the cost per each step in the horizon, fx ð; Þ describes the
evolution of the system state trajectory, hðÞ the constraints in a
general (input-state-parameters) form and hf ðÞ the terminal
constraint with qðÞ a vector of parameters which can include, for
example, the profile of interest to be followed along the prediction
horizon.
This general setting will be tailored in the next subsection for
the system model given in Section ‘System and model description’.
That is, an optimal controller will be provided, which takes into
account the microgrid limitations and consumer needs whilst
minimizing the cost of the energy taken from the external grid.

Microgrid management by battery scheduling

Fig. 6. Electricity market price profile.

The goal is to control the battery such that the electricity taken
Optimization-based control for battery scheduling from the external grid is minimized. While the formulation (12)
allows for arbitrary dynamics, in practice a switching dynamics
This section presents first the basic elements of a general opti- as the one in (1) increases significantly the numerical difficulty.
mization-based control problem and then specifies it in detail for Consequently, the switching dynamics (1) is transformed into the
the reliable energy management of the microgrid system. Linear-Time Invariant (LTI) dynamics:
bðt þ 1Þ ¼ ð1  sÞbðtÞ þ ðg b ðtÞ þ bb ðtÞ  bc ðtÞ  be ðtÞÞDt; ð14Þ
Principles of optimization-based control
with the addition of supplementary constraints which take into
Generally, optimization-based control stands for a control account the physical limitation that the battery can only charge
design which optimizes some criterion (usually stability, robust- and discharge at a given moment of time and translated as mixed-
ness and/or performance objectives) in the presence of various lim- integer conditions:
itations (state/input constraints, dynamic profiles to be followed, 8
> bc ðtÞ < M aðtÞ;
etc.). This is a broad definition which actually can cover the classical
>
>
< be ðtÞ < M aðtÞ;
optimal control, the Linear Matrix Inequality (LMI)-based tech- ð15Þ
>
> bb ðtÞ < Mð1  aðtÞÞ;
niques, model predictive control or interpolation-based techniques. >
:
We embrace the powerful MPC technique, which considers a g b ðtÞ < Mð1  aðtÞÞ;
cost function over a finite prediction horizon and provides a with M a constant chosen appropriately (that is, significantly larger
sequence of inputs which respect the given constraints and mini- than the rest of the variables and playing the role of a relaxation
mize the cost. The control action uðt j tÞ for a given state xðt j tÞ is constant), and aðtÞ 2 f0; 1g the auxiliary binary variable (which
obtained from the control sequence u , fuðt j tÞ; uðt þ 1 j tÞ; . . . ; can activate or not the relaxation).
uðt þ N p  1 j tÞg as the result of the optimization problem: The mixed integer conditions ensure the complementarity of
N
X p 1 the input signals. The battery ‘‘charge’’ signals can be non-zero
arg min V f ðxðt þ N p j tÞ; qðt þ Np j tÞÞ þ V n ðxðt þ s only if the ‘‘discharge’’ signals are held at zero, and vice versa. It
u s¼0 follows then, that by adding these additional constraints which
j tÞ; uðt þ s j tÞ; qðt þ s j tÞÞ; ð12Þ contain binary variables the problem becomes mixed-integer and
is put in a form amendable to numerical implementations.
subject to the set of constraints:
8
Remark 4. As explained before, by adding binary variables we can
< xðt þ s þ 1 j tÞ ¼ fx ðxðt þ s j tÞ; uðt þ s j tÞÞ; s ¼ 0 : Np  1;
>
hðxðt þ s j tÞ; uðt þ s j tÞ; qðt þ s j tÞÞ 6 0; s ¼ 0 : Np  1; enforce a switching behavior for the battery. Nonetheless, when
>
: these new constraints are not active (e.g., when aðtÞ ¼ 1 in (15))
hf ðxðt þ Np j tÞ; qðt þ Np j tÞÞ 6 0;
we want them to be redundant. Input bc ðtÞ is already upper
ð13Þ bounded in (9), we do not wish that bc ðtÞ < M  1 imposes a harsher
404 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409

bound. Therefore, we need to consider a sufficiently large scalar M that holding now and selling in the future is economically
such that constraints (15) are redundant with respect to the advantageous.
original ones (i.e., the constraints described by (9)).
Now that a more flexible formulation for the battery dynamics Further remarks and results
was provided ((14) and (15)), we can concentrate in the following
on the optimization problem formulation. This section provides extensions to the base MPC framework,
For a reliable energy management of the microgrid by battery which prove its efficiency and show its flexibility to cope with var-
scheduling, we consider the recursive construction of an opti- ious situations. For dealing with a reliable energy management
mal open-loop control sequence u ¼ fuðt j tÞ; uðt þ 1 j tÞ; . . . ; uðtþ problem under uncertainty, a cost function with time-varying
T T T T
N p  1 j tÞg, with uðt j tÞ ¼ ½pT ðt j tÞ be ðt j tÞ bc ðt j tÞ bb ðt j tÞ as in weights can be formulated. Also, a fault tolerance implementation
(9) over a finite constrained receding horizon, which leads to a may be included within the MPC framework.
feedback control policy by the effective application of the first
control action as system input: Cost function with time-varying weights
NX
p 1

u ¼ arg min Cðt þ s j tÞ; ð16Þ Note that the increase of the prediction horizon length in (16)
uðtjtÞ;uðtþ1jtÞ;...;uðtþN p 1jtÞ
s¼0 means that the optimization problem minimizes the cost along this
entire horizon. The cost function, in turn, may be affected by uncer-
subject to the set of constraints: tainties in a way that cost values further away along the prediction
8
> bðt þ s þ 1 j tÞ ¼ ð1  sÞbðt þ s j tÞ þ ðg b ðt þ s j tÞ þ bb ðt þ s j tÞ horizon are less reliable than the ones closer to the present. A solu-
>
>
>
> þbc ðt þ s j tÞ þ be ðt þ s j tÞÞDt; tion is to assign in the optimization problem less importance to the
>
>
>
>
>
>
> bc ðt þ s j tÞ < Maðt þ s j tÞ; s ¼ 0 : N p  1; cost values which are further in the future. This can be done by
>
>
>
>
> be ðt þ s j tÞ < Maðt þ s j tÞ; s ¼ 0 : N p  1; varying the weight associated to each cost value over the predic-
>
>
>
>
> bb ðt þ s j tÞ < Mð1  aðt þ s j tÞÞ; s ¼ 0 : N p  1; tion horizon. More precisely, the cost function in (16) can be
>
>
>
< g b ðt þ s j tÞ < Mð1  aðt þ s j tÞÞ; s ¼ 0 : N p  1;
> rewritten as:
gðt þ s þ 1 j tÞ ¼ f ðgðt þ s j tÞ; v ðt þ s j tÞÞ; s ¼ 0 : N p  1; NX
p 1
>
> 1
>
> 0 6 g b ðt þ s j tÞ 6 gðt þ s j tÞ; s ¼ 0 : N p  1;
>
> Cðt þ s j tÞ : ð17Þ
>
> bc ðt þ s j tÞ þ pðt þ s j tÞ P dðt þ s j tÞ; s ¼ 0 : N p  1; 1þst
>
> s¼0
>
>
>
> Bmin 6 bðt þ s j tÞ 6 Bmax ; s ¼ 0 : N p  1;
>
> Basically, we have that the cost at time k becomes less and less
>
>
> Dmin 6 Dbðt þ s j tÞ 6 Dmax ; s ¼ 0 : N p  1;
>
>
> 1
important as k increases because the term 1þkt is decreasing. There-
> U 6 uðt þ s j tÞ 6 U ; s ¼ 0 : N  1;
>
>
> min max p
: fore, uncertainty further out in the prediction horizon has a small
Cðt þ s j tÞ ¼ eðt þ s j tÞ  Dt½bb ðt þ s j tÞ þ pðt þ s j tÞ  be ðt þ s j tÞ; s ¼ 0 : N p  1;
influence on the total cost. Of course, some other time-decreasing
where Np is the step length of the prediction horizon. The profiles terms can be used, e.g. (see, for example [13]):
introduced in Section ‘Reference profiles’ appear as parameters here NX
p 1

(e.g., the consumer load dðt j tÞ, the energy cost eðt j tÞ, etc.). With Cðt þ s j tÞcst ; ð18Þ
respect to the general formulation (12), here we consider the run- s¼0

ning cost equal to the terminal cost so that the terminal step does where c 2 ð0; 1Þ. The weight value decreases exponentially, with
not appear explicitly. The cost is variable in the sense that the speed depending on the value taken by the rate c.
weight matrix may change with time due to the variation in energy
price,5 but otherwise is linear in the input values.6 Therefore, we
Fault tolerant control
may denote this problem as one of Mixed-Integer Linear Program-
ming (MILP). For this type of problems, efficient solvers exist and a
In the following, we consider failures of the wind power gener-
reasonably large prediction horizon can be used [29,31].
ator that lead to output outage (i.e., gðtÞ ! 0). These failure events
It is important to mention that the increase of the prediction
have different characteristics depending on the wind speed: more
horizon will lead to a decrease of the electricity cost taken from
specifically, we differentiate between ‘‘normal-speed’’ and ‘‘high-
the external grid. Having a large horizon means that the effects
speed’’ events, which are characterized by different times to failure
of a schedule can be analyzed over a longer period, thus increasing
and recovery, as detailed in Table 1 taken from [18] and also con-
the versatility of the scheme. This may mean that the scheduler
sidered in [22]. The Mean Time to Failure (MTTF) and Mean Time to
takes the ‘‘longer view’’ to look for an optimal plan and discards
Repair (MTTR) are 2183 and 97 h for normal wind speed condi-
‘‘shortsighted’’ decisions (for example the battery will not sell
tions, respectively, and 364 and 364 h for high-speed conditions,
energy now if it knows that the consumer will need it in medium
respectively. This distinction accounts for the fact that during nor-
term, as given by the chosen prediction horizon).
mal wind speed conditions, faults occur relatively rarely and cause
Note that the way and timing in which the battery charges and
minor damage to the wind generator, which can be repaired in rel-
discharges its energy in the optimally chosen solution must be
atively short time (i.e., 97 h). During extreme wind conditions, the
seen within the whole sequence of scheduling decisions taken over
wind generator suffers additional stress, which gives a shorter
the given horizon. Hence, it might be possible that a decision
MTTF and faults requiring longer time for repair (i.e., 364 h).
which seems expensive at the moment when is taken is actually
optimal in the long run. For example, the battery can store energy
instead of giving it to the user (and thus force it to buy energy from Table 1
the grid) at the current moment of time, if the price profile ‘‘tells’’ Failure and repair rates of the renewable generator.

Expected Normal wind Extreme wind speed conditions

5
Note that we consider the same price for the selling and buying electricity. times (h) speed conditions (i.e., (i.e., v(t) 30 m/s)
6
If the amount required by the user represents an important percentage from the v(t) [15, 30) m/s)
external available power, then eðtÞ will be correlated with pðtÞ (i.e., increased demand MTTF 2183 364
will reflect in price increase). Note that MPC approach can deal with this by modifying MTTR 97 364
the definition of the cost function (16).
 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409 405

Table 2
Numerical data for the microgrid components.

Battery parameters
M Bmin [W h] Bmax [W h] Dmin [W] Dmax [W]
104 5:103 3:103 6:103 3:103 3:103
Power generator parameters
Pr [W] vci [m/s] vr [m/s] vco [m/s]
6:103 3 12 20

Uncertainty in the power generator parameters

P r [W] P r [W] v ci [m/s] v ci [m/s] vr [m/s] vr [m/s] v co [m/s] v co [m/s]
4:103 75.102 2.3 4.4 10 15 19 25

Prediction horizon Control input constraints

Np = 10 Umin ¼ ½0 0 0 0T , Umax ¼ ½1 6 1 6T 103

To cope with this in the proposed MPC formulation, we need to Next, in Fig. 9 we illustrate both gðtÞ, the power generated by
set up a Fault Tolerant Control (FTC) scheme capable of ensuring a the windturbine and g b ðtÞ, the value actually taken by the battery.
normal functioning of the microgrid in the presence of faults via As expected, notice that the latter is sometimes less than the for-
control reconfiguration. Alternatively, it has to provide at least a mer. This happens whenever the battery is in discharge mode or
graceful degradation, that is, provide enough advance warning close to being full. Notice also the tube around the nominal curve.
and a way for winding down safely the microgrid components. This denotes that the actual values which can be given by the gen-
The first step in FTC is represented by the detection of the fault erator output will vary within a range. This is useful in the MPC
events and their subsequent isolation. This is done based on wind algorithms where we have to predict future values. For this simu-
power generator output comparison and wind speed measure- lation we took the nominal values (the ‘‘middle’’ of the uncertainty
ment. The measured output is compared with the normal-func- range), but other choices are valid (e.g., play it safe and choose the
tioning tube (according to the transfer function (3) for the lowest values from the uncertainty range).7
current wind speed), and if it falls out of this expected range then In Fig. 10 we show that the consumer demand is satisfied at all
we have detection. Next, the measurement of the wind speed times. This is done through a combination of battery and external
allows to isolate between the specific fault modes (normal or grid power. It can be seen that the sum of their inputs is always at
extreme wind speed conditions). least equal to the consumer demand. This means that the con-
From the point of view of the scheduling logic, the fault occur- sumer is always receiving the electrical power demanded. Also,
rences are external, in the sense that they do not affect the battery with square line we plot how much power is drawn overall from
dynamics but only the input that the battery receives from the the grid pðtÞ þ bb ðtÞ  be ðtÞ (power given from the external grid to
generator. Hence, once a fault is detected the only thing to do is the consumer plus power given from the grid to the battery minus
to update the profile prediction. Upon detection, the MPC ‘‘knows’’ power given from the battery to the grid).
that it should not expect power from the generator and has to rely Finally, note the way we represent the various signals in the
solely on the external grid. Once the windturbine switches back to above plots: the input control signals are assumed to be constant
healthy functioning, the generator returns to normal functioning from the current instant on the current interval, t to the next
and the profile prediction returns to the nominal one. instant t þ 1, the output signals are assumed to be constant on
the previous interval, from the previous time instant t  1 to t
Simulation results and comparison and the charge of the battery evolves piecewise linear.

In this section, we first present simulation results for a test sys- Comparison results
tem (RTS-96) developed for bulk power system reliability evalua-
tion studies [10]. Then, a comparison of our approach with In the following, we consider the microgrid scheme of Fig. 11
another approach based on agent-based modeling and reinforce- taken from.
ment learning [22] is provided. [22] and proceed with some simulations in order to investigate
the benefits of the proposed MPC framework. The scheme in Fig. 11
differs from our more general scheme in Fig. 1 in that we consider a
Simulation results
more realistic model, that is, the battery that can send and take
energy directly to the external grid, variable values for the electric-
Consider the microgrid architecture illustrated in Fig. 1 with the
ity price and generator output.
dynamic model (14) for the battery, the discretization step
Dt ¼ 1 h and the reference profiles described in Section ‘Reference
Remark 5. Note that an even more realistic benchmark would
profiles’. The numerical values of the parameters used for the sim-
imply the addition of electrical power lines from the renewable
ulations are presented in Table 2.
generator to the consumer and to the external grid, respectively.
Fig. 7 illustrates the charge stored in the battery bðtÞ along the
This would allow the possibility of selling electricity to the external
simulation horizon (i.e., 150 h) and also the charge variation DbðtÞ.
grid when the level of charge in the battery is sufficient for
Note that whenever there is a discharge (the value of the variation
covering local needs, thus avoiding energy waste and empowering
is negative), the value of the charge in the battery decreases. The
the renewable energy source as the major power supply in the
reverse holds for the charging process.
Fig. 8 illustrates the various control signals. It can be seen that
there is a complex interplay between the battery and its neighbor- 7
Note that the MPC strategy employed can handle perturbations and disturbances
ing components. Particularly, at some instants, it appears that it is even if not explicitly taken into account in the model description. In particular, noise
more efficient to give and take energy from the external grid rather disturbances and variation in the generator output are considered and following the
than send it to the user. simulation we observe that they are dealt with satisfactorily.
406 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409

Fig. 7. Battery charge level and its variation.

Fig. 8. Control signals.

Fig. 9. Generator power vs. generator power sent to the battery and the consumer demands.

microgrid system. Within the methodology proposed, this would related simulations results presented in Section ‘Simulation results
entail some modifications in the constraints formulation of and comparison’, with no conceptual changes from the method-
Section ‘Constraints’, in the optimization problem 16 and the ological viewpoint.
 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409 407

Fig. 10. Consumer demands and battery power usage.

Table 3
Numerical data considered in [22].

Parameters Time steps

t t+1 t+2
Scenario 1 with initial battery charge Bmin = 3000 W h
Wind power output g(t) [W h] 1200 1200 1200
Load d(t) [W h] 4400 5200 5200
Scenario 2 with initial battery charge Bmin = 3000 W h
Wind power output g(t) [W h] 6000 4800 4800
Load d(t) [W h] 2800 2800 2800

Fig. 11. Microgrid architecture in [22]. where eðtÞ is assumed constant and equal to one price unit per watt
for all time periods as in [22].
The same reference profiles described in Section ‘Reference pro-
The increase of the utilization of electricity from the battery is
files’ and the numerical values of the parameters shown in Table 3
estimated by V 0 , which is defined as the ratio of the cumulative
are considered for exemplification, as they were used in [22]
power taken from the battery to the yearly cumulative load. The
within a reinforcement learning algorithm on medium-term (2
increase of the utilization rate of the wind turbine is evaluated
steps-ahead) scenarios for providing the schedule of battery oper-
by V 1 , which is defined as the ratio of the yearly cumulative power
ation: at time step t, a sequence of three input is computed and
taken from the wind generator to the yearly cumulative available
successively applied to the battery; only then the appropriate sub-
wind power output. Finally, parameter E indicates the cumulative
sequent charging and discharging actions on the battery are iden-
annual expenses for power purchases from the external grid. Note
tified. The scenarios consider annual peak and minimum loads of
that in our proposed MPC algorithm the cost function is a trunca-
6000 W h and 2000 W h, respectively, maximum battery charge
tion of E along the current prediction horizon.
Bmax ¼ 6000 W h, and the power generator parameters of Table 2.
For a year long simulation with the data of Table 3 in [22] the
For meaningful comparison between the two approaches we
minimal and maximal values8 for these indicators are in the forth
use the following performance indicators [22]:
row of Table 4. In the sixth row are the values obtained by our algo-
Psmax
bc ðtÞ rithm (with a prediction horizon Np ¼ 10).
V 0 ¼ Pt¼0
smax ; ð19Þ While the different nature of the algorithms makes a point-by-
t¼0 dðtÞ
Psmax point comparison hard to accomplish, we note that under similar
g b ðtÞ
V 1 ¼ Pt¼0
smax ; ð20Þ
t¼0 gðtÞ
!
X
smax X
smax 8
They are minimal and maximal values of the performance indicators obtained
E¼ dðtÞ  bc ðtÞ eðtÞ; ð21Þ from 40 repetitions of the simulation, performed in order to validate the efficiency of
t¼0 t¼0 the learning process [22].
408 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409

Table 4
Values of the performance indicators (21) obtain with two different algorithms.

V0 V1 E
Minimal Maximal Minimal Maximal Minimal Maximal
Values of the performance indicators obtained in [22] with a reinforcement learning algorithm
0.102 0.109 0.176 0.186 2:863  107 2:890  107
Values of the performance indicators obtained with the proposed MPC algorithm
0.196 0.389 1:807  106

conditions the present MPC algorithm performs noticeably better Acknowledgments

in what regards the criteria (21). This can be primarily justified
by the longer prediction horizon used here. Moreover, one of the The authors would like to thank Elizaveta Kuznetsova, PhD stu-
strengths of our MPC algorithm is that it can be expanded easily dent within the Chair on Systems Science and the Energetic Chal-
to large horizons without the manual intervention required by lenge for the fruitful discussions during this work.
the reinforcement learning method.
Finally, several observations can be made: References

1. The MPC algorithm has a variable prediction horizon which [1] Alanne K, Saari A. Distributed energy generation and sustainable development.
allows for increasingly optimal input values; this is different Renew Sustain Energy Rev 2006;10(6):539–58.
[2] Chabaud A, Eynard J, Grieu S. Optimal design of energy production and storage
from [22] where the prediction horizon is always fixed at 2 systems in buildings. In: Proceedings of the 4th inverse problems, design and
steps. Both approaches have an exponential increase in number optimization symposium; 2013. p. 1–10.
of branches (due to the charge/discharge switching at each [3] Chen C, Duan S, Cai T, Liu B, Hu G. Smart energy management system for
optimal microgrid economic operation. IET Renew Power Gener 2011;5(3):
step): the advantage of the MPC scheme is that it considers this
258–67.
implicitly through the addition of binary variables, so that [4] Colson C, Nehrir M, Gunderson R. Multi-agent microgrid power management.
branches are ‘‘automatically’’ created and processed by the In: Proceedings of the 18th IFAC world congress; 2010. p. 77–82.
MILP solver. [5] Costa LM, Kariniotakis G. A stochastic dynamic programming model for
optimal use of local energy resources in a market environment. In: IEEE
2. Another advantage is that the MPC framework models and uses Lausanne power tech; 2007. p. 449–54.
the dynamic behavior of the microgrid components (in particu- [6] Diaf S, Diaf D, Belhamel M, Haddadi M, Louche A. A methodology for optimal
lar, the battery charge and discharge model). sizing of autonomous hybrid PV/wind system. Energy Policy 2007;35(11):
5708–18.
3. In the MPC scheme, it is easy to add constraints regardless of [7] Dou C, Liu B. Transient control for micro-grid with multiple distributed
their nature (convex or non-convex). generations based on hybrid system theory. Int J Electr Power Energy Syst
4. The proposed predictive control algorithm is based on a more 2012;42(1):408–17.
[8] Edlund K, Bendtsen J, Jørgensen J. Hierarchical model-based predictive control
realistic model: variable values for the electricity price and gen- of a power plant portfolio. Contr Eng Practice 2011;19(10):1126–36.
erator output, battery that can also send and take energy [9] El-Sharkh M, Rahman A, Alam M. Evolutionary programming-based
directly to the external grid. methodology for economical output power from PEM fuel cell for micro-grid
application. J Power Sources 2005;139(1):165–9.
5. Still, the MPC remains sensitive to changes in the profile values, [10] Grigg C, Wong P, Albrecht P, Allan R, Bhavaraju M, Billinton R, et al. The IEEE
the effect of perturbations is cumulative and the numerical reliability test system – 1996. A report prepared by the reliability test system
computation (due to the presence of binary variables) can be task force of the application of probability methods subcommittee. IEEE Trans
Power Syst 1999;14(3):1010–20.
significant.
[11] Handschin E, Neise F, Neumann H, Schultz R. Optimal operation of dispersed
generation under uncertainty using mathematical programming. Int J Electr
Power Energy Syst 2006;28(9):618–26.
[12] Hooshmand A, Malki H, Mohammadpour J. Power flow management of
Conclusions
microgrid networks using model predictive control. Comput Math Appl
2012;64(5):869–76.
In this paper, we propose an approach for reliable microgrid [13] Hovd M, Braatz R. Handling state and output constraints in MPC using time-
energy management based on receding horizon control to mini- dependent weights. In: Proceedings of the 20th American control conference,
vol. 3; 2001. p. 2418–23.
mize a defined cost function. A scheduling of battery usage is [14] Hovgaard T, Larsen L, Jorgensen J. Robust economic MPC for a power
determined. The predictive control framework allows to naturally management scenario with uncertainties. In: Proceedings of the 50th IEEE
take into consideration variable cost values, power consumption conference on decision and control and European control conference; 2011. p.
1515–20.
and generation profiles, as well as functional constraints under [15] Jimeno J, Anduaga J, Oyarzabal J, de Muro A. Architecture of a microgrid energy
uncertainty due to variations in the power generator model. The management system. Eur Trans Electr Power 2011;21(2):1142–58.
simulation results using real numerical data for a reliability test [16] Jun Z, Junfeng L, Jie W, Ngan H. A multi-agent solution to energy management
in hybrid renewable energy generation system. Renew Energy 2011;36(5):
system have proved the feasibility of the proposed approach. With 1352–63.
the considered management strategy, a good equilibrium between [17] Justus C, Hargraves W, Yalcin A. Nationwide assessment of potential output
decentralized energy production, energy needs and integration from wind-powered generators. J Appl Meteorol 1976;15(7):673–8.
[18] Karki R, Billinton R. Reliability/cost implications of PV and wind energy
into the grid can be found. utilization in small isolated power systems. IEEE Trans Energy Convers
As future work, we envision several directions of improvement 2001;16(4):368–73.
for the MPC scheme (robustness by taking explicit consideration of [19] Katiraei F, Iravani M. Power management strategies for a microgrid with
multiple distributed generation units. IEEE Trans Power Syst 2006;21(4):
the disturbances, set-theoretic description for fault tolerant
1821–31.
components, improvements in the cost function formulation and [20] Krause T, Beck E, Cherkaoui R, Germond A, Andersson G, Ernst D. A comparison
constraints, etc.). Furthermore, at a higher level, we envision of Nash equilibria analysis and agent-based modelling for power markets. Int J
to analyze interconnected microgrid systems and discuss Electr Power Energy Syst 2006;28(9):599–607.
[21] Kuznetsova E, Culver K, Zio E. Complexity and vulnerability of smartgrid
centralized/distributed/decentralized algorithms for scheduling systems. In: Proceedings of the European safety and reliability conference;
control. 2011. p. 2474–82.
 I. Prodan, E. Zio / Electrical Power and Energy Systems 61 (2014) 399–409 409

[22] Kuznetsova E, Li YF, Ruiz C, Zio E, Ault G, Bell K. Reinforcement learning for [31] Prodan I, Stoican F, Olaru S, Niculescu SI. Enhancements on the hyperplanes
microgrid energy management. Energy J 2013;59(0):133–46. arrangements in mixed-integer techniques. J Optimiz Theory Appl 2012;
[23] Logenthiran T, Srinivasan D, Wong D. Multi-agent coordination for der in 154(2):549–72.
microgrid. In: Proceedings of the IEEE international conference on sustainable [32] Prodan I, Zio E. Predictive control for a reliable microgrid energy management
energy technologies; 2008. p. 77–82. under uncertainties. In: Proceedings of the European safety and reliability
[24] Lund H. Renewable energy strategies for sustainable development. Energy conference; 2013. p. 2919–24.
2007;32(6):912–9. [33] Rawlings J, Mayne D. Model Predictive Control: Theory and Design. Nob Hill
[25] McEvoy A, Markvart T, Castaner L, Markvart T, Castaner L. Practical handbook Publishing; 2009.
of photovoltaics: fundamentals and applications: fundamentals and [34] Richalet J, O’Donovan D. Predictive functional control: principles and
applications. Elsevier; 2003. industrial applications. Springer; 2009.
[26] Mohamed F, Koivo H. System modelling and online optimal management of [35] Tan X, Li Q, Wang H. Advances and trends of energy storage technology in
microgrid using mesh adaptive direct search. Int J Electr Power Energy Syst microgrid. Int J Electr Power Energy Syst 2013;44(1):179–91.
2010;32(5):398–407. [36] Tsikalakis A, Hatziargyriou N. Centralized control for optimizing microgrids
[27] Negenborn R, De Schutter B, Hellendoorn J. Multi-agent model predictive operation. In: IEEE power and energy society general meeting; 2011. p. 1–8.
control for transportation networks: serial versus parallel schemes. Eng Appl [37] Weidlich A, Veit D. A critical survey of agent-based wholesale electricity
Artif Intell 2008;21(3):353–66. market models. Energy Econ 2008;30(4):1728–59.
[28] Negenborn R, Houwing M, De Schutter B, Hellendoorn J. Model predictive [38] Xie L, Ilic M. Model predictive economic/environmental dispatch of power
control for residential energy resources using a mixed-logical dynamic model. systems with intermittent resources. In: IEEE power & energy society general
In: International conference on networking, sensing and control; 2009. meeting; 2009. p. 1–6.
p. 702–7. [39] Zervas P, Sarimveis H, Palyvos J, Markatos N. Model-based optimal control of a
[29] Neumaier A, Shcherbina O. Safe bounds in linear and mixed-integer linear hybrid power generation system consisting of photovoltaic arrays and fuel
programming. Math Program 2004;99(2):283–96. cells. J Power Sources 2008;181(2):327–38.
[30] Østergaard KZ, Stoustrup J, Brath P. Linear parameter varying control of wind [40] Zio E, Aven T. Uncertainties in smart grids behavior and modeling: what are
turbines covering both partial load and full load conditions. Int J Robust the risks and vulnerabilities? How to analyze them? Energy Policy 2011;
Nonlinear Contr 2009;19(1):92–116. 39(10):6308–20.