Yoash Levron Juri Belikov

½ The Andrew and Erna Viterbi Faculty of Electrical ½ Department of Software Science, Tallinn Uni-
Engineering, Technion—Israel Institute of Technology, versity of Technology, Akadeemia tee 15a, 12618
Haifa 3200003, Israel Tallinn, Estonia
R yoashl@ee.technion.ac.il R juri.belikov@taltech.ee

Lecture 4: Control of Energy Storage Devices

This lecture focuses on management and control of energy storage devices. We will consider
several examples in which these devices are used for energy balancing, load leveling, peak
shaving, and energy trading.

Basic parameters of storage devices

Two key parameters of energy storage devices are energy density, which is the capacity per
unit mass or volume, and power density, which is the maximum output power per unit mass
or volume. Common energy storage technologies include:

3 Mechanical: hydroelectric energy storage (pumped storage)1 , flywheels, compressed

air, thermal storage;
3 Electrochemical: rechargeable batteries, flow batteries, fuel cells;
3 Electrical: capacitors, inductors, superconducting magnetic coils.

1000
Fuel cells
Energy density [W·h/kg]

100
Conventional
batteries
10
Ultracapacitors
1

0.1
Conventional
capacitors
0.01
10 100 1000 10000
Power density [W/kg]

Figure 1: Energy density vs. power density.

Storage devices with high power density are crucial for stability of electric power systems.
A classic example is the kinetic energy stored in the rotors of synchronous generators. As ex-
plained in previous lectures, this kinetic energy supports instantaneous changes in the load,
and is essential for regulating the grid frequency. Today, one possible challenge associated
with renewable energy integration is the low rotational inertia of renewable energy sources,
which in several cases may lead to stability problems [1, 2]. To address this challenge the
total system inertia may be increased by means of additional storage devices.
1
Today the largest form of energy storage used in power grids is pumped storage in hydroelectric dams.

Series of lectures on power system dynamics. The lectures are freely available on
https://a-lab.ee/projects/dq0-dynamics
Lecture 4: Control of Energy Storage Devices 2

Storage devices with high capacity are mostly used for energy shifting and energy balancing.
The main idea is to store surplus energy at times when the power demand is low, and then
to use it when the main source cannot supply the energy needed, or when generation is
difficult or expensive. Typical applications in power systems include:

3 Energy balancing, Load leveling, or Peak shaving. In electric power systems the load is
constantly varying. Storage devices may be used to shift generation from times of peak
load to off-peak hours. This lowers the peak of the generated power, and improves the
overall system efficiency.
3 Renewable energy integration. Renewable sources provide variable power that is not
always matched to the load. If the energy generated by renewable sources cannot be
consumed immediately it can be stored and used later. Energy storage technologies
seem to be essential for large-scale integration of renewable sources [3].
3 Energy trading. Storage devices enable to buy energy at a low price, and then to sell
this energy at a higher price. In addition to generating profit this also helps to match
the power supply to the power demand, and stabilizes the energy cost.
3 Emergency preparedness. Storage devices may supply energy in case of a malfunction
in the generation or transmission systems. This function is vital for sensitive facilities
such as hospitals, military bases, etc.

Basic dynamic model of a storage device

Consider a grid-connected storage device, as shown in Fig. 2.

P (t)
=⇒
Storage
Grid
device

Figure 2: A grid-connected storage device.

A basic dynamic model of the device is

(
dE ηc (E, P )P, P ≥ 0,
= −1
(1)
dt ηd (E, P )P, P < 0,

where

3 E(t) is the stored energy;

3 P (t) is the total power flowing into the device;
3 ηc (E, P ) is the charging efficiency;
3 ηd (E, P ) is the discharging efficiency.

Both ηc (E, P ) and ηd (E, P ) have values in the range (0, 1].
As an example assume that P = −|P | and ηd = 0.8. In this case
dE 1
= P = −1.25|P |. (2)
dt 0.8
Lecture 4: Control of Energy Storage Devices 3

The energy derivative is higher (in absolute value) than the power flowing into the grid.
In addition, the round trip efficiency (ηr ) is the efficiency of a charge-discharge cycle. Con-
sider a device which is charged from Elow to Ehigh , and then discharged from Ehigh to Elow .
The round trip efficiency is the ratio between the energy extracted from the device to the
energy stored in the device: R
− P (t)<0 P (τ )dτ
ηr = R . (3)
P (t)≥0
P (τ )dτ
Changing the integration variable based on (1) yields
R Ehigh
Elow
ηd (E, P )dE
ηr = R Ehigh , (4)
Elow
ηc−1 (E, P )dE

and if ηc and ηd are constants then

R Ehigh
ηd Elow dE ηd (Ehigh − Elow )
ηr = R Ehigh = −1 = ηc ηd . (5)
ηc−1 Elow dE ηc (Ehigh − Elow )

In this case the round-trip efficiency is the multiplication of the charging and discharging
efficiencies, as expected.

Managing storage devices connected to non-controllable power sources

Consider a power system with a generator and a load that are non-controllable. A classic
example is a standalone system powered by a photovoltaic source, as found in many com-
mercial applications (streetlights, bus stations ...). Another example may be a remote area in
a power grid that includes mainly loads and renewable sources. In such systems the gener-
ated power is usually different from the momentary load, so a storage device must be used
for energy balancing.
As an example consider the simple power system shown in Fig. 3. The system includes a
photovoltaic array with a given output power Pg (t), a load with a given power PL (t), and a
storage device with power Ps (t). The storage device is assumed to be lossless.

Pg PL Load
PV array ⇒ ⇒
⇒

Ps

Figure 3: Solar cell connected to a storage device and load.

The generated power is given by

(
PL (t), if E ≥ Emax and Ppv (t) ≥ PL (t),
Pg (t) = (6)
Ppv (t), otherwise,

where Ppv (t) is the maximum power point of the photovoltaic array, and Emax is the capacity
of the storage device. According to this equation if the storage device is full then the surplus
Lecture 4: Control of Energy Storage Devices 4

power must be lost. As a result, the power flowing into the storage device is
(
0, if E ≥ Emax and Ppv (t) ≥ PL (t),
Ps (t) = Pg (t) − PL (t) = (7)
Ppv (t) − PL (t), otherwise,

and the stored energy is

Z t Z t
E(t) = Ps (τ )dτ = (Pg (τ ) − PL (τ )) dτ. (8)
0 0

Typical waveforms are illustrated in Fig. 4.

Ppv

PL

Time

Figure 4: Typical waveforms for a system with a storage device and a non-controllable power source.
If the storage device is full then the surplus power must be lost.

Managing storage devices connected to controllable power sources

Consider now a system consisting of generators, storage devices and loads, in which the
generated power can be controlled. In such a system it is necessary to decide at every mo-
ment how much energy should be generated, and how much energy should be stored. The
best mix is found by solving an optimization problem, in which the objective is to maximize
the efficiency or to minimize the overall cost. Typical applications include:

3 Grid energy storage. The storage device is connected to the grid, and operates alongside
traditional generators. This enables to minimize the total cost, the total fuel consump-
tion, or the peak of the generated power. The problem here is to decide how much
energy to store, and when to store it.
3 Hybrid electric vehicles. The storage device, often a battery, is connected to a fueled
engine. When the mechanical load is low energy is stored, and when the mechanical
load is high the stored energy is transferred to the load. This enables the main engine
to operate with a lower peak power and at a higher efficiency. The challenge here is
again to decide how much energy to store, and when to store it.

We will now develop the optimal energy management strategy in a system with a single
controllable generator and a single storage device, as shown in Fig. 5.
To simplify the analysis it is assumed that the load profile can be estimated with reasonable
accuracy, meaning that the power PL (t) consumed over the time interval [0, T ] is known.
This assumption is justified for some types of loads, but may not apply for others.
Lecture 4: Control of Energy Storage Devices 5

F (Pg (t))
fuel consumption
or any other cost
function
Pg (t)

stored
Ps (t)
control energy
E(t)
PL (t)

load

Figure 5: A general system consisting of a controllable generator (source), a load, and an energy
storage device.

The generator has an output power Pg (t) that can be controlled, and is characterized by a
cost function F (Pg ) that is twice differentiable, monotonically increasing and strictly convex.
This function may represent fuel consumption or another cost. The generated energy is
defined as Z t
Eg (t) = Pg (τ )dτ, (9)
0
and the total cost is Z T
Ftot = F (Pg (τ ))dτ. (10)
0
Furthermore we define the load energy as
Z t
EL (t) = PL (τ )dτ. (11)
0

The storage device is assumed to be ideal with ηc = ηd = 1. The power flowing into the
device is Ps (t) = Pg (t) − PL (t), and the stored energy is given by
Z t Z t
E(t) = Ps (τ )dτ = (Pg (τ ) − PL (τ )) dτ. (12)
0 0

Following (9), (11) and (12) we have

Eg (t) = E(t) + EL (t). (13)

In addition, the stored energy is limited by the device capacity, such that

0 ≤ E(t) ≤ Emax . (14)

The challenge is to determine the generated power Pg (t) that minimizes the total cost:
Z T
minimize Ftot = F (Pg (t))dt,
0
Z t
(15)
subject to E(t) = (Pg (τ ) − PL (τ )) dτ, 0 ≤ E(t) ≤ Emax ,
0
E(0) = Ei , E(T ) = Ef .
Lecture 4: Control of Energy Storage Devices 6

where the constants Ei and Ef denote the energy stored at the initial and final times. Using
(13), this problem may be reformulated as follows:
Z T
minimize Ftot = F (Pg (τ ))dτ,
0
subject to EL (t) ≤ Eg (t) ≤ EL (t) + Emax ,
(16)
Eg (0) = Ei , Eg (T ) = Ef + EL (T ),
d
Pg (t) = Eg .
dt
In this form we search for a bounded function Eg (t) that minimizes the total cost. This
problem is solved in [4].
To understand the properties of the optimal solution, consider first a time interval [t1 , t2 ] in
which Eg (t) is between the bounds, that is

EL (t) < Eg (t) < EL (t) + Emax .

In this interval the original problem (16) may be written as

Z t2
minimize F (Pg (τ ))dτ,
t1 (17)
d
subject to Pg (t) = Eg , Eg (t1 ) = E1 , Eg (t2 ) = E2 ,
dt
where E1 and E2 are given constants. It can be shown2 that the solution to this subproblem is
characterized by a constant generated power Pg (t). As a result the optimal generated energy
Eg (t) must be a straight line, as illustrated in Fig. 6.

E2
EL (t) + Emax

Eg (t)
EL (t)
E1

t1 t2 t [sec]

Figure 6: When between bounds, the optimal generated energy Eg (t) must follow a straight line.

Another feature of the optimal solution is that Eg (t) must be tangent to the bounds, as illus-
trated in Fig. 7. This property is proven in [4].
Typically these two features completely define the optimal solution, and hold regardless
of the cost function F (Pg ), as long as this function is twice differentiable, monotonically
increasing and strictly convex. If this is true we may choose any cost function that obeys
these conditions, and the optimal solution will be the same. One possible choice is
q
F (Pg ) = 1 + Pg2 , (18)
2
This can be proven based on Pontryagin’s minimum principle, which is not discussed in this text.
Lecture 4: Control of Energy Storage Devices 7

tangent
point

EL (t) + Emax

Eg (t)
EL (t)

tangent
point

Figure 7: The optimal energy Eg (t) must be tangent to the bounds.

and the resulting total cost is

s  2
Z T Z T Z T 
q dEg 
Ftot = F (Pg (τ ))dτ = 1 + Pg2 (τ )dτ = 1+ dτ, (19)
0 0 0 dt t=τ

which is the total length of the curve Eg (t).

Based on the arguments above we can conclude that

3 The optimal generated energy Eg (t) does not depend on the cost function F (·),
3 The optimal generated energy minimizes the total length of the curve Eg (t),

and therefore

the optimal generated energy Eg (t) follows the shortest path between the
bounds EL (t) and EL (t) + Emax .

This result provides a graphical design procedure that may be used to calculate the optimal
solution:

3 Plot the lower bound EL (t) and the upper bound EL (t) + Emax .
3 Choose the initial and final values of the generated energy Eg (t). A typical choice is
Eg (0) = EL (0) and Eg (T ) = EL (T ), which is equivalent to E(0) = E(T ) = 0.
3 Plot the shortest path that connects Eg (0) and Eg (T ) and is between bounds. This is
the optimal generated energy Eg (t).
3 All the other functions may be computed directly. For instance, the stored energy is
given by E(t) = Eg (t) − EL (t).

Several examples are shown in Figs. 8 and 9.

The shortest path method allows simple planning of the optimal generated energy, and en-
ables designers to estimate the size of the storage device for a specific application. Several
properties of the optimal solution are:

3 If the capacity Emax is very low then the generated power is approximately equal to
the load. In this case the storage device has little effect.
3 If the capacity Emax is high enough then the generated energy approximately follows
a straight line. The generated power is approximately constant and is equal to the
average load.
Lecture 4: Control of Energy Storage Devices 8

(a) Emax = 15 MJ (b) Emax = 8 MJ

(c) Emax = 5 MJ (d) Emax = 2 MJ

Figure 8: The optimal generated energy Eg (t) is the shortest path between EL (t) and EL (t) + Emax ,
and the generated power Pg (t) is “as constant as possible”. For high capacity values Pg (t) is approx-
imately equal to the average load, and for low capacity values Pg (t) ≈ PL (t).

Figure 9: The generated energy for various capacities. In each case the optimal generated energy
follows the shortest path.
Lecture 4: Control of Energy Storage Devices 9

As shown in [5], another property of the “shortest path” method is that it minimizes the
peak of the generated power Pg (t). To see this, consider the cost function
F (Pg (t)) = |Pg (t)|m . (20)
When m → ∞ we have
Z T  m1
1
lim Ftot = lim
m
|Pg (t)|m dt = max{|Pg (t)|}. (21)
m→∞ m→∞ 0 t

Since typically the optimal solution is the same for every value of m, the shortest path
method minimizes the peak power max{|Pg (t)|}. This result is demonstrated in Fig. 10.

(a) Low capacity, high peak power (b) High capacity, low peak power

Figure 10: The shortest path method minimizes the peak of the generated power Pg (t).

Energy trading
Storage devices may be also used for energy trading [6]. Energy is bought when the price is
low (usually at night), and sold when the price is high (usually during the day). In addition
to generating profits, this also helps to match the energy supply to the energy demand.
As an example, consider the following energy trading optimization problem:
Z T
maximize − C(t)P (t)dt,
0
subject to 0 ≤ E(t) ≤ Emax , (22)
(
dE ηc (E, P )P, P ≥ 0,
= −1
dt ηd (E, P )P, P < 0,
Lecture 4: Control of Energy Storage Devices 10

where

3 C(t) is the cost of energy,

3 P (t) is the power flowing into the storage device,
3 E(t) is the stored energy,
3 ηc (E, P ) is the charging efficiency,
3 ηd (E, P ) is the discharging efficiency.

For simplicity it is assumed that

(
C1 , 0 ≤ t < αT,
C(t) =
C2 , αT ≤ t < T,
(23)
ηc (E, P ) = ηc = constant,
ηd (E, P ) = ηd = constant,
RT
with 0 < C1 < C2 and 0 < α < 1. The integral − 0 C(t)P (t)dt appearing in the objective
represents the profit over a cycle.
The optimal energy management strategy is to buy energy when the price is low, and to sell
energy when the price is high. One optimal solution to the problem above is
(
Emax
, 0 ≤ t < αT,
P (t) = ηc αTηd Emax (24)
− (1−α)T , αT ≤ t < T,

which is illustrated in Fig. 11.

C2 C2

C(t)
C1 C1

Emax
E(t)

Figure 11: Energy is bought when the price is low, and sold when the price is high.

The profit over one cycle is

Z T  
Emax C1
− C(t)P (t)dt = −C1 + C2 ηd Emax = C2 ηd − Emax , (25)
0 ηc ηc
and it is positive if
C1 C1
C2 ηd − >0 =⇒ ηc ηd > . (26)
ηc C2
Lecture 4: Control of Energy Storage Devices 11

We can conclude that in order to make a profit the round-trip efficiency must be higher
than the price ratio. For instance:

3 If the storage device is lossless (ηc = ηd = 1) then the profit over a cycle is (C2 −C1 )Emax .
In this case the profit is proportional to the price difference, and to the device capacity.
3 If there is no price difference (C2 = C1 ) then the profit is zero or negative.
3 If C1 = 0 a positive profit is guaranteed, regardless of the device efficiency.

In practice, most energy storage control problems are more complex than the ones men-
tioned above, and are generally solved by numeric techniques such as linear programming,
dynamic programming, or Pontryagin’s minimum principle. Several examples are shown
in the reference list below, and a summary of the mostly used methods is presented in [7].

References
[1] A. Ulbig, T. S. Borsche, and G. Andersson, “Impact of low rotational inertia on power
system stability and operation,” in The 19th IFAC World Congress, 2014, pp. 7290–7297.

[2] F. Milano, F. Dörfler, G. Hug, D. J. Hill, and G. Verbič, “Foundations and challenges of
low-inertia systems,” in Power Systems Computation Conference, 2018, pp. 1–25.

[3] REN21. (2018) Renewables global futures report: Great debates towards 100%
renewable energy. [Online]. Available: http://www.ren21.net/future-of-renewables/
global-futures-report/

[4] Y. Levron and D. Shmilovitz, “Optimal power management in fueled systems with fi-
nite storage capacity,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 57,
no. 8, pp. 2221–2231, 2010.

[5] Y. Levron and D. Shmilovitz, “Power systems’ optimal peak-shaving applying sec-
ondary storage,” Electric Power Systems Research, vol. 89, pp. 80–84, 2012.

[6] D. Lifshitz and G. Weiss, “Optimal control of a capacitor-type energy storage system,”
IEEE Transactions on Automatic Control, vol. 60, no. 1, pp. 216–220, 2015.

[7] R. Machlev, N. Zargari, N. R. Chowdhury, J. Belikov, and Y. Levron, “A review of op-

timal control methods for energy storage systems - energy trading, energy balancing
and electric vehicles,” Journal of Energy Storage, vol. 32, p. 101787, 2020.

[8] Y. Levron, J. M. Guerrero, and Y. Beck, “Optimal power flow in microgrids with energy
storage,” IEEE Transactions on Power Systems, vol. 28, no. 3, pp. 3226–3234, 2013.

[9] B. Steffen and C. Weber, “Optimal operation of pumped-hydro storage plants with con-
tinuous time-varying power prices,” European Journal of Operational Research, vol. 252,
no. 1, pp. 308–321, 2016.

[10] J. L. Duchaud, G. Notton, C. Darras, and C. Voyant, “Power ramp-rate control algo-
rithm with optimal state of charge reference via dynamic programming,” Energy, vol.
149, pp. 709–717, 2018.

[11] H. Dagdougui, A. Ouammi, and R. Sacile, “Optimal control of a network of power

microgrids using the Pontryagin’s minimum principle,” IEEE Transactions on Control
Systems Technology, vol. 22, no. 5, pp. 1942–1948, 2014.
Lecture 4: Control of Energy Storage Devices 12

[12] L. Bayón, J. M. Grau, M. M. Ruiz, and P. M. Suárez, “A comparative economic study of

two configurations of hydro-wind power plants,” Energy, vol. 112, pp. 8–16, 2016.

[13] B. Heymann, J. F. Bonnans, P. Martinon, F. J. Silva, F. Lanas, and G. Jimenez-Estevez,

“Continuous optimal control approaches to microgrid energy management,” Energy
Systems, vol. 9, no. 1, pp. 59–77, 2017.

[14] Q. Li, W. Huang, W. Chen, Y. Yan, W. Shang, and M. Li, “Regenerative braking energy
recovery strategy based on Pontryagin’s minimum principle for fell cell/supercapacitor
hybrid locomotive,” International Journal of Hydrogen Energy, vol. 44, no. 11, pp. 5454–
5461, 2019.

[15] L. R. Cirocco, J. Boland, M. Belusko, F. Bruno, and P. Pudney, “Controlling stored en-
ergy in a concentrating solar thermal power plant to maximise revenue,” IET Renewable
Power Generation, vol. 9, no. 4, pp. 379–388, 2015.

[16] Y. Parvini and A. Vahidi, “Optimal charging of ultracapacitors during regenerative

braking,” in International Electric Vehicle Conference, 2012.

[17] A. Serpi, M. Porru, and A. Damiano, “An optimal power and energy management by
hybrid energy storage systems in microgrids,” Energies, vol. 10, no. 11, p. 1909, 2017.

[18] A. M. Dissanayake and N. C. Ekneligoda, “Transient optimization of parallel connected

inverters in islanded AC microgrids,” IEEE Transactions on Smart Grids, 2018.

[19] H. Dagdougui, A. Ouammi, L. Dessaint, and R. Sacile, “Global energy management

system for cooperative networked residential green buildings,” IET Renewable Power
Generation, vol. 10, no. 8, pp. 1237–1244, 2016.

[20] B.-H. Nguyen, R. German, J. P. F. Trovao, and A. Bouscayrol, “Real-time energy man-
agement of battery/supercapacitor electric vehicles based on an adaptation of Pontrya-
gin’s minimum principle,” IEEE Transactions on Vehicular Technology, vol. 68, no. 1, pp.
203–212, 2019.

[21] J. Du, X. Zhang, T. Wang, Z. Song, X. Yang, H. Wang, M. Ouyang, and X. Wu, “Battery
degradation minimization oriented energy management strategy for plug-in hybrid
electric bus with multi-energy storage system,” Energy, vol. 165, pp. 153–163, 2018.

[22] S. Uebel, N. Murgovski, C. Tempelhahn, and B. Baker, “Optimal energy management

and velocity control of hybrid electric vehicles,” IEEE Transactions on Vehicular Technol-
ogy, vol. 67, no. 1, pp. 327–337, 2018.

[23] Y. Parvini, A. Vahidi, and S. A. Fayazi, “Heuristic versus optimal charging of superca-
pacitors, lithium-ion, and lead-acid batteries: an efficiency point of view,” IEEE Trans-
actions on Control Systems Technology, vol. 26, no. 1, pp. 167–180, 2018.

[24] A. Nguyen, J. Lauber, and M. Dambrine, “Optimal control based algorithms for en-
ergy management of automotive power systems with battery/supercapacitor storage
devices,” Energy Conversion and Management, vol. 87, pp. 410–420, 2014.

[25] E. Vinot and R. Trigui, “Optimal energy management of HEVs with hybrid storage
system,” Energy Conversion and Management, vol. 76, pp. 437–452, 2013.

[26] P. Denholm, E. Ela, B. Kirby, and M. Milligan, “Role of energy storage with renewable
electricity generation,” NREL, Tech. Rep., jan 2010.
Lecture 4: Control of Energy Storage Devices 13

[27] A. Fahima, R. Ofir, J. Belikov, and Y. Levron, “Minimal energy storage required for
stability of low inertia distributed sources,” in IEEE International Energy Conference (En-
ergycon), Limassol, Cyprus, June 2018, pp. 1–5.

[28] D. Akinyele, J. Belikov, and Y. Levron, “Battery storage technologies for electrical ap-
plications: Impact in stand-alone photovoltaic systems,” Energies, vol. 10, no. 11, 2017.

[29] J. Wu, Y. Zou, X. Zhang, T. Liu, Z. Kong, and D. He, “An online correction predictive
EMS for a hybrid electric tracked vehicle based on dynamic programming and rein-
forcement learning,” IEEE Access, vol. 7, pp. 98 252–98 266, 2019.

[30] PSD, “Control of energy storage devices,” 2019, [Online] Available https://a-lab.ee/
sites/default/files/PS_Lecture_4.pdf, Accessed on December 08, 2019.

[31] Argonne National Laboratory, “Electric vehicle testing,” 2019, [Online] Available https:
//www.anl.gov/es/electric-vehicle-testing, Accessed on December 27, 2019.

[32] Y. Hu, C. Chen, J. He, B. Yang, and X. Guan, “IoT-based proactive energy supply control
for connected electric vehicles,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 7395–
7405, 2019.

[33] N. Zargari, Y. Levron, and J. Belikov, “Optimal control of energy storage devices based
on Pontryagin’s minimum principle and the shortest path method,” in IEEE PES Inno-
vative Smart Grid Technologies Europe, Sept. 2019.

[34] Y. Li and X. Jiao, “Real-time energy management for commute HEVs using modified
A-ECMS with traffic information recognition,” IET Intelligent Transport Systems, vol. 13,
no. 4, pp. 729–737, 2019.

[35] A. A. K. Arani, G. B. Gharehpetian, and M. Abedi, “Review on energy storage sys-

tems control methods in microgrids,” International Journal of Electrical Power & Energy
Systems, vol. 107, pp. 745–757, 2019.

[36] R. H. Byrne, T. A. Nguyen, D. A. Copp, B. R. Chalamala, and I. Gyuk, “Energy manage-

ment and optimization methods for grid energy storage systems,” IEEE Access, vol. 6,
pp. 13 231–13 260, 2018.

[37] H. Marefat, M. Jalalmaab, and N. L. Azad, “Energy management of battery electric

vehicles hybridized with supercapacitor using stochastic dynamic programming,” in
2018 SICE International Symposium on Control Systems (SICE ISCS). IEEE, 2018, pp.
199–205.

[38] B. Liu, L. Li, X. Wang, and S. Cheng, “Hybrid electric vehicle downshifting strategy
based on stochastic dynamic programming during regenerative braking process,” IEEE
Transactions on Vehicular Technology, vol. 67, no. 6, pp. 4716–4727, 2018.

[39] A. Bhattacharya, J. P. Kharoufeh, and B. Zeng, “Managing energy storage in micro-

grids: A multistage stochastic programming approach,” IEEE Transactions on Smart
Grids, vol. 9, no. 1, pp. 483–496, 2018.

[40] G. Díaz, J. Gómez-Aleixandre, J. Coto, and O. Conejero, “Maximum income resulting

from energy arbitrage by battery systems subject to cycle aging and price uncertainty
from a dynamic programming perspective,” Energy, vol. 156, pp. 647–660, 2018.
Lecture 4: Control of Energy Storage Devices 14

[41] W. Labidi, T. Chahed, and S.-E. Elayoubi, “Optimal battery management strategies in
mobile networks powered by a smart grid,” IEEE Transactions on Green Communications
and Networking, vol. 2, no. 3, pp. 859–867, 2018.

[42] K. Abdulla, J. D. Hoog, V. Muenzel, F. Suits, K. Steer, A. Wirth, and S. Halgamuge,

“Optimal operation of energy storage systems considering forecasts and battery degra-
dation,” IEEE Transactions on Smart Grids, vol. 9, no. 3, pp. 2086–2096, 2018.

[43] J. Jin, Y. Xu, Y. Khalid, and N. U. L. Hassan, “Optimal operation of energy storage with
random renewable generation and AC/DC loads,” IEEE Transactions on Smart Grids,
vol. 9, no. 3, pp. 2314–2326, 2018.

[44] K. Mahmud, M. J. Hossain, and G. E. Town, “Peak-load reduction by coordinated re-

sponse of photovoltaics, battery storage, and electric vehicles,” IEEE Access, vol. 6, pp.
29 353–29 365, 2018.

[45] Z. D. Asher, D. A. Baker, and T. H. Bradley, “Prediction error applied to hybrid electric
vehicle optimal fuel economy,” IEEE Transactions on Control Systems Technology, vol. 26,
no. 6, pp. 2121–2134, 2018.

[46] Y. Shi, B. Xu, D. Wang, and B. Zhang, “Using battery storage for peak shaving and
frequency regulation: Joint optimization for superlinear gains,” IEEE Transactions on
Power Systems, vol. 33, no. 3, pp. 2882–2894, 2018.

[47] Q. Jiang, F. Ossart, and C. Marchand, “Comparative study of real-time HEV energy
management strategies,” IEEE Transactions on Vehicular Technology, vol. 66, no. 12, pp.
10 875–10 888, 2017.

[48] J. Jin and Y. Xu, “Optimal storage operation under demand charge,” IEEE Transactions
on Power Systems, vol. 32, no. 1, pp. 795–808, 2017.

[49] A. A. Malikopoulos, “A multiobjective optimization framework for online stochastic

optimal control in hybrid electric vehicles,” IEEE Transactions on Control Systems Tech-
nology, vol. 24, no. 2, pp. 440–450, 2016.

[50] L. Jia and L. Tong, “Renewables and storage in distribution systems: Centralized vs.
decentralized integration,” IEEE Journal on Selected Areas in Communications, vol. 34,
no. 3, pp. 665–674, March 2016.

[51] X. Wu, X. Hu, S. Moura, X. Yin, and V. Pickert, “Stochastic control of smart home en-
ergy management with plug-in electric vehicle battery energy storage and photovoltaic
array,” Journal of Power Sources, vol. 333, pp. 203–212, 2016.

[52] C. Vagg, S. Akehurst, C. J. Brace, and L. Ash, “Stochastic dynamic programming in

the real-world control of hybrid electric vehicles,” IEEE Transactions on Control Systems
Technology, vol. 24, no. 3, pp. 853–866, 2016.

[53] M. Obi and R. Bass, “Trends and challenges of grid-connected photovoltaic systems –
A review,” Renewable and Sustainable Energy Reviews, vol. 58, pp. 1082–1094, 2016.

[54] P. Zhang, F. Yan, and C. Du, “A comprehensive analysis of energy management strate-
gies for hybrid electric vehicles based on bibliometrics,” Renewable and Sustainable En-
ergy Reviews, vol. 48, pp. 88–104, 2015.
Lecture 4: Control of Energy Storage Devices 15

[55] X. Zeng and J. Wang, “A parallel hybrid electric vehicle energy management strategy
using stochastic model predictive control with road grade preview,” IEEE Transactions
on Control Systems Technology, vol. 23, no. 6, pp. 2416–2423, 2015.

[56] P. Harsha and M. Dahleh, “Optimal management and sizing of energy storage under
dynamic pricing for the efficient integration of renewable energy,” IEEE Transactions on
Power Systems, vol. 30, no. 3, pp. 1164–1181, 2015.

[57] X. Xi, R. Sioshansi, and V. Marano, “A stochastic dynamic programming model for co-
optimization of distributed energy storage,” Energy Systems, vol. 5, no. 3, pp. 475–505,
2014.

[58] A. Castillo and D. F. Gayme, “Grid-scale energy storage applications in renewable en-
ergy integration: A survey,” Energy Conversion and Management, vol. 87, pp. 885–894,
2014.

[59] S. Lakshminarayana, L. Yang, H. V. Poor, T. Q. S. Quek, and J. Zhang, “Risk-aware

energy procurement with renewable energy and storage,” in International Conference on
Smart Grid Communications, 2014, pp. 176–181.

[60] Z. Wang and S. Wang, “Grid power peak shaving and valley filling using vehicle-to-
grid systems,” IEEE Transactions on Power Delivery, vol. 28, no. 3, pp. 1822–1829, 2013.

[61] T. H. Mehr, M. A. S. Masoum, and N. Jabalameli, “Grid-connected Lithium-ion battery

energy storage system for load leveling and peak shaving,” in Australasian Universities
Power Engineering Conference, 2013, pp. 1–6.

[62] H.-I. Su and A. El Gamal, “Modeling and analysis of the role of energy storage for
renewable integration: Power balancing,” IEEE Transactions on Power Systems, vol. 28,
no. 4, pp. 4109–4117, 2013.

[63] J. Donadee, “Optimal operation of energy storage for arbitrage and ancillary service
capacity: The infinite horizon approach,” in 2013 North American Power Symposium
(NAPS). IEEE, 2013, pp. 1–6.

[64] H. Akhavan-Hejazi and H. Mohsenian-Rad, “Optimal operation of independent stor-

age systems in energy and reserve markets with high wind penetration,” IEEE Transac-
tions on Smart Grids, vol. 5, no. 2, pp. 1088–1097, 2013.

[65] Z. Shu and P. Jirutitijaroen, “Optimal operation strategy of energy storage system for
grid-connected wind power plants,” IEEE Transactions on Sustainable Energy, vol. 5,
no. 1, pp. 190–199, 2013.

[66] D. F. Opila, X. Wang, R. McGee, R. B. Gillespie, J. A. Cook, and J. W. Grizzle, “An energy
management controller to optimally trade off fuel economy and drivability for hybrid
vehicles,” IEEE Transactions on Control Systems Technology, vol. 20, no. 6, pp. 1490–1505,
2012.

[67] S. J. Moura, H. K. Fathy, D. S. Callaway, and J. L. Stein, “A stochastic optimal control

approach for power management in plug-in hybrid electric vehicles,” IEEE Transactions
on Control Systems Technology, vol. 19, no. 3, pp. 545–555, 2011.

[68] L. Serrao, S. Onori, A. Sciarretta, Y. Guezennec, and G. Rizzoni, “Optimal energy man-
agement of hybrid electric vehicles including battery aging,” in American Control Con-
ference, 2011, pp. 2125–2130.
Lecture 4: Control of Energy Storage Devices 16

[69] N. Löhndorf and S. Minner, “Optimal day-ahead trading and storage of renewable
energies—an approximate dynamic programming approach,” Energy Systems, vol. 1,
no. 1, pp. 61–77, 2010.

[70] C. Romaus, K. Gathmann, and J. Böcker, “Optimal energy management for a hybrid
energy storage system for electric vehicles based on stochastic dynamic programming,”
in Vehicle Power and Propulsion Conference, 2010, pp. 1–6.

[71] W. B. Powell, Approximate Dynamic Programming: Solving the curses of dimensionality.

John Wiley & Sons, 2007.

[72] L. Johannesson, M. Asbogard, and B. Egardt, “Assessing the potential of predictive

control for hybrid vehicle powertrains using stochastic dynamic programming,” IEEE
Transactions on Intelligent Transportation Systems, vol. 8, no. 1, pp. 71–83, 2007.

[73] P. Rutquist, C. Breitholtz, and T. Wik, “An eigenvalue approach to infinite-horizon op-
timal control,” IFAC Proceedings Volumes, vol. 38, no. 1, pp. 39–44, 2005.

[74] A. Brahma, Y. Guezennec, and G. Rizzoni, “Optimal energy management in series hy-
brid electric vehicles,” in American Control Conference, vol. 1, no. 6, 2000, pp. 60–64.

[75] N. R. Chowdhury, R. Ofir, N. Zargari, D. Baimel, J. Belikov, and Y. Levron, “Opti-

mal control of lossy storage devices based on Pontryagin’s Minimum Principle,” IEEE
Transactions on Energy Conversion, 2020.

[76] Y. Levron and J. Belikov, “Performance limits of low inertia systems based on minimum
energy control,” in European Control Conference, 2019.