64

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

Opposition-Based Differential Evolution

Shahryar Rahnamayan, Member, IEEE, Hamid R. Tizhoosh, and Magdy M. A. Salama, Fellow, IEEE

AbstractEvolutionary algorithms (EAs) are well-known optimization approaches to deal with nonlinear and complex problems.
However, these population-based algorithms are computationally
expensive due to the slow nature of the evolutionary process. This
paper presents a novel algorithm to accelerate the differential evolution (DE). The proposed opposition-based DE (ODE) employs
opposition-based learning (OBL) for population initialization and
also for generation jumping. In this work, opposite numbers have
been utilized to improve the convergence rate of DE. A comprehensive set of 58 complex benchmark functions including a wide range
of dimensions is employed for experimental verification. The influence of dimensionality, population size, jumping rate, and various
mutation strategies are also investigated. Additionally, the contribution of opposite numbers is empirically verified. We also provide a comparison of ODE to fuzzy adaptive DE (FADE). Experimental results confirm that the ODE outperforms the original DE
and FADE in terms of convergence speed and solution accuracy.
Index TermsDifferential evolution (DE), evolutionary algorithms, opposition-based learning, opposite numbers, optimiztion.

I. INTRODUCTION

IFFERENTIAL EVOLUTION (DE) was proposed by

Price and Storn in 1995 [1], [2]. It is an effective, robust,
and simple global optimization algorithm [3] which only has
a few control parameters. According to frequently reported
comprehensive studies [3][6], DE outperforms many other
optimization methods in terms of convergence speed and robustness over common benchmark functions and real-world
problems. Generally speaking, all population-based optimization algorithms, no exception for DE, suffer from long
computational times because of their evolutionary/stochastic
nature. This crucial drawback sometimes limits their application to offline problems with little or no real-time constraints.
The concept of opposition-based learning (OBL) was introduced by Tizhoosh [7] and has thus far been applied to accelerate reinforcement learning [8][10] and backpropagation
learning in neural networks [11]. The main idea behind OBL
is the simultaneous consideration of an estimate and its corresponding opposite estimate (i.e., guess and opposite guess) in
Manuscript received October 20, 2006; revised January 10, 2007. The work
of S. Rahnamayan was supported by Ontario Graduate Scholarship (OGS).
S. Rahnamayan is with the Pattern Analysis and Machine Intelligence Laboratory, University of Waterloo, Department of Systems Design Engineering,
Waterloo, ON N2L 3G1, Canada (e-mail: shahryar@pami.uwaterloo.ca).
H. R. Tizhoosh is with the Pattern Analysis and Machine Intelligence Laboratory, University of Waterloo, Department of Systems Design Engineering,
Waterloo, ON N2L 3G1, Canada (e-mail: tizhoosh@uwaterloo.ca).
M. M. A. Salama is with the Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail:
m.salama@ece.uwaterloo.ca).
Digital Object Identifier 10.1109/TEVC.2007.894200

order to achieve a better approximation for the current candidate

solution. In this paper, OBL has been utilized to accelerate the
convergence rate of DE. Hence, our proposed approach has been
called opposition-based differential evolution (ODE). ODE uses
opposite numbers during population initialization and also for
generating new populations during the evolutionary process. To
the best of our knowledge, this is the first time that opposite
numbers have been utilized to speed up the convergence rate of
an optimization algorithm.
Purely random resampling or selection of solutions from a
given population has the chance of visiting or even revisiting
unproductive regions of the search space. As we will demonstrate, the chance of this occurring is lower for opposite numbers than it is for purely random ones. In fact, a mathematical
proof has been proposed to show that, in general, opposite numbers are more likely to be closer to the optimal solution than
purely random ones [12]. In this paper, the usefulness of opposite numbers is empirically investigated by replacing them with
random numbers and investigating the consequences. Our experimental results serve to confirm the theoretical expectations.
Furthermore, the proposed schemes for population initialization
and generation jumping are simple and general enough to be embedded inside other DE versions.
This paper provides a comprehensive set of experimental
verifications of our proposed approach. Specifically, we investigate: 1) the convergence speed and robustness; 2) the
effect of dimensionality; 3) the advantage of opposite points
over random ones; 4) the population size; 5) the mutation
operator; 6) the setting of newly added control parameter, the
jumping rate, and 7) ODE is compared with fuzzy adaptive DE
(FADE). For all these experiments, a comprehensive set of 58
well-known benchmark functions is employed. Furthermore,
in order to fairly compare the algorithms, a complete set of
performance measures has been utilized. These measures are
the average number of function calls (NFCs), success rate (SR),
average SR, number of unsolved problems, number of functions
for which the algorithm outperforms its competitor, acceleration rate (AR), and the average of the best found solutions
and the corresponding standard deviation as calculated after
specific number of function evaluations has been reached.
Organization of the rest of this paper is as follows. The DE
algorithm is shortly reviewed in Section II. In Section III, the
concept of OBL is briefly explained. The proposed algorithm
is presented in Section IV and a comprehensive set of experimental results are provided in Section V. Finally, the work is
concluded in Section VI. Appendix A lists all test functions, and
Appendix B provides complementary results.

1089-778X/$25.00 2007 IEEE

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

65

TABLE I
f ). THE BEST RESULT OF EACH MUTATION STRATEGY IS EMPHASIZED
COMPARISON OF DE AND ODE ON FOUR DIFFERENT MUTATION STRATEGIES (f
IN BOLDFACE AND THE BEST RESULT AMONG FOUR MUTATION STRATEGIES (EIGHT RESULTS) IS HIGHLIGHTED IN ITALIC FONTS

II. A BRIEF INTRODUCTION TO DIFFERENTIAL EVOLUTION

Differential evolution (DE) is a population-based and directed search method [25], [26]. Like other evolutionary
algorithms (EAs), it starts with an initial population vector,
which is randomly generated when no preliminary knowledge about the solution space is available. Let us assume that
are solution vectors in generation
(
size). Successive populations are generated
by adding the weighted difference of two randomly selected
vectors to a third randomly selected vector. For classical DE
, the mutation, crossover, and selection
operators are straightforwardly defined as follows.
in generation a mutant
MutationFor each vector
is defined by
vector
(1)
where
and
, and
are mutually
.
different random integer indices selected from
, and are different so that
is required.
Further,
is a real constant which determines the amplification
of the added differential variation of
. Larger
values for result in higher diversity in the generated population and lower values cause faster convergence.
CrossoverDE utilizes the crossover operation to generate
new solutions by shuffling competing vectors and also to increase the diversity of the population. It defines the following
trial vector:
(2)
where

problem dimension) and

(3)

is the predefined crossover rate constant, and

is the th evaluation of a uniform random number
is a random parameter index,
generator.
chosen once for each to make sure that at least one parameter
. Most popular
is always selected from the mutated vector
are in the range of
[27].
values for
SelectionIt is an approach which must decide which vector
or
) should be a member of next (new) generation
(
. For a maximization problem, the vector with the higher
fitness value is chosen. There are other variants based on different mutation and crossover strategies [2].
Other studies have been conducted to enhance the performance of the original DE algorithm, such as, adaptively determining DEs control parameters. The FADE algorithm was introduced by Liu and Lampinen [13]. They employed a fuzzy
logic controller to set the mutation and crossover rates. In the
same direction, Brest et al. [15] proposed self-adapting control
parameter settings. Teo proposed a dynamic population sizing
strategy based on self-adaptation [17] and Ali and Trn [18] introduced auxiliary population and automatic calculating of the
amplification factor , for the difference vector.
Some others have experimented with multipopulation ideas.
Tasoulis et al. proposed parallel DE [19] where they assign each
subpopulation to a different processor node. Shi et al. [20] partitioned high-dimensional search spaces into smaller spaces and
used multiple cooperating subpopulations to find the solution.
They called this method cooperative coevolutionary differential
evolution.
Hybridization with different algorithms is another direction
for improvement. Sun et al. [21] proposed a new hybrid algorithm based on a combination of DE and estimation of distribution algorithm. This technique uses a probability model to determine promising regions in order to focus the search process
on those areas. Noman and Iba [22] incorporated local search

66

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

TABLE II
f ). THE BEST RESULT OF EACH MUTATION
COMPARISON OF DE AND ODE ON FOUR DIFFERENT MUTATION STRATEGIES CONTINUED FROM TABLE I (f
STRATEGY IS EMPHASIZED IN BOLDFACE AND THE BEST RESULT AMONG FOUR MUTATION STRATEGIES (EIGHT RESULTS) IS HIGHLIGHTED IN ITALIC FONTS

into the classical DE. They employed fittest individual refinement which is a crossover-based local search. Fan and Lampinen
[23] introduced a new local search operation, trigonometric mutation, in order to obtain a better tradeoff between convergence
speed and robustness. Kaelo and Ali [24] employed reinforcement learning and different schemes for generating fitter trial
points.
Although the proposed algorithm in this paper also attempts
to enhance DE, its methodology is completely different from
all aforementioned works. That is, a first attempt to accelerate
convergence speed of DE by utilizing the scheme of OBL. We
use OBL for population initialization and the production of new
generations.
III. OPPOSITION-BASED LEARNING (OBL)
Generally speaking, evolutionary optimization methods
start with some initial solutions (initial population) and try to
improve them toward some optimal solution(s). The process
of searching terminates when some predefined criteria are
satisfied. In the absence of a priori information about the solution, we usually start with random guesses. The computation
time, among others, is related to the distance of these initial
guesses from the optimal solution. We can improve our chance
of starting with a closer (fitter) solution by simultaneously
checking the opposite solution. By doing this, the fitter one
(guess or opposite guess) can be chosen as an initial solution.
In fact, according to probability theory, 50% of the time a
guess is further from the solution than its opposite guess.
Therefore, starting with the closer of the two guesses (as judged
by its fitness) has the potential to accelerate convergence. The
same approach can be applied not only to initial solutions but
also continuously to each solution in the current population.
However, before concentrating on OBL, we need to define the
concept of opposite numbers [7].

Definition (Opposite Number)Let

number. The opposite number is defined by

be a real
(4)

Similarly, this definition can be extended to higher dimensions as follows [7].

Definition (Opposite Point)Let
be
a point in -dimensional space, where
and
. The opposite point
is completely defined by its components
(5)
Now, by employing the opposite point definition, the opposition-based optimization can be defined as follows.
Opposition-Based OptimizationLet
be a point in -dimensional space (i.e., a candidate soluis a fitness function which is used to meation). Assume
sure the candidates fitness. According to the definition of the
is the opposite of
opposite point,
. Now, if
, then point can be replaced with ; otherwise, we continue with . Hence, the point
and its opposite point are evaluated simultaneously in order to
continue with the fitter one.
IV. PROPOSED ALGORITHM
Similar to all population-based optimization algorithms, two
main steps are distinguishable for DE, namely, population initialization and producing new generations by evolutionary operations such as mutation, crossover, and selection. We will enhance these two steps using the OBL scheme. The original DE is
chosen as a parent algorithm and the proposed opposition-based
ideas are embedded in DE to accelerate its convergence speed.
Corresponding pseudocode for the proposed approach (ODE) is

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

67

TABLE III
PSEUDOCODE FOR OPPOSITION-BASED DIFFERENTIAL EVOLUTION (ODE). P : INITIAL POPULATION, OP : OPPOSITE OF INITIAL POPULATION, N : POPULATION
SIZE, P : CURRENT POPULATION, OP: OPPOSITE OF CURRENT POPULATION, V : NOISE VECTOR, U : TRIAL VECTOR, D : PROBLEM DIMENSION, [a ; b ]: RANGE OF
: MAXIMUM NUMBER OF
THE j TH VARIABLE, BFV: BEST FITNESS VALUE SO FAR, VTR: VALUE TO REACH, NFC: NUMBER OF FUNCTION CALLS, MAX
FUNCTION CALLS, F: MUTATION CONSTANT, rand(0; 1): UNIFORMLY GENERATED RANDOM NUMBER, C : CROSSOVER RATE, f ( 1 ): OBJECTIVE FUNCTION, P :
POPULATION OF NEXT GENERATION, J : JUMPING RATE, min : MINIMUM VALUE OF THE j TH VARIABLE IN THE CURRENT POPULATION, max : MAXIMUM
VALUE OF THE j TH VARIABLE IN THE CURRENT POPULATION. STEPS 15 AND 2632 ARE IMPLEMENTATIONS OF OPPOSITION-BASED INITIALIZATION AND
OPPOSITION-BASED GENERATION JUMPING, RESPECTIVELY

given in Table III. Newly added/extended code segments will be

explained in the following subsections.
A. Opposition-Based Population Initialization
According to our review of optimization literature, random
number generation, in absence of a priori knowledge, is the
common choice to create an initial population. Therefore, as
mentioned in Section III, by utilizing OBL, we can obtain fitter
starting candidate solutions even when there is no a priori
knowledge about the solution(s). Steps 15 from Table III show
the implementation of opposition-based initialization for ODE.
The following steps present that procedure.
1) Initialize the population
randomly.
2) Calculate opposite population by

(6)

and
denote th variable of the th vector
where
of the population and the opposite-population, respectively.
3) Select the
fittest individuals from
as initial
population.
B. Opposition-Based Generation Jumping
By applying a similar approach to the current population, the
evolutionary process can be forced to jump to a new solution
candidate, which ideally is fitter than the current one. Based
(i.e., jumping probability), after generon a jumping rate
ating new populations by mutation, crossover, and selection,
fittest individthe opposite population is calculated and the
uals are selected from the union of the current population and
the opposite population. Unlike opposition-based initialization,
generation jumping calculates the opposite population dynamically. Instead of using variables predefined interval boundaries

68

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

, generation jumping calculates the opposite of each

and maximum
variable based on minimum
values of that variable in the current population
(7)
By staying within variables interval static boundaries, we
would jump outside of the already shrunken search space
and the knowledge of the current reduced space (converged
population) would be lost. Hence, we calculate opposite
points by using variables current interval in the population
which is, as the search does progress, in.
creasingly smaller than the corresponding initial range
Steps 2632 from Table III show the implementation of opposition-based generation jumping for ODE. Discussion about the
is given in Section V-F.
suitable value for jumping rate
V. EXPERIMENTAL VERIFICATION
Test FunctionsA comprehensive set of benchmark functions, including 58 different global optimization problems, has
been used for performance verification of the proposed approach
(although utilizing a much smaller set of benchmark functions
for this purpose is commonly acceptable, e.g., [32]). The definition of the benchmark functions and their global optimum(s)
are listed in Appendix A.
Comparison Strategies and MetricsWe compare the convergence speed of DE and ODE by measuring the number of
function calls (NFCs) which is the most commonly used metric
in literature [3][6], [28], [29], [35]. A smaller NFC means
higher convergence speed. The termination criterion is to find
a value smaller than the value-to-reach (VTR) before reaching
the maximum number of function calls
. In order to
minimize the effect of the stochastic nature of the algorithms on
the metric, the reported NFCs for each function is the average
over 50 trials. In order to compare convergence speeds, we use
the acceleration rate (AR) which is defined as follows, based
on the NFCs for the two algorithms DE and ODE:
(8)
where
means ODE is faster.
The number of times, for which the algorithm successfully
reaches the VTR for each test function is measured as the success rate (SR)
(9)
and the avFurther, the average acceleration rate
over test functions are calculated
erage success rate
as follows:
(10)
(11)
Parameter SettingsParameter settings for all conducted
experiments are as follows unless a change is mentioned (the

same setting has been used in literature cited after of each parameter).
[15], [33], [34].
Population size,
Differential amplification factor,
[2], [4], [13],
[15], [18].
[2], [4], [13],
Crossover probability constant,
[15], [18].
(discussed in
Jumping rate constant,
Section V-F).
Mutation strategy [2]: DE/rand/1/bin (classic version of
DE) [2], [3], [15], [16], [26].
.
Maximum NFCs,
[35].
Value to reach, VTR
In order to maintain a reliable and fair comparison, for all 7
experiments: 1) the parameter settings are the same as above
for all experiments, unless we mention new settings for one
or some of them to serve the purpose of that parameter study;
2) for all conducted experiments, the reported values are the
average of the results for 50 independent runs, and most importantly; 3) extra fitness evaluation is required for the opposite points (both in population initialization and also generation
jumping phases) are counted.
ResultsA comprehensive set of experiments are conducted
and categorized as follows. In Section V-A, DE and ODE are
compared in terms of convergence speed and robustness. The
effect of problem dimensionality is investigated in Section V-B.
The contribution of opposite points to the achieved acceleration
results is demonstrated in Section V-C. The effect of population
size is studied in Section V-D. Comparison of DE and ODE
over some other mutation strategies is performed in Section V-E.
Discussion about the newly added control parameter, jumping
rate, is covered in Section V-F. Finally, comparison of the ODE
with FADE is given in Section V-G.

A. Experiment Series 1: Comparison of DE and ODE

First of all, we need to compare the parent algorithm DE with
ODE in terms of convergence speed and robustness. The results
of solving 58 benchmark functions (see Appendix A) are given
in Table IV. The best result of the NFCs and the SR for each
function are highlighted in boldface. The average success rates
and the average acceleration rate
on 58 test
functions are shown in the last row of the table.
ODE outperforms DE on 40 test functions, while DE surpasses ODE on 15 functions. Except for , the rest of the 14
. Average acfunctions are low-dimensional functions
is 1.44, which means ODE is on average
celeration rate
44% faster than DE. While the average success rate
for both of them is equal to 0.86, both algorithms fail to solve
, and
; in addition, DE fails to solve
and ODE is
unsuccessful on . Some sample graphs for the performance
comparison between DE and ODE are given in Fig. 1. These
curves (best solution versus NFCs) show that how ODE converges faster than DE toward the optimal solution.
Results analysisWith the same control parameter settings
for both algorithms and fixing the jumping rate for the ODE
, their SRs are comparable, while ODE shows better

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

TABLE IV
COMPARISON OF DE AND ODE. D: DIMENSION, NFC: NUMBER OF FUNCTION
CALLS, SR: SUCCESS RATE, AR: ACCELERATION RATE. THE LAST ROW OF THE
TABLE PRESENTS THE AVERAGE SUCCESS RATES (SR ) AND THE AVERAGE
ACCELERATION RATE (AR ). THE BEST RESULTS FOR EACH CASE ARE
HIGHLIGHTED IN BOLDFACE

69

B. Experiment Series 2: Influence of Dimensionality

In order to investigate the effect of the problem dimensionand
ality, the same experiments are repeated for
for each scalable function from our test set. All other
and
control parameters are kept unchanged. Results for
are given in Table V for 40 test functions.
According to the obtained results, ODE surpasses DE on 34
test functions, while DE outperforms ODE on two functions
and
). Both algorithms are unable to solve
(
,
,
, and
before meeting
the maximum NFCs. The average AR is equal to 1.73, meaning
that ODE performs 73% faster than DE. The average SR for DE
and ODE are 0.82 and 0.81, respectively.
For 11 functions (
and
), the AR is increased by growing dimensionality. ODE
achieves a more desirable results for all but four of the functions
, and
) where no improvement can be observed.
(
An interesting effect for
is that for dimensions 5 and 10
, see Table IV), DE performs better than ODE;
(for
but when the dimension is increased to 20, ODE shows better
results in terms of NFC and SR. Furthermore, DE cannot solve
for
, but ODE solves it in 35% of the trials.
In the bottom of Table V, the average SRs and the average
,
are presented. For functions
ARs for functions with
, the overall SR for DE is 4% higher
with dimension of
than ODEs (0.98 versus 0.94) and the overall AR is 1.67. For
, the overall AR is 1.81 and the
functions with dimension of
SRs for DE and ODE are 0.66 and 0.67, respectively.
Results analysisDecreasing the overall SR for DE and
ODE was predictable because by doubling the problem dimension, algorithms are sometimes unable to solve the problem
before reaching the maximum NFCs (which is a fixed number
for all experiments). However, as seen, ODE performs better for
high-dimensional problems. The higher average AR belongs to
.
the functions with dimension
C. Experiment Series 3: Contribution of Opposite Points
In this section, we want to verify that the achieved AR is really
due to utilizing opposite points. For this purpose, all parts of the
proposed algorithm are kept untouched and instead of using opposite points for the population initialization and the generation
jumping, uniformly generated random points are employed. In
order to have a fair competition for this case, (exactly like what
we did for opposite points), the current interval (dynamic interval) of the variables are used to generate new random points
in the generation jumping phase. So, line 4 from Table III should
be changed to

convergence speed than DE (44% faster in overall). The jumping

rate is an important control parameter which, if optimally set,
can achieve even better results. The discussion about this parameter is covered in Section V-F.

where
generates a uniformly distributed random
. This change is for the initializanumber on the interval
tion part, so the predefined boundaries of variables
have been used to generate new random numbers. In fact,
instead of generating
random individuals, this time we
candidate solutions.
generate

70

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

Fig. 1. Sample graphs (best solution versus NFCs) for performance comparison between DE and ODE. (a) f , ODE is 1.83 times faster. (b) f , ODE is 1.81 times
faster. (c) f , ODE is 1.64 times faster. (d) f , ODE is 1.72 times faster.

In the same manner, line 30 should be replaced with

As mentioned before, for generation jumping, the current

are used to generate
boundaries of variables
random numbers. Finally, in order to have the same selection
method, lines 5 and 31 in Table III are substituted with

After these modifications, the random version of ODE

(called RDE) is introduced. Now, we are ready to apply this
algorithm to solve our test problems. All control parameters
are kept the same to ensure a fair comparison. Results for the
current algorithm are presented in Table VI; also the results
of DE and ODE (from Table IV) are repeated in this table
to ease the comparison among these three competitors (DE,
ODE, and RDE). Two ARs are reported in this table. The
first column compares DE and ODE and the second one comand
pares DE and RDE [
].
As seen, ODE outperforms DE and RDE on 40 of the functions. DE performs better than ODE and RDE on 15 functions.
The RDE can outperform DE (but not ODE) on just three
, and
(emphasized in boldface under the
functions
column). The average AR (DE versus RDE) is 0.87,

which means the RDE is 13% slower than its parent algorithm.
The average SR is almost the same for all of them (0.86, 0.86,
and 0.87 for DE, ODE, and RDE, respectively).
Results analysisJust by replacing the opposite numbers
with additional random numberswhile the random numbers
are generated uniformly in the variables dynamic intervals and
the rest of the proposed algorithm is kept untouchedthe avto 0.87
, which
erage AR drops from 1.44
is a 57% reduction in speed. This clearly demonstrates that the
achieved improvements are due to usage of opposite points, and
that the same level of improvement cannot be achieved via additional random sampling.
D. Experiment Series 4: Effect of Population Size
In order to investigate the effect of the population size, the
)
same experiments (conducted in Section V-A for
and
. The results for
are repeated for
and
are given in Tables XII and XIII,
respectively. In order to discuss the population size, the overall
results of three tables (Tables IV, XII, and XIII) are summarized
in Table VII.
, the average SR for DE and ODE is 0.79 and
For
0.77, respectively (DE performs marginally better than ODE).
However, DE fails to solve nine functions, while ODE fails on
seven. ODE outperforms DE on 35 functions; this number is 15
for
for DE. The average AR is 1.05 for this case (

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

TABLE V
COMPARISON OF DE AND ODE FOR DIMENSION SIZES D=2 AND 2D
FOR ALL SCALABLE FUNCTIONS OF THE TEST SUITE. IN THE BOTTOM
OF THE TABLE, THE AVERAGE SUCCESS RATES AND THE AVERAGE
ACCELERATION RATES FOR FUNCTIONS WITH D=2; 2D , AND FOR
BOTH (OVERALL) ARE PRESENTED. THE BEST RESULT OF NFC
AND SR FOR EACH CASE ARE HIGHLIGHTED IN BOLDFACE

71

. However, ODE outperforms DE on

functions
40 functions, whereas DE outperforms ODE on 15 functions.
The average AR is 1.44 for this case.
, ODE outperforms DE on all mentioned meaFor
versus
, respectively), and the
sures (
average AR is 1.86. As shown in the last two rows of Table VII,
ODE performs better than DE in terms of all calculated measures.
Results analysisAccording to the results of Sections V-A
and V-B, for the majority of functions, ODE performs better
when the dimension of the problems increases. On the other
hand, for higher dimensional problems a larger population size
). According to the results
should be employed (e.g.,
of this section, ODE performs better for larger population sizes.
E. Experiments Series 5: Effect of Various Mutation Operators
More than ten mutation strategies have been developed for
DE [2], [3]. Although many researchers report results for one of
these mutation strategies, most works [2], [15], [16] use the standard one, namely, DE/rand/1/bin, as we did. In this work, three
other well-known mutation strategies, namely, DE/rand/1/exp,
DE/rand/2/exp, and DE/rand/2/bin are selected to investigate
the effect of the mutation operator. The results are presented in
) and Table II (for
). The overall
Table I (for
results of these two tables are summarized in compact form in
Table VIII to ease the comparison.
For all mutation strategies, ODE performs better than DE by
looking at the total NFCs, average SR, number of solved functions, number of functions for which ODE outperforms DE, and
the average AR.
Results analysisAccording to Table VIII, the best mutation strategy for DE and also for ODE is the DE/rand/1/bin. This
confirms choosing mutation strategy DE/rand/1/bin as a standard operator by other researchers [2], [15], [16].
F. Experiment Series 6: Proper Setting of Jumping Rate

is excluded as an exceptional case in order to prevent from

dominating the statistic). By carefully looking at the results, we
can recognize that when the population size is reduced from 100
, and
) for which
to 50, four functions (namely,
ODE has outperformed DE, are now (for population size 50)
solved faster by DE. However, this was predictable because the
dimension of those functions are 20, 10, 30, and 30, respectively,
is a small population size to solve these functions.
and
as a proper value for the popMany authors have proposed
ulation size [13], [14]. On the other hand, as we know, ODE
reduces the population diversity by its selection method and so
for small population size, we need to reduce the jumping rate to
control the diversity reduction. Here, the jumping rate was kept
the same for all population sizes and equal to 0.3.
, DE and ODE show an equal average success
For
, and they fail to solve an equal number of
rate

In the proposed algorithm, a new control parameter

is
added to DEs parameters (
, and
). Although this parameter was fixed for all experiments, the performance of ODE
can vary for different values. The jumping rate for our current
without any effort to find an optimal
study was set to
value. In some trials, we observed that a jumping rate higher
than 0.6 is not suitable for many functions and causes a premature convergence or an unusual growing of the number of funcmeans no jumping.
tion evaluations. On the other hand,
By this way, simply the mean value of 0 and 0.6,
,
was selected for all conducted experiments as a default value.
In this section, we try to find an optimal jumping rate
for each test function from a discrete set of jumping rate values
to answer the question whether a general recommendation for
setting can be offered. Now, we are faced with a fundamental
question: In order to find optimal jumping rate, should we look
for the minimum NFCs or the maximum SR? Both measures
are important factors in an optimization process. So, two individual objectives should be considered simultaneously. In order

72

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

TABLE VI
COMPARISON OF DE, ODE, AND RDE. THE BEST RESULT FOR EACH CASE IS HIGHLIGHTED IN BOLDFACE

to combine these two measures, a new measure, called success

performance (SP), has been introduced as follows [35]:

(12)
By this definition, the two following algorithms have equal
.
performances

and
.
and
.
Now, we repeat the conducted experiments in Section V-A
with step size of 0.1 (i.e., 50 trials per function
for
per jumping rate value
). Due
to space limitations, we do not show all the results, only the

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

TABLE VII
THE SUMMARIZED RESULTS FROM TABLES IV, XII, AND XIII.
n AND n
ARE THE NUMBER OF FUNCTIONS FOR WHICH
IS THE NUMBER
DE OUTPERFORMS ODE AND VICE VERSA. n
OF UNSOLVED FUNCTIONS BY THE ALGORITHM (SR = 0)

obtained optimal value for the jumping rate with respect to the
success performance as given in Table IX.
As seen, the optimal values for the jumping rate are dis. However, jumping
tributed over the discrete interval
rates of 0.3 and 0.6 are repeated more than other values in
this table. Higher jumping rates mostly belong to the low-dimensional functions and lower ones to the high-dimensional
functions. The average value of the obtained optimal jumping
is equal to 0.37 for our test functions.
rates
Some sample graphs (SP versus ) are shown in Fig. 2 to
illustrate the effect of the jumping rate on success performance.
indicates the success performance
The point specified by
show
of the DE; the rest of points
the success performance of ODE. As mentioned before, we can
observe a sharp increase in the SP for hard functions (e.g.,
,
, and
) on higher jumping rates. Also, the SP
decreases for easy functions by increasing the jumping rate (see
, and
). Almost a smooth behavior for all functions
is recognizable for
(it was observed
even for many functions which their graphs are not presented
could be more
here). Hence, working in this interval
reliable for unknown optimization problems.
Results analysisLike DEs other control parameters, the
optimal jumping rate should have a problem-oriented value.
for
Our limited experiments suggest the range of
an unknown optimization problem. A first attempt can be
. Furthermore, for high-diconducted with
mensional problems, a smaller jumping rate is suggested.
G. Experiment Series 7: ODE Versus FADE
The primary purpose of this work is to introduce the notion
of opposition into the design and implementation of DE and
demonstrate its benefits. Many other extensions of DE, if not all,
can also be reconsidered to incorporate the opposition concept.
In this sense, ODE should be regarded as an example and not
as competitor to other DE versions. However, in order to assess
the performance of ODE, a comparison with at least one other
algorithm may be beneficial.
We have compared ODE with the FADE method of Liu and
Lampinen [13]. They tested FADE for ten well-known benchmark functions, of which we have nine of them in our testbed.
The comparison strategy is different for this experiment. The
algorithms are run 100 times. Subsequently, for equal (fixed)
NFCs, the average and standard deviation of the best solutions
are calculated for the purpose of comparison. The same settings for parameters [13] have been used in the current experiment to assure a fair comparison. The population size is equal

73

and instead of using the generation number for DE and

to
ODE an equal number of function evaluations have been used
as the termination criteria (since in each generation NFCs for DE and ODE is not equal). The dimension of
the functions, corresponding generation numbers, and obtained
results (best mean and standard deviation of 100 runs) for DE,
ODE, and FADE are given in Table X. Results for FADE are
taken from [13,Table VI, p. 459]. A t-test at a 0.05 significance
level has been used to compare results of ODE against those of
DE and FADE.
Results analysisAccording to the t-test, ODE performs
better than DE on 9 functions (out of 16). There is no difference
for the rest of functions. ODE surpasses FADE on 12 functions,
and they perform the same on the rest. So, DE and FADE cannot
show better results than ODE even on one function. Although,
the comparison of a nonadaptive algorithm (ODE) to an adaptive one (FADE) is not fair, interestingly, the results confirm that
ODE performs outstandingly, it is not only better than DE but
also better than FADE. It is important to consider here that ODE
performs better than the original DE according to the current
comparison strategy as well.
VI. CONCLUSION
In this paper, the concept of OBL has been employed to accelerate DE. OBL was utilized to introduce opposition-based population initialization and opposition-based generation jumping.
By embedding these two steps within DE, ODE was proposed.
We can summarize the experimental results as follows.
DE and ODE were compared for different problem dimen, and
); the results confirm that ODE persions (
forms better over high-dimensional problems. For these
kind of problems, a large population size is required. On
the other hand, ODE performs well with larger population sizes. These two facts support each other and make
ODE more suitable for higher dimensional problems. Further study is required to solidify this expectation.
By replacing opposite points with uniformly generated
random points in the same variables range, the AR was
reduced by 57%. Therefore, the contribution of opposite
points to the acceleration process was confirmed and was
not reproducible by additional random sampling.
DE and ODE were compared for different population sizes
, and
). ODE performed better for larger
(
population sizes, which is usually required for more complex problems. In order to achieve better results for smaller
population sizes, small jumping rates are suggested.
Comparison of DE and ODE over various mutation strategies was also conducted. For all cases, ODE performed the
same or better than DE with respect to the NFCs, average
SR, and other performance measures.
The influence of the jumping rate was studied and the range
was recommended for an unknown optimization
problem. Most of the functions presented a reliable acceleration improvement and almost a smooth behavior in this
interval. Although, the optimal jumping rate can be somewhere out of this range, higher jumping rates are generally
not recommended.

74

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

TABLE VIII
THE SUMMARIZED RESULTS FROM TABLES I AND II.
NFC IS THE SUMMATION OF THE NUMBER OF FUNCTION CALLS
IS THE NUMBER OF UNSOLVED
(JUST FOR THE FUNCTIONS WHICH ALL EIGHT COMPETITORS COULD SOLVE). n
AND n
ARE THE NUMBER OF FUNCTIONS FOR WHICH DE OUTPERFORMS ODE AND
FUNCTIONS (SR = 0).n
VICE VERSA. N IS THE NUMBER OF FUNCTIONS FOR WHICH THE ALGORITHM COULD OUTPERFORMS OTHER
IS THE AVERAGE ACCELERATION RATE
ALGORITHMS. AR

TABLE IX
OPTIMAL JUMPING RATE J
FOR ALL TEST FUNCTIONS WITH
RESPECT TO THE SUCCESS PERFORMANCE (SP) ON INTERVAL
(0; 0:6] WITH STEP SIZE OF 0.1

Schwefels problem 1.2

Rosenbrocks valley

Rastrigins function

The main motivation for the current work was utilizing the
notion of opposition to accelerate DE. In order to have a comparison with other methods (other than the original DE), ODE
was also compared with FADE. The results clearly confirmed
that ODE performs better than FADE in terms of convergence
rate and solution accuracy on the utilized 58 benchmark functions.
Utilizing opposite numbers to accelerate an optimization
method is a new concept. Further studies are still required to
investigate its benefits, weaknesses, and limitations. This work
can be considered as a first step in this direction. The main
claim is not defeating DE or any of its numerous versions but
to introduce a new notion into optimization via metaheuristicsthe notion of opposition.
Possible directions for future work include the adaptive setting of the jumping rate, proposing other possibilities to implement ODE (e.g., opposition-based mutation strategies), and applying the same or similar scheme to accelerate other population-based methods (e.g., GA and PSO).

Griewangks function

Sum of different power

Ackleys path function

Beale function
APPENDIX A
LIST OF BENCHMARK FUNCTIONS
Sphere model
Colville function
Axis parallel hyperellipsoid

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

75

TABLE X
COMPARISON OF DE, ODE, AND FUZZY ADAPTIVE DE (FADE). MEAN BEST AND STANDARD DEVIATION (STD DEV) OF 100 RUNS ARE REPORTED.
: . T-TEST
FOR THE DE AND ODE, THE EQUAL NUMBER OF FUNCTION CALLS ARE USED INSTEAD OF GENERATION NUMBERS N
IS USED TO COMPARE ODE AGAINST DE AND FADE. INDICATES THE T -VALUE OF 99 DEGREE OF FREEDOM IS SIGNIFICANT AT
INDICATES OPTIMAL MINIMUM OF THE FUNCTION
A 0.05 LEVEL OF SIGNIFICANCE BY TWO-TAILED T-TEST. f

( 2 #Gen )

Easom function

where

Hartmann function 1

Six Hump Camel back function

where

Levy function

Hartmann function 2

Matyas function

76

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

TABLE XI
NAME OF PROBLEMS FOR f -f

Zakharov function

Braninss function

Fig. 2. Graphs of success performance (SP) versus jumping rate (J 2 (0; 0:6]
with step size of 0.1) for some sample functions. The point declared by J = 0
shows the SP of the DE; the rest of points (0:1; 0:2; 0:3; 0:4; 0:5; 0:6) show the
SP of the ODE.

Perm function

Schwefels problem 2.22

Schwefels problem 2.21

Michalewicz function

Step function

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

TABLE XII
COMPARISON OF DE AND ODE (

= 50)

77

TABLE XIII
COMPARISON OF DE AND ODE (

= 200)

Kowaliks function
Quartic function, i.e., noise

where and
next page.

is shown in the equation at the top of the

78

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION, VOL. 12, NO. 1, FEBRUARY 2008

Shekels Family

Inverted cosine wave function (Masters)

The names of functions

are listed in Table XI.
For more information about these functions, see [31].

where

APPENDIX B
COMPLEMENTARY RESULTS
and
The results for
Tables XII and XIII, respectively.

are given in

ACKNOWLEDGMENT
The authors would like to thank Prof. X. Yao and three anonymous referees for their detailed and constructive comments that
helped us to increase the quality of this work.

Tripod function

REFERENCES

where
for
, otherwise,
De Jongs function 4 (no noise)

Alpine function

Schaffers function 6

Pathological function

[1] R. Storn and K. Price, Differential evolutionA simple and efficient

adaptive scheme for global optimization over continuous spaces,
Berkeley, CA, Tech. Rep. TR-95-012, 1995.
[2] , Differential evolutionA simple and efficient heuristic for
global optimization over continuous spaces, in Journal of Global
Optimization 11. Norwell, MA: Kluwer, 1997, pp. 341359.
[3] K. Price, R. M. Storn, and J. A. Lampinen, Differential Evolution: A
Practical Approach to Global Optimization (Natural Computing Series), 1st ed. New York: Springer, 2005, ISBN: 3540209506.
[4] J. Vesterstroem and R. Thomsen, A comparative study of differential
evolution, particle swarm optimization, and evolutionary algorithms on
numerical benchmark problems, Proc. Congr. Evol. Comput., vol. 2,
pp. 19801987, 2004.
[5] J. Andre, P. Siarry, and T. Dognon, An improvement of the standard
genetic algorithm fighting premature convergence in continuous optimization, Advance in Engineering Software 32, pp. 4960, 2001.
[6] O. Hrstka and A. Kucerov, Improvement of real coded genetic algorithm based on differential operators preventing premature convergence, Advance in Engineering Software 35, pp. 237246, 2004.
[7] H. R. Tizhoosh, Opposition-based learning: A new scheme for machine intelligence, in Proc. Int. Conf. Comput. Intell. Modeling Control and Autom., Vienna, Austria, 2005, vol. I, pp. 695701.
[8] , Reinforcement learning based on actions and opposite actions,
in Proc. ICGST Int. Conf. Artif. Intell. Mach. Learn., Cairo, Egypt,
2005.
[9] , Opposition-based reinforcement learning, J. Advanced
Comput. Intell. Intelligent Inform., vol. 10, no. 3, pp. 578585, 2006.
[10] M. Shokri, H. R. Tizhoosh, and M. Kamel, Opposition-based Q()
algorithm, in Proc. IEEE World Congr. Comput. Intell., Vancouver,
BC, Canada, 2006, pp. 646653.
[11] M. Ventresca and H. R. Tizhoosh, Improving the convergence of
backpropagation by opposite transfer functions, in Proc. EEE World
Congr. Comput. Intell., Vancouver, BC, Canada, 2006, pp. 95279534.
[12] S. Rahnamayan, H. R. Tizhoosh, and M. M. A Salama, Opposition
versus randomness in soft computing techniques, Elsevier J. Applied
Soft Comput., Aug. 2006, submitted for publication.

RAHNAMAYAN et al.: OPPOSITION-BASED DIFFERENTIAL EVOLUTION

[13] J. Liu and J. Lampinen, A fuzzy adaptive differential evolution algorithm, Soft Computing-A Fusion of Foundations, Methodologies and
Applications, vol. 9, no. 6, pp. 448462, 2005.
[14] R. Storn, On the usage of differential evolution for function optimization, in Proc. Biennial Conf. North Amer. Fuzzy Inf. Process. Soc.,
1996, pp. 519523.

Self[15] J. Brest, S. Greiner, B. Boskovic , M. Mernik, and V. Zumer,

adapting control parameters in differential evolution: A comparative
study on numerical benchmark problems, IEEE Trans. Evol. Comput.,
vol. 10, no. 6, pp. 646657, Dec. 2006.
[16] J. Sun, Q. Zhang, and E. P. K. Tsang, DE/EDA: A new evolutionary
algorithm for global optimization, Inf. Sci., vol. 169, pp. 249262,
2005.
[17] J. Teo, Exploring dynamic self-adaptive populations in differential
evolution, Soft ComputingA Fusion of Foundations, Methodologies
and Applications, vol. 10, no. 8, pp. 673686, 2006.
[18] M. M. Ali and A. Trn, Population set-based global optimization algorithms: Some modifications and numerical studies, Comput. Oper.
Res., vol. 31, no. 10, pp. 17031725, 2004.
[19] D. K. Tasoulis, N. G. Pavlidis, V. P. Plagianakos, and M. N. Vrahatis,
Parallel differential evolution. [Online]. Available: http://citeseer.ist.
psu.edu/661975.html
[20] Y.-J. Shi, H.-F. Teng, and Z.-Q. Li, Cooperative Co-evolutionary differential evolution for function optimization, in Proc. 1st Int. Conf.
Advances in Natural Comput., Changsha, China, Aug. 2729, 2005,
pp. 10801088.
[21] J. Sun, Q. Zhang, and E. Tsang, DE/EDA: A new evolutionary algorithm for global optimization, Inf. Sci., vol. 169, pp. 249262, 2004.
[22] N. Noman and H. Iba, Enhancing differential evolution performance
with local search for high dimensional function optimization, in Proc.
Conf. Genetic Evol. Comput., Washington, DC, 2005, pp. 967974.
[23] H.-Y. Fan and J. Lampinen, A trigonometric mutation operation to
differential evolution, J. Global Optim., vol. 27, no. 1, pp. 105129,
2003.
[24] P. Kaelo and M. M. Ali, Probabilistic adaptation of point generation
schemes in some global optimization algorithms, J. Optim. Methods
Softw., vol. 21, no. 3, pp. 343357, 2006.
[25] K. Price, An Introduction to Differential Evolution, D. Corne, M.
Dorigo, and F. Glover, Eds. London, U.K.: McGraw-Hill, 1999, pp.
79108, ISBN:007-709506-5, New Ideas in Optimization.
[26] G. C. Onwubolu and B. V. Babu, New Optimization Techniques in Engineering. New York: Springer, 2004.
[27] S. Das, A. Konar, and U. Chakraborty, Improved differential evolution
algorithms for handling noisy optimization problems, in Proc. IEEE
Congr. Evol. Comput., 2005, vol. 2, pp. 16911698.
[28] S. Rahnamayan, H. R. Tizhoosh, and M. M. A. Salama, Oppositionbased differential evolution algorithms, in Proc. IEEE Congr. Evol.
Comput., Vancouver, BC, Canada, 2006, pp. 73637370, (held as a part
of IEEE World Congr. Comput. Intell.).
[29] , Opposition-based differential evolution for optimization of
noisy problems, in Proc. IEEE Congr. Evol. Comput., Vancouver,
BC, Canada, 2006, pp. 67566763, (held as a part of IEEE World
Congr. Comput. Intell..
[30] V. K. Koumousis and C. P. Katsaras, A sawtooth genetic algorithm
combining the effects of variable population size and reinitialization to
enhance performance, IEEE Trans. Evol. Comput., vol. 10, no. 1, pp.
1928, Feb. 2006.
[31] M. M. Ali, C. Khompatraporn, and Z. B. Zabinsky, A numerical evaluation of several stochastic algorithms on selected continuous global
optimization test problems, J. Global Optim., vol. 31, pp. 635672,
2005.
[32] V. K. Koumousis and C. P. Katsaras, A sawtooth genetic algorithm
combining the effects of variable population size and reinitialization to
enhance performance, IEEE Trans. Evol. Comput., vol. 10, no. 1, pp.
1928, 2006.

79

[33] X. Yao, Y. Liu, and G. Lin, Evolutionary programming made faster,

IEEE Trans. Evol. Comput., vol. 3, no. 2, p. 82, July 1999.
[34] C. Y. Lee and X. Yao, Evolutionary programming using mutations based on the Lvy probability distribution, IEEE Trans. Evol.
Comput., vol. 8, no. 1, pp. 113, Feb. 2004.
[35] P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y.-P. Chen, A. Auger,
and S. Tiwari, Problem definitions and evaluation criteria for the CEC
2005 special session on real-parameter optimization, Nanyang Tech.
Univ., Singapore and KanGAL, Kanpur Genetic Algorithms Lab., IIT,
Kanpur, India, Tech. Rep., Rep. No. 2005005, May 2005.

Shahryar Rahnamayan (S06M06) received the

B.Sc. and M.S. degrees both (with Honors) in software engineering from the Shahid Beheshti University, Tehran, Iran. He is currently working towards the
Ph.D. degree in the field of evolutionary algorithms
at the University of Waterloo, Waterloo, ON, Canada.
His current research interests are evolutionary algorithms, machine intelligence, and computer vision.
During the Ph.D. program, he has published 15 conference papers and 4 journals.
Mr. Rahnamayan is a member of the Association
for Computing Machinery (ACM). He was awarded the Ontario Graduate Scholarship (OGS), the Presidents Graduate Scholarship (PGS), and the Canadian
Institute of Health Research (CIHR) Fellowship.

Hamid R. Tizhoosh received the M.S. degree

in electrical engineering from the University of
Technology, Aachen, Germany, in 1995 and the
Ph.D. degree in computer vision from the University
of Magdeburg, Magdeburg, Germany, in 2000.
From 1993 to 1996, he worked at Management
of Intelligent Technologies, Ltd. (M.I.T. GmbH),
Aachen, Germany, in the area of industrial image
processing. He was active as the Scientist in the
Engineering Department of Photon Dynamics,
Markham, Canada, until 2001. He also conducted
research in the Knowledge/Intelligence Systems Laboratory, University of
Toronto, ON, Canada, in 2001. Since September 2001, he has been a Faculty
Member at the Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada. His research encompasses machine intelligence,
computer vision, and opposition-based learning.

Magdy M. A. Salama (S75M77SM98F02)

received the B.Sc. and M.S. degrees in electrical
engineering from Cairo University, Cairo, Egypt, in
1971 and 1973, respectively, and the Ph.D. degree
in electrical engineering from the University of
Waterloo, Waterloo, ON, Canada, in 1977.
Currently, he is a Professor in the Electrical and
Computer Engineering Department, University of
Waterloo. His research interests include the operation and control of electric distribution systems
and biomedical engineering applications. He has
consulted widely with government agencies and the electrical authority.
Dr. Salama is a registered Professional Engineer in the Province of Ontario.