Received December 31, 2020, accepted January 12, 2021, date of publication January 26, 2021, date of current

version February 4, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3054636

An Adaptive Multi-Population Optimization

Algorithm for Global Continuous Optimization
ZHIXI LI , (Graduate Student Member, IEEE), VINCENT TAM , (Senior Member, IEEE),
AND LAWRENCE K. YEUNG , (Senior Member, IEEE)
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong
Corresponding author: Zhixi Li (lizx@eee.hku.hk)

ABSTRACT Nowadays, there are various optimization problems that exact mathematical methods are not
applicable. Metaheuristics are considered as efficient approaches for finding the solutions. Yet there are many
real-world problems that consist of different properties. For instance, financial portfolio optimization may
contain many dimensions for different sets of assets, which suggests the need of a more adaptive metaheuris-
tic method for tackling such problems. However, few existing metaheuristics can achieve robust performance
across these variable problems even though they may obtain impressive results in specific benchmark
problems. In this paper, a metaheuristic named the Adaptive Multi-Population Optimization (AMPO) is
proposed for continuous optimization. The algorithm hybridizes yet modifies several useful operations like
mutation and memory retention from evolutionary algorithms and swarm intelligence (SI) techniques in a
multi-population manner. Furthermore, the diverse control on multiple populations, solution cloning and
reset operation are designed. Compared with other metaheuristics, the AMPO can attain an adaptive balance
between the capabilities of exploration and exploitation for various optimization problems. To demonstrate
its effectiveness, the AMPO is evaluated on 28 well-known benchmark functions. Also, the parameter
sensitivity analysis and search behavior study are conducted. Finally, the AMPO is validated on its
applicability through a portfolio optimization problem as a challenging example of real-world applications.
The benchmark results show that the AMPO achieves a better performance than those of nine state-of-
the-art metaheuristics including the IEEE CEC winning algorithms, recent SI and multi-population/hybrid
metaheuristics. Besides, the AMPO can consistently produce a good performance in portfolio optimization.

INDEX TERMS Continuous optimization, evolutionary algorithm, metaheuristic, multi-population, portfo-

lio optimization, swarm intelligence.

I. INTRODUCTION many optimization problems in our daily lives, like finding

Optimization problems are prevalent in various scientific the shortest vehicle route to a destination [10] and resource
and engineering domains. For instance, in the field of arti- allocation to satisfy performance goals [11].
ficial intelligence, researchers often seek to optimize various Since many optimization problems are too complex to be
machine learning models, such as the hyper-parameter opti- solved with a satisfactory solution by exact mathematical
mization of Support Vector Machines [1], [2] and network approaches in a reasonable time, metaheuristic optimiza-
architecture search of Neural Networks [3]–[5], to obtain tion algorithms have recently captured extensive attention
a better performance. In finance, investors usually pursue and achieved some success in solving such problems [12].
an optimal portfolio aiming to maximize the return while In the past decades, a number of metaheuristic optimization
minimizing the risk [6], [7]. In industrial design and man- algorithms were proposed, among which most are inspired
ufacturing, engineers always encounter numerous optimiza- by some natural phenomena and developed based on all
tion problems for various products and scenarios, such as kinds of metaphors. Such algorithms can be roughly catego-
optimizing aerodynamic shapes for aircraft, cars, bridges [8], rized into evolutionary algorithms (EAs) [13]–[15], swarm
and optimizing supply chain management [9]. Also, there are intelligence (SI) [16]–[18] approaches, physics/chemistry-
based metaheuristics [19]–[22] and others [23]–[25]. EAs
The associate editor coordinating the review of this manuscript and are inspired by biological evolutionary processes. Genetic
approving it for publication was Jagdish Chand Bansal. Algorithm (GA) [26], Differential Evolution (DE) [27]

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
19960 VOLUME 9, 2021
Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

and Evolution Strategies [28] can be regarded as the algorithm with minor change(s), thus limiting their adaptiv-
well-known algorithms. SI algorithms imitate intelligent ity and effectiveness. Besides, the communication strategy
behaviors of creatures in nature. Among them, Particle between sub-populations should be explored to prevent from
Swarm Optimization (PSO) [29] derived from the swarm any premature convergence [51].
behavior of birds and fishes is an early work. Until now, The above issues motivate us to propose the presented
the research of SI algorithms has been very active such metaheuristic called the Adaptive Multi-Population Opti-
that new algorithms using various metaphors were pro- mization Algorithm (AMPO). The AMPO algorithm is a
posed from time to time. Some representative examples metaheuristic for continuous optimization problems. The
include the Ant Colony Optimization (ACO) [30], Artifi- algorithm combines some useful operations from EAs and
cial Bee Colony (ABC) algorithm [31], Grey Wolf Opti- SI approaches. More importantly, the AMPO can achieve
mizer (GWO) [32], Social Spider Algorithm (SSA) [33], an adaptive balance between exploitation and exploration
Whale Optimization Algorithm (WOA) [34], Grasshop- through the unique design of the diversification of the search
per Optimization Algorithm (GOA) [35], Earthworm Opti- strategies in a multi-population manner. The solution cloning
mization Algorithm (EOA) [36], Moth Search (MS) [37], and reset operations serve as a more efficient communi-
Naked Mole-Rat Algorithm (NMA) [38], Monarch Butter- cation mechanism between the sub-populations. Therefore,
fly Optimization (MBO) [39], Harris Hawks Optimization the algorithm can be applied to solve various optimization
(HHO) [40], Sailfish Optimizer (SFO) [41] and Slime Mould problems more effectively and efficiently.
Algorithm (SMA) [42]. To demonstrate its effectiveness and efficiency, the AMPO
Although some metaheuristics exist, the number of is evaluated on 28 well-known benchmark functions in terms
effective algorithms is still small relative to the board spec- of solution quality measure, convergence rate, scalability,
trum of optimization problems. Undoubtedly, this moti- stability and computational cost. Furthermore, the parameter
vates researchers to continuously develop new algorithms sensitivity analysis and search behavior study are conducted
to fill the gap. On the other hand, some recently proposed to investigate the factors that contribute to the success of
metaphor-based algorithms are criticized due to the abuse of the algorithm, which can provide useful guidelines for future
metaphor [43], and to some extent their ultimate performance research and applications. Lastly, the AMPO is applied to a
particularly in solving real-world problems is questionable. significant computational finance application, i.e. portfolio
One of the main obstacles is that many practical problems optimization. In the benchmark function tests, the optimiza-
have different characteristics. For instance, financial portfolio tion results of the AMPO are compared with nine state-of-
optimization can be with different dimensions given by dif- the-art metaheuristics, namely three recent SI approaches,
ferent sets of assets, which implies an optimization algorithm three IEEE CEC winning algorithms and three advanced
with a high adaptivity is required for resolving such problems. multi-population and hybrid metaheuristics. In the portfolio
In spite of some studies [44]–[47] applied some metaheuris- optimization problem, the AMPO algorithm competes with
tics to portfolio optimization, the scalability and adaptivity three other algorithms that achieved outstanding performance
of such approaches are not examined carefully yet. Also, in this problem reported in the literature.
the scale of the datasets the OR-Library [48] frequently used The rest of this paper is organized as follows. In Section II,
by many researchers is relatively small, i.e. up to 225 assets the AMPO is described in detail, including its definitions,
only. Accordingly, many existing metaheuristics that obtain operations and implementations. The design ideas of the
impressive results in some benchmark problems usually fail AMPO and its differences from other metaheuristics are
to solve such challenging practical problems. discussed there as well. Section III covers the experimental
In fact, the capabilities of exploration and exploitation results and the related discussion on the benchmark function
should be carefully considered when designing an optimiza- tests. The parameter sensitivity analysis and search behavior
tion algorithm [49]. Both can be balanced through a diver- study of the AMPO are analyzed and discussed in Section IV
sity control on the population [50]. Clearly, most of the and Section V respectively. The AMPO is applied to solve the
aforementioned metaheuristics are single-population-based financial portfolio optimization in Section VI. We conclude
algorithms. While in recent years, devising an optimization this work and shed light on various potential future directions
algorithm in a multi-population framework is an emerging in Section VII.
research direction for developing a more effective meta-
heuristic because the population diversity can be better main- II. THE PROPOSED ALGORITHM
tained, and also different areas can be searched simultane- A. DEFINITIONS
ously [51]. Some recently proposed multi-population-based The AMPO, originally from our preliminary study [55],
metaheuristics include the Multi-Swarm Particle Swarm is formally proposed here for solving unconstrained, static
Optimization [52], Multi-Population Ensemble Differen- and single-objective continuous optimization problems.
tial Evolution (MPEDE) [53], and Multi-Population Fruit Unlike most of the population-based metaheuristic algo-
Fly Outpost Optimization Algorithm (MPFOA) [54]. How- rithms, the AMPO is essentially a multi-population-based
ever, most of the existing studies construct sub-populations metaheuristic that diversifies the search strategies to enhance
only relying on the same search strategy of the original the optimization capability and adaptivity. Such so-called

VOLUME 9, 2021 19961

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

adaptivity is that the algorithm is able to dynamically change LSt+1

i = Gaussian(0, σit+1 ) × X ti (6)
its search behavior at the time it is run based on information X i+1 = X i + LSt+1 (7)
i
available so as to adapt to different problems.
The AMPO algorithm comprises individuals, which are where LStiis the search step size of the local search
further classified into five sub-populations (also called group individual i at the t th iteration, γ ∈ (0, 1) is the
groups) including the random search group, global search constant decay rate, and Gaussian(0, σit+1 ) is a random
group, local search group, leader group and migrating group. number generator based on a Gaussian distribution with
Except for the leader group and migrating group, the sizes of a mean of zero and a standard deviation of σit+1 .
these sub-populations are dynamic during the search. Also, As the value of γ is smaller than 0, the updated standard
different search strategies are assigned to the individual(s) deviation σit+1 input to the Gaussian random number
in each sub-population. For convenience, the global search generator will decrease with the number of iterations
group, local search group and leader group are uniformly going up. Then the updated search step size will be more
called the source group. likely smaller (in absolute value). Thus, a local search
• Definition 1: An Individual group individual will tend to search for a better solution
In the AMPO, the population size is denoted by Npop . around its previous one.
Each individual has a feasible solution as shown below. Also, the local search group is designed with the trans-
formation probabilities donated by
X ti = [xi,1
t
, xi,2
t
, . . . , xi,n
t T
] (1)
PLS
LS ∈ (0, 1)
where i ∈ {1, 2, . . . , Npop } represents the index of an
PLS
GS ∈ (0, 1)
individual, X ti denotes the feasible solution of the ith
individual at the t th iteration, and n is the dimensionality s.t. PLS LS
LS + PGS = 1 (8)
of the problem, i.e. the total number of the decision where PLS LS
LS and PGS represent the transformation prob-
variables. abilities of an individual of the random search group
• Definition 2: The Random Search Group paired up with a local search group individual trans-
The individuals of the random search group basically forming to an individual of the local search group (LS)
carry out random search in the entire search space, i.e. or the global search group (GS) in the transformation
X ti = U (LB, UB) (2) operation.
• Definition 5: The Leader Group
where LB = [lb1 , lb2 , . . . , lbn ]T and UB = In the AMPO, there is always only one individual in the
[ub1 , ub2 , . . . , ubn ]T each denote the lower bounds and leader group. Such individual represents the best solu-
upper bounds of the corresponding domain of the vari- tion obtained by the population so far, possibly except
ables being considered, and U (·) is a random number for the migrating group. As shown in (9), the update of
generator function based on the uniform distribution. its solution is paused.
An individual of the random search group can possibly
be transformed into the global search group or the local X t+1
i = X ti (9)
search group by the transformation operation that will Similarly, the leader group is also associated with the
be introduced in Section II-B4. transformation probabilities PLD LD
LS and PGS representing
• Definition 3: The Global Search Group the transformation probabilities of a random search
The individuals of the global search group with rela- group individual after being paired up with the individ-
tively larger search step sizes perform global search. The ual of the leader group.
solution of a global search group individual at the t th
iteration is updated by the following equations. PLD
LS ∈ (0, 1)
PLD
GS ∈ (0, 1)
GSt+1
i = w × GSti + r × (gbest t − X ti ) (3)
s.t. PLD LD
LS + PGS = 1 (10)
X t+1
i = X ti + GSt+1
i (4)
• Definition 6: The Migrating Group
where GSti = [gsti,1 , gsti,2 , . . . , gsti,n ]T is called the
The migrating group conducts the search independently
search step size of the global search group individual
and possibly contributes some potentially better solu-
i at the t th iteration, w ∈ (0, 1) is the weighting factor,
tions to the leader group by the migration operation that
gbest t is the best solution obtained by the population up
will be described in Section II-B5.
to the t th iteration, and r is a random number produced
The sub-population size of the migrating group is deter-
by the standard uniform distribution of U (0, 1)
mined by the partition rate PR. Specifically, its size is
• Definition 4: The Local Search Group
Npop × (1 − PR). In this thesis, the migrating group is
The solution of a local search group individual is
powered by a classical Differential Evolution (DE) algo-
updated by
rithm called DE/rand/1/bin [27], [56]. Its user-controlled
σit+1 = γ × σit (5) parameters follow the suggested values in [57].

19962 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

B. OPERATIONS individuals at each iteration. Meanwhile, the previous indi-

In the AMPO algorithm, the initialization, selection, update, vidual in the leader group is downgraded to the local search
transformation, migration and reset are six essential opera- group individual. Finally, the stored variable gbest t used by
tions. the global search group individuals in (3) is updated as well.
1) INITIALIZATION
3) UPDATE
At the starting point of the algorithm, the iteration number t
Algorithm 2 clearly manifests the pseudo-code of the update
is initialized to 0. The solution of each individual is randomly
operation. The update operation is carried out at each itera-
generated in the search space as shown below.
tion to make individuals search solutions. Depending on the
X t=0
i = LB + r × (UB − LB) (11) sub-population type of individuals, different search strategies
are performed according to (2)—(9).
where r ∼ U (0, 1) is a randomly generated number. During the update operation, the boundary constraints are
Then the values of SG and δ in the update functions of the handled as shown in (14).
global search group and local search group are initialized
xi,j , lbj ≤ xi,j ≤ ubj

by (12) and (13) respectively. t t

U (LB, UB)
t
x̂i,j = lbj , xi,j
t < lb
j (14)
GSt=0 = (12)
i
ubj , xi,j > ubj
 t
10

δit=0 = U (0.1, 1) (13)
t is the constrained value of the jth variable of the
where x̂i,j
Lastly, apart from the migrating group individuals, all the solution of the individual i at the t th iteration.
other individuals are initialized as the random search group.
Algorithm 2 AMPO Update Operation
Algorithm 1 AMPO Initialization Operation
Input: All individuals: individuals
Input: User-controlled parameters: Npop , PLD LS Output: All individuals with updated solutions:
LS , PLS , PR,
γ and ω Optimization problem information individuals
Output: Newly created individuals: individuals 1 for each ind in individuals do
1 individuals ← ∅; 2 switch ind.type do
2 i ← 1; 3 case random search
3 while (i ≤ Npop ) do 4 Update the individual’s solution according
4 Initialize a new individual ind with a solution to to (2);
(11); 5 end
5 if (i < int(Npop ∗ PR)) then 6 case global search
6 Initialize the control factors in the update 7 Update the control facotrs of the individual
functions according to (12) & (13); according to (3);
7 ind.type ← ’random search’; 8 Update the individual’s solution according
8 else to (4);
9 ind.type ← ’migrating’; 9 end
10 end 10 case local search
11 Insert ind into individuals; 11 Update the control facotrs of the individual
12 i ← i + 1; according to (5) & (6);
13 end 12 Update individual’s solution according to
14 return individuals; (7);
13 end
14 case leader
Algorithm 1 shows the detailed implementation of such 15 Update the individual’s solution according
to (9);
initialization process. The inputs include the aforementioned 16 end
user-controlled parameters as well as information of the prob- 17 case migrating
lem to be optimized, i.e. the population size (Npop ), partition 18 Update the individual’s solution according to
rate for the migrating group (PR), transformation probabili- the DE/rand/1/bin algorithm;
LS 19 end
ties (PLD
LS and PLS ), control factors used in the search func- 20 endsw
tions (γ and w), dimensionality of the optimization problem 21 Handle the bound constraints according to (14);
(D), and lower and upper bounds (boundi and boundu ). The 22 end
23 return individuals;
outputs are the initialized individuals.

2) SELECTION
In the emphselection operation, the individual with the best 4) TRANSFORMATION
solution obtained so far among the random search group, The transformation operation, as the communication strategy
global search group and local search group is selected as between the sub-populations, is a critical factor for the suc-
the leader group individual after fitness evaluations for all cess of the AMPO. Essentially, the goal of the transformation

VOLUME 9, 2021 19963

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

Algorithm 3 AMPO Transformation Operation

Input: The source individuals: sourceInds
⊆ {local search, global search, leader} The
target individuals: targetInds ⊆ {random search}
The problem dimensionality: n
Output: The individuals with updated solutions
1 // A solution with a smaller fitness
value means a higher quality
solution.
2 Sort sourceInds by the ascending order of the fitness
values;
3 Sort targetInds by the descending order of the fitness
values;
4 for each sourceInd ∈ sourceInds do
5 if (|targetInds| > 0) then
FIGURE 1. AMPO Solution Cloning Mechanism. 6 Select the first targetInd from the targetInds;
7 Get the transformation probabilities of the
sourceInd;
operation is to spread the solution information among all the 8 switch sourceInd.type do
individuals so as to empower the search effectiveness. 9 case leader
The detailed implementation of the transformation oper- 10 if (rand(0, 1) ≤ PLDLS ) then
ation is demonstrated in Algorithm 3, where a three-step 11 transformingTtype ← ’local search’;
mechanism is performed: 12 else
13 transformingType ← ’global
First of all, the individuals belonging to the leader group, search’;
global search group and local search group assemble the 14 end
source group with all the solution qualities sorted in descend- 15 end
ing order while the individuals of the random search group 16 case local search
form the target group with all the solution qualities sorted in 17 if (rand(0, 1) ≤ PLSLS ) then
18 transformingType ← ’local search’;
ascending order. 19 else
Secondly, each sorted individual of the source group is 20 transformingType ← ’global
paired up with the corresponding individual in the target search’;
group. Then, each individual of the target group will be trans- 21 end
22 end
formed in the local search group or the global search group 23 case global search
depending on the transformation probabilities (as defined in 24 transformingType ← ’global search’;
Section II-A) of that paired individual in the source group. 25 end
Lastly, a solution cloning mechanism is executed on each 26 endsw
transformed individual as provided below. 27 targetInd.type ← transformingType;
28 switch transformingType do
• Transform to a global search group individual 29 case global search
When a random search group individual is transformed 30 for idx = 0 to n − 1 do
to a global search group individual by a source group 31 if rand(0, 1) ≤ 0.5 then
32 targetInd.solution[idx] ←
individual, a single-side crossover is adopted. As illus- sourceInd.solution[idx];
trated in Fig. 1a, each variable of the n-dimensional 33 end
solution of the random search group individual is ran- 34 end
domly replaced by that of the source one with a fixed 35 end
36 case local search
probability of 0.5. 37 targetInd.solution ←
• Transform to a local search group individual sourceInd.solution;
When a random search group individual transformed 38 end
by a source group individual becomes a local search 39 endsw
group individual, the whole solution of the source group 40 Remove targetInd from targetInds;
41 end
individual is duplicated to the random search group one 42 end
as shown in Fig. 1b.
Such process is conducted at each iteration until all the
individuals of the random search group are transformed, or all
the individuals in the source group have been paired up with lation so that its search capability of solving more complex
the individuals of the random search group. optimization problems can be enhanced.
Since the migrating group works independently, it may
5) MIGRATION discover better solutions compared with all the other sub-
The migration operation is introduced as an additional oper- populations. One simple migration strategy is that once a
ation of the AMPO framework to further diversify the popu- better solution is found, the solution will be migrated to the

19964 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

whole population directly. Yet this heuristic operation may parameters used in the update operation are re-initialized.
break search trajectories of the concerned AMPO algorithm Then a downgraded sub-population type is assigned. The
by itself, thus possibly leading to an unsatisfactory and unsta- output is the reset individuals.
ble performance. Therefore, whenever the migrating group
finds a better solution, an adaptive probability is predefined Algorithm 5 AMPO Reset Operation
to migrate such solution to replace the current best solution
Input: Individuals: resetIndividuals
maintained by the leader group individual. Such solution ⊆ {local search, global search} Random search
migrating is under the present condition as illustrated in individuals: randomIndividuals
Algorithm 4. ⊆ {random search}
Output: Updated reset individuals: resetIndividuals
1 if (|randomIndividuals| = 0) then
Algorithm 4 AMPO Migration Operation
2 resetPercent ← rand(0.1, 0.9);
Input: The leader group individual: leaderInd The 3 resetNum ← |resetIndividuals| ∗ resetPercent ;
migrating group individuals: migratingInds The 4 rIns ← {the first resetNum individuals} ⊆
current iteration no: t The total number of resetIndividuals;
iterations: T 5 for each ind ∈ rIns do
Output: The updated leader group individual: leaderInd 6 Re-initialize the solution of the ind referring to
The updated stored best solution: gbest t (11) ;
1 hbestSolution, bestFitnessi ← migratingInds ; 7 Re-initialize the control factors of the ind
2 if (rand(0, 1) ≤ (0.5 ∗ t/T )) then according to (12) & (13);
3 if (bestFitness < leaderInd.fitness) then 8 switch ind.type do
4 leaderInd.solution ← bestSolution; 9 case local search
5 leaderInd.fitness ← bestFitness; 10 ind.type ← ’global search’ ;
6 gbest t ← bestSolution 11 end
7 end 12 case global search
8 end 13 ind.type ← ’random search’ ;
9 return leaderInd, gbest ;
t
14 end
15 endsw
16 end
17 end
6) RESET 18 return resetIndividuals;
Due to the powerful transformation operation, all the indi-
viduals of the random search group may be transformed
into other groups very soon, thus possibly resulting in a low C. THE ALGORITHMIC FLOW OF THE AMPO
exploration capability. Accordingly, once no random search Fig. 2 manifests the algorithmic flow of the AMPO. At first,
group individuals are available, the reset operation as a restart the user-controlled parameters of the algorithm and the objec-
mechanism will be performed to carefully reset a specified tive function of the optimization problem are given as the
percentage of the individuals from the global search group input to run the AMPO.
and local search group to continue with the exploration pro- First, the initialization operation presented in Algorithm 1
cess. This control parameter is denoted by resetPercent as is performed.
a randomly generated number between 0.1 and 0.9 at each Then, during the optimization process, the operations are
iteration. This design can help avoid losing all the search executed successively: i) the function evaluation is to cal-
information accumulated so far during the search process. culate the results of the objective function for all individu-
In other words, the leader group that stores the best-so-far als. Such results are also called fitness values; ii) the selec-
solution is never reset. tion operation is conducted; iii) the transformation operation
Furthermore, a downgrading mechanism is implemented described in Algorithm 3 is executed to spread the solution
in the reset operation: an individual picked up from the information among the population; iv) the migration is to
local search group moves into the global search group while possibly migrate a better solution from the migrating group
another selected individual of the global search group is to the leader group (see Algorithm 4); v) the update operation
shifted into the random search group during the reset process. (see Algorithm 2) is to search solutions using different search
With the relaxation of the search restrictions through resetting strategies; vi) the reset operation (see Algorithm 5) is trig-
individuals, the search capability of the algorithm can be gered once no random search group individuals are available.
improved gradually to explore the other parts of the search Such process is executed iteratively until the predefined
space. termination criteria is satisfied, e.g. the maximum number of
The detailed implementation of this operation is given in iterations is reached. Finally, the best solution obtained by the
Algorithm 5. The individuals of the global search group algorithm is provided as the output.
and local search group are the input. They may be selected The source code of the AMPO can be found at
to be reset. In the operation, their solutions as well as the https://github.com/rayzxli/AMPO.

VOLUME 9, 2021 19965

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

effectively [59]. Nevertheless, it also has several shortcom-

ings. For instance, it will converge prematurely and have
low exploitation capability in the later search stage where
the particles are close to each other or global optimum [60].
Thus, keeping the advantages of the PSO, we first design a
memory retention mechanism that the leader group acts as;
second, the search guided by the global memory in the PSO is
equipped with the search strategy of the global search group
to perform global search.
For some newly proposed SI algorithms, such as the
WOA [34] and SSA [33], both simulate the foraging behav-
iors of animals. Basically, the inventors also followed the
basic idea of the PSO but partially modified the search strate-
gies to enhance the optimization ability. Taking the WOA as
an example, the WOA mathematically models the search for
prey and bubble-net attacking behavior of humpback whales
as the exploration and exploitation processes, respectively.
To a certain extent, this design readily improves the opti-
mization capability, but it still suffers premature convergence,
resulting in trapping in local optima easily [61].
Besides, the design of the AMPO is also motivated by the
GA, which is a typical EA algorithm. Since the GA has no
memory retention, and its crossover and mutation operators
are performed randomly, an individual’s good solution may
be lost if that individual is not selected, thus causing a lack
of fast convergence towards the optimal solution [62]. The
operations of the AMPO refer to the selection, mutation and
crossover operations in the GA. However, the AMPO has
many differences from the GA. First, in the GA, the individual
FIGURE 2. The Algorithmic Flow of the AMPO. with a higher quality solution may be selected with a higher
probability to participate in the crossover and mutation while
only the individual with the best-so-far solution is selected
D. THE DESIGN IDEAS AND DIFFERENCES FROM and stored in the memory in the AMPO, which overcomes the
EXISTING METAHEURISTICS aforementioned drawback of the GA. Second, the Gaussian
Exploration and exploitation are two important factors in mutation with a declining standard deviation is adopted for
designing an optimization algorithm [49], [50]. Exploration the local search group in the update operation in the AMPO
related to global search is the process of searching for new so that the exploitation ability can be adjusted dynamically.
solutions in the search space. Exploitation related to local Third, the crossover operator is modified to the single-side
search is the process of refining good solutions that have been crossover in the transformation operation.
discovered. In order to dynamically balance both capabilities Since the exploration and exploitation balance can be
for tackling different optimization problems, the design of the achieved through diversity maintenance of the popula-
AMPO algorithm has absorbed the strengths of EAs and SI tion [50], we build the algorithm based on a multi-population
approaches. Several advantageous operations are hybridized framework. Compared with other multi-population meta-
and modified in the AMPO, thereby constructing different heuristics, the AMPO contains an effective communication
search strategies in a multi-population manner. strategy between the sub-populations, in which the diversity
Most of the existing metaheuristics are SI algorithms. of the population can be dynamically maintained by the
A common feature of such algorithms is that they have selection, transformation, migration and reset operations for
memory retention that can store the (sub-)optimal solutions resolving different problems adaptively. By contrast, many
obtained by individuals in the population. The search can then existing studies adopted simple multi-population designs
be partly guided by the memory. For example, in the PSO, such as the latest developed MOFOA [54]. The MOFOA
the position of each particle represents a feasible solution, has no communication strategy between sub-populations,
which is updated by the best previous position of the particle and it only outputs the best solution selected from all
(local memory) and the position attained by the best particle sub-populations at the end of iterations. Besides, the sizes of
in the swarm (global memory). The PSO has a high global such sub-populations are fixed all the time.
search capability and fast computing speed [58]. Also, it is Moreover, the design of the migration in the AMPO
known to solve large-scale nonlinear optimization problems also provides a useful interface for researchers and users to

19966 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 1. Benchmark Functions.

integrate their own algorithms to the algorithmic framework • Group III: F11—F17 are scalable multi-modal
conveniently in tackling specific problems, which greatly functions;
increases the flexibility of the AMPO algorithm. In other • Group IV: F18—F20 are shifted and/or rotated
words, another metaheuristic can directly serve as the search functions;
algorithm to empower the migrating group without any mod- • Group V: F21—F28 are hybrid and composition
ifications. As the migration is a one-way operation, i.e. pass- functions.
ing the solution from the migrating group to the leader group, Such benchmark functions cover the most possible character-
this means the solution obtained by the AMPO framework is istics (uni/multi-modal, convex/non-convex, differentiable/
not be worse than that generated by the integrated algorithm non-differentiable and separable/non-separable) of real-world
itself. optimization problems. Generally, in F01—F05, each func-
tion has a fixed dimension with a lot of local optima. F06—
F17 are scalable so that the optimizing quality and scal-
III. THE BENCHMARK FUNCTION TESTS ability of the algorithm can be measured. F18—F20 are
A. THE BENCHMARK FUNCTIONS some complicated functions in which the shift and rotation
In this work, a suite of 28 well-known functions are selected transformations are assigned. F21—F28 are employed to
from [63], [64]. These benchmark functions have been widely evaluate the algorithm in handling the problems consisting
used in many previous studies to evaluate all kinds of meta- of different sub-components with different features.
heuristic algorithms. Table 1 lists the information about these
functions that can be divided into the following groups. B. THE EVALUATION METHOD
• Group I: F01—F05 are two-dimensional multi-modal In this part of the experiment, each benchmark function is
functions; tested for 31 runs for each algorithm. In each run, a max-
• Group II: F06—F10 are scalable uni-modal functions; imum number of function evaluations (MaxFEs) that is set

VOLUME 9, 2021 19967

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 2. The Comparative Results of the AMPO and Recent Swarm Intelligence Algorithms on the Benchmark Test.

to 104 × n (n is the dimension of the problem) serves as the C. THE COMPARISONS WITH THE STATE-OF-THE-ART
stopping criteria [63]. Except for the functions in Group I, n is ALGORITHMS
set to 30. 1) A COMPARISON WITH RECENT SWARM INTELLIGENCE
The experimental results are examined via the means and ALGORITHMS
standard deviations of the optimal fitness values attained by Three successful SI algorithms proposed recently, including
each algorithm in all the runs. For convenience, the results the aforementioned SSA and WOA and SFO [41], are com-
are converted to f (X) − f (X ∗ ) where f (X ∗ ) is the real global pared with the AMPO.
optimum of the function as listed in Table 1. Thus, the global Table 2 presents the comparative results of the AMPO and
optima for all the functions should be 0. Besides, a series the three competing algorithms. The statistical result of the
of Wilcoxon rank-sum tests [65] with a significance level Wilcoxon rank-sum test is presented next to the mean value
0.05 are performed so as to validate whether such results are for each algorithm: the ‘ ’ + represents that the performance
statistically significant or not. The null hypothesis is that the of the competing algorithm is better than that of the proposed
AMPO performs similarly with the competitor when solving AMPO algorithm with statistical significance; the ‘ ’ - indi-
each benchmark function. cates that the competing algorithm is significantly inferior to
In all the following experiments, the user-controlled the proposed algorithm; the ‘ ’ = means that the performance
parameters of each competing algorithm refer to its tuned of the two algorithms is consistent. It can be observed from
or suggested values reported in its corresponding litera- the table:
ture unless otherwise specified. Concerning the AMPO, • The AMPO overall outperforms the compared algo-
the parameter combination (Npop =50, PLD LS rithms in terms of the statistical test results. In particular,
LS =0.8, PLS =0.8,
PR =0.6, γ =0.9 and w =0.1) is utilized in all the remaining it discovers the exact best solutions in 12 cases, i.e. F01,
experiments of this paper. It is noted that this parameter F06—F10, F12—F14 and F16—F18. This reveals
set may not be the best one for solving all optimization that the AMPO is able to completely solve all uni-modal
problems. Therefore, the parameter tuning is still recom- and a few multi-modal problems in this test;
mended for dealing with any problem to achieve a better • Among all the benchmark functions, the AMPO pro-
performance. duces significantly better results when compared with
All algorithms are implemented in Python 3.6.9 and the SFO;
executed on a desktop computer installed with the • The optimization results of AMPO is better than those
Intel i9-7900X
R processor running at 3.3 GHz and 64 GB of the SSA in 23 functions. For Group I functions, the
of RAM. statistical tests show that the AMPO outperforms the

19968 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 3. The Comparative Results of the AMPO and CEC Winning Algorithms on the Benchmark Test.

SSA in F4 and F5 while it is inferior to the SSA algorithm have achieved remarkable success in the IEEE
in F2 and F3. It is noted that the AMPO attains the CEC competitions. In this part of the experiment, the AMPO
mean fitness (i.e. 4.3142E-13) that is much better than is compared with some winning algorithms: LSHADE [68],
the value (i.e. 6.4292E-02) generated by the SSA in LSHADE-cnEpSin (denoted by LS-cnEPsin later) [69] and
F03; however, the inferential statistics gives an opposite SHADE-ILS [70].
result. When investigating it further, we find that the Table 3 indicates such results in which the results of
SSA gets the exact global optima in 29 runs but falls the AMPO are put together to make the comparison more
into local optima that is far from the global optimum straightforward and convenient. From the table, we have the
in 2 runs. By contrast, the AMPO achieves the global following observations:
optimum in 12 runs, and it is close to the global optimum • According to the statistical results, the performance of
in the other runs. Except for F02, F03, F20 and F22, all the AMPO algorithm is as a whole better than those of
the results generated by the AMPO are better than those all the three CEC winning algorithms;
of the SSA with a statistical significance. On the other • Each competing algorithm itself outperforms the AMPO
hand, the optimization capability of the SSA seems to in 5 functions. Most of the better results generated by
be out of work in tackling F10 where it produces a large the LSHADE, LS-cnEpSin and SHADE-ILS algorithms
fitness value. On the contrary, the AMPO gets the global are located in Group I. By contrast, the performance
optimal solution exactly; of the AMPO is fairly good in coping with those 30-
• No functions in which the WOA outperforms the AMPO Dimensional functions. This suggests that the AMPO
are found. The performances of the two algorithms are enjoys few advantages to dealing with such low dimen-
consistent in 9 functions; sional problems;
• In addition to the statistical test, the average rank is • In terms of the descriptive statistics, the aver-
calculated for each algorithm on all functions. The aver- age rankings of the AMPO, LS-cnEpSin, LSHADE
age ranks are 1.11, 2.57, 2.86 and 3.21 for the AMPO, and SHADE-ILS are respectively 1.43, 2.86, 2.87
WOA, SSA and SFO, respectively. Given such results, and 2.89.
the WOA is ranked second, next only to the AMPO in
this test.
3) A COMPARISON WITH THE ADVANCED HYBRID AND
2) A COMPARISON WITH THE CEC WINNING ALGORITHMS MULTI-POPULATION ALGORITHMS
According to [66], the variants of the Successful-History The AMPO algorithm is also compared with three
based Adaptive Differential Evolution (SHADE) [67] advanced hybrid and multi-population algorithms, i.e.

VOLUME 9, 2021 19969

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 4. The Comparative Results of the AMPO and Hybrid and Multi-population Algorithms on the Benchmark Test.

the Multi-population Ensemble Differential Evolution • Regarding the descriptive statistics, the average rank-
(MPEDE) [53], Hybrid Firefly and Particle Swarm Opti- ings of the AMPO, MOFOA, MPEDE and HFPSO are
mization (HFPSO) [71], and MOFOA that is discussed in respectively 1.07, 1.96, 2.73, and 3.68.
Section II-D.
The detailed results are presented in Table 4, in which we D. THE CONVERGENCE TEST
can see that: In addition to the solution quality, we are also interested in
• The performance of AMPO still exceeds all the three the rate of convergence. Therefore, the convergence tests on
algorithms here; all the compared algorithms are carried out.
• The competing algorithms are also designed in hybrid Since each function is tested for 31 runs for each algorithm,
and multi-population manners. Nevertheless, only the the run with the median fitness result is selected to plot the
MPEDE algorithm achieves better results than our pro- convergence curve. Fig. 3—Fig. 7 display the convergence
posed algorithm in 2 out of all 28 functions. On the other results, in which the x-axis is the fitness function evalua-
hand, its results are worse than the AMPO in other 24 tions (FE) consumed and the y-axis is the best fitness value
functions; obtained so far. Also, as some algorithms may produce large
• For the functions in Group II & Group III, the per- fitness values at the beginning of running, the maximum value
formance between the MOFOA and AMPO algorithms of y-axis is limited to the median value of such finesses to
is almost consistent except for F07. Yet the MOFOA make the observation clearer. The findings are summarized
fails to deliver satisfactory results for the function in the in the following points:
Group I & Group IV compared to the AMPO. According • The AMPO generally converges faster than other algo-
to [54], the MOFOA also adopts the Gaussian mutation rithms and hence possesses a better convergence capa-
(called outpost mechanism) that is similar to our design bility for the tested optimization problems;
but with a difference that the variance is fixed as 1 • For Group I functions, the convergence speed of the
in the MOFOA. Such results imply that the MOFOA AMPO is a bit slower than a few compared algorithms.
has excellent convergence speed; however, a lack of Also, the optimization capability of the AMPO on
sufficient adaptivity of working on a wider range of these functions is weaker than those of the competitors,
problems. Generally speaking, the MOFOA is the sec- as stated in Section III-C. This finding suggests that the
ond best in this comparison in light of the statistical AMPO algorithm may have relatively few advantages in
result. solving these low-dimensional problems;

19970 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 3. The Results of Convergence Test (F01-F06).

• For F19 and F20, the convergence rates of the AMPO E. THE STABILITY ANALYSIS
are only next to those of the SHADE-ILS algorithm. Despite the standard deviations shown in Section III-C,
• In the remaining functions, the AMPO achieves the best the box plots of fitness variation are illustrated
performance over all the competitors in terms of the in Fig. 8—Fig. 12, in which we may observe the distribution
convergence rate. of the results generated by each algorithm, especially outliers

VOLUME 9, 2021 19971

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 4. The Results of Convergence Test (F07-F12).

in all the runs on the function. We have a few observations • In F19, the AMPO optimizer produces several outliers.
that: On the other hand, except for the SHADEILS, all the
• The body of the box of the AMPO is the shortest one other algorithms are unstable in optimizing this problem
among all algorithms in F1—F18 and F23—F28. The as well.
AMPO even has no outlier points in these cases. This Due to the effective multi-population design in which the
demonstrates that the performance of AMPO is very computational resources can be dynamically allocated to dif-
stable; ferent sup-population supported by different search strate-
• Only the AMPO algorithm gets zero values of the stan- gies (such search behavior is further revealed in Section V).
dard deviation in F27 and F28; In each run, the AMPO algorithm is able to adapt to the

19972 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 5. The Results of Convergence Test (F13-F18).

problem based on the obtained information. Thus, although costs between all optimizers. Table 5 lists the average execu-
the search trajectories of all the runs are different from each tion time of all the runs in terms of CPU seconds. It can be
other, the AMPO still has a quite stable performance. seen that:
• The AMPO runs the fastest in three cases, i.e. F7, F14
F. A COMPARISON OF THE COMPUTATIONAL COSTS and F17;
During running the tests presented in Section III-C, we also • Two other metrics are calculated to evaluate the overall
record the time performance to compare the computational time performance of each algorithm. The average time

VOLUME 9, 2021 19973

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 6. The Results of Convergence Test (F19-F24).

is the mean value of all times, and the average rank is time-consuming calculation of the Euclidean distance
the mean ranking of all times. In terms of the average and the covariance matrix.
rank, the AMPO is the second fastest algorithm, only
following the SSA, while it ranks first on the basis of
the average time; G. THE SCALABILITY TEST
• Given the overall ranking, the slowest algorithm Many practical optimization problems in computational
is the LS-cnEpSin algorithm, which involves the finance as well as other areas have various dimensions.

19974 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 7. The Results of Convergence Test (F25-F28).

The good scalability is very significant for a metaheuristic able to find more precise solutions around the best one at
algorithm to solve such problems. hand. As the AMPO is empowered by the adaptive search
Therefore, in addition to the 30-dimension benchmark framework, the algorithm can allocate more resources to the
function tests, the simulations on 50-D, 100-D and 200-D local search group, which helps the algorithm search for more
benchmark problems in Group II & III are performed to precise solutions. Such behavior is observed in Section V.
evaluate the scalability of the AMPO. In order to make a
thorough comparison, three compared algorithms that attain IV. THE PARAMETER SENSITIVITY ANALYSIS
good results in the above test, i.e. the WOA, LS-cnEpSin and Apart from the max number of iterations (or MaxFEs), there
MOFOA, are involved in the test. The results are presented are six user-controlled parameters in the AMPO: Npop , w, γ ,
in Table 6—Table 8, where the measurement is the mean PR, PLD LS
LS and PLS that are defined in Section II-A. As the selec-
fitness of all the 31 runs. tion of user-controlled parameters may dramatically affect the
From the tables, we can see that the AMPO algorithm still performance of one metaheuristic algorithm, we systemati-
achieves the best performance in all the problems no matter cally investigate the possible impacts of different parameter
with 50, 100 and 200 dimensions. Except for F11 and F15, settings on the performance of the AMPO according to the
the AMPO has exactly found the global optimal solutions. practice in our previous work [72].
In fact, the exploitation of an optimizer may dramatically The parameter set used in the above experiment (see
affect the ultimate fitness of the searched solution particularly Section III-B) is considered as the standardized parameter
in solving some high-dimensional problems like F07 and set for the subsequent discussion. Since it is very com-
F16. That is, due to a large-scale set of the decision variables, putationally expensive to explore all combinations of such
the fitness value of the optimization problem may become control parameters, the parameter sensitivity analysis test
very large as long as some variables of a solution are not focuses on a specific parameter while fixing all other param-
accurate enough. This means that the algorithm should be eters. For example, the population size in the standardized

VOLUME 9, 2021 19975

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 8. The Box Plot of Fitness Variation (F01-F06).

set is 50, whereas it is set to 10, 30, 100, or 200 for the IV and V respectively. The AMPO with the standardized
subsequent sensitivity analysis with all other parameters parameter set acts as the baseline algorithm whose results
remained unchanged. The test is conducted on a total of are presented in Section III-C. Also, a series of statistical
15 representative functions, i.e. F02, F03 and F04, F06, tests similar to those practices in Section III-B are per-
F08 and F10, F12, F15 and F17, F19 and F20 and formed to compare the tested algorithms with the baseline
F23, F25, F26 and F26 selected from Groups I, II, III, algorithm.

19976 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 9. The Box Plot of Fitness Variation (F07-F12).

The results are presented in Table 9—Table 14, where we a population size that is much smaller or larger than
can see that: the problem dimensionality tends to produce worse
• Under the same MaxFEs, the algorithm with the pop- results for these problems. For example, in the cases
ulation size of 30 overall achieves a better perfor- of Npop = 10 and Npop = 200, the algorithm
mance than that of the baseline algorithm in terms of is inferior to the baseline in five and six functions
the statistical results. It appears that the AMPO with respectively;

VOLUME 9, 2021 19977

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 10. The Box Plot of Fitness Variation (F13-F18).

• At least in these functions, the performance of the This parameter is used to decrease the standard deviation
AMPO is not very sensitive to the weighting factor w of the Gaussian function of the local search group. If the
that is used in the search step size update function of the value is too small, the search step size of the local search
global search group; group will decrease quickly, thus easily leading to a slow
• The selection of the constant decay rate γ has a sig- convergence to the global optima;
nificant impact on the performance when it is set In the case of PLD LS
• LS = 0.6 or PLS = 0.6, no difference in
to a small value, e.g. 0.1, 0.3 or 0.5 in this test. the performance is found between the tested algorithm

19978 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 11. The Box Plot of Fitness Variation (F19-F24).

and baseline algorithm. However, setting the parameter •A small or large value of PR may produce worse results
of PLD LS
LS or PLS to small values has a slight negative in this test.
effect on the performance of the algorithm observed in The above observations indicate that selecting medium
this experiment. Thus, setting these parameters to small values of Npop (e.g. 30 & 50), large values of γ (e.g. 0.9),
values is not advisable; and medium to large values of PLD LS (e.g. 0.6 & 0.8 ) and

VOLUME 9, 2021 19979

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 12. The Box Plot of Fitness Variation (F25-F28).

PLS
LS (e.g. 0.6 & 0.8) are suggested to improve the adaptivity search group, global search group and random search group
and effectiveness of the algorithm for optimizing various are recorded in this way.
problems. To determine the value of PR, the trial-and-error PD  
method is advised when solving different problems. d=1 Srijd
 
Srij = (15)
Pn D
V. THE SEARCH BEHAVIOR STUDY j=1 |Srij |
Sri = (16)
In this section, we further investigate the search behavior n
PI
of the AMPO, including the dynamics of the search step i=1 Sri
size and sub-population size. Ten representative benchmark Sr = (17)
I
functions are involved in the study. We re-run the AMPO on PR
r=1 Sr
these functions with the same Experimental Setup as stated S= (18)
R
in Section III-B except for the stopping criteria. In this study,
the search will be terminated once the MaxFEs or global where r, i, j and d represent the run no, iteration no, indi-
optimal solution is reached so that the observation can be vidual no and problem dimension, respectively; n is the
conducted more clearly. sub-population size, D means the total dimension of the
Firstly, for each function, the search step sizes of the problem, I is the total number of iterations, R is the number
local search group and global search group individuals are of runs, and Srijd is a real value of the search step size for one
recorded on each iteration in each run. These metrics are then dimension.
averaged on all the runs as illustrated in (15)—(18). Such Secondly, the contribution rate is constructed to investigate
averaging mechanism is also applied to the following metrics which sub-population contributes better solutions to the algo-
observed. In other words, the sub-population sizes of the local rithm during the search. In each run, all iterations in which

19980 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 5. The Comparison of the Average Execution Time (in CPU Seconds).

TABLE 6. The Results of the Scalability Test for 50-D Benchmark TABLE 8. The Results of the Scalability Test for 200-D Benchmark
Functions. Functions.

TABLE 7. The Results of the Scalability Test for 100-D Benchmark

Functions.
such a solution is recorded as one contribution for this sub-
population. At the end of the run, the contribution rate of each
sub-population is calculated based on its percentage of total
contributions in all effective iterations.
Lastly, the reset rate is also recorded. Such rate is defined
as the percentage of the iterations where the reset operation
is performed in all iterations.
Table 15 shows the statistical results of the metrics above.
From the table, we may observe that:
• The search step sizes of the local search group are much
smaller than those of the global search group. For exam-
ple, they are 0.0056 versus 0.6335 in F04. However,
the search step size of the local search group is even a bit
the algorithm achieves a better solution than the last itera- larger than that of the global search group in some cases,
tion are counted, which is called the effective iteration, and namely F02, F19 and F12. This is because the so-called
then the sub-population type of the individual who generates local search group individuals enlarge their search step

VOLUME 9, 2021 19981

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 9. The parameter sensitivity analysis of the results on Npop .

TABLE 10. The parameter sensitivity analysis of the results on w .

TABLE 11. The parameter sensitivity analysis of the results on γ .

sizes via the reset operation when the algorithm falls into dynamic. Once the algorithm gets close to the global
local optima in these cases; optimal, the local search group will expand for a bet-
• The sub-population sizes of the local search group, ter exploitation ability via the transformation and reset
global search group and random search group are operations. The examples include F04, F06 and F08.

19982 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 12. The parameter sensitivity analysis of the results on PR. functions) is the local search group. The opposite phe-
nomenon can be seen in F02 and F19. In F15, the contri-
bution rates of the global search group and local search
group are closing to each other, and also the random
search group contributes 9.73% of better solutions dur-
ing the search. The migrating group is able to contribute
good solution(s) to the search as well, such as in F02;
• During the search, the reset operation is performed in
40%+ of the iterations. As described in Section II-B6,
this operation is vital to the success of the AMPO.
Similar to the convergence test, the run with the median
fitness is selected to plot Fig. 13, where four functions
are illustrated. Generally, the search step size of the local
search group declines quickly with finding better solutions
(see Fig. 13a & Fig. 13b). On the other hand, as stated
above, the local search group individuals themselves may
LD .
TABLE 13. The parameter sensitivity analysis of the results on PLS also increase their search step sizes once no better solutions
are found (see 13d). Specifically, there is no search step
size of the local search group recorded in some periods
shown in Fig. 13c. This is because no local search group
individuals are available during that period. In fact, F15
involves a random noise, which produces different fitness
values in each evaluation, even with the same solution input.
It is difficult for the AMPO to converge into the optimal
solution, thereby downsizing the local search group while
upsizing the global search group. The contribution rates of
the local search group and the global search group are around
40% and 50%. Owing to this orchestration, the AMPO beats
all the other competitors in this problem, as presented in
Section III-C.
By the study, the AMPO has demonstrated its powerful
adaptivity in tackling various problems.
LS .
TABLE 14. The parameter sensitivity analysis of the results on PLS
VI. REAL-WORLD APPLICATION: PORTFOLIO
OPTIMIZATION
A. THE PROBLEM FORMULATION
Portfolio optimization is a significant problem in computa-
tional finance. Investors usually want to maximize returns and
minimize risks through portfolio diversification, i.e. allocat-
ing a fixed amount of capital into a collection of assets.
According to the mean-variance model formulated by [73],
the variance is regarded as a measure of risks that investors
expose to. The optimization problem is presented in (19) as
below.
X n
max E(R(X)) = xi ri
i=1
n X
X n
min V (R(X)) = xi xj σij
i=1 j=1
By contrast, the algorithm enlarges the size of the global X n
search group and random search group to improve the s.t. xi ∈ X = {xi ∈ R | xi = 1, 0 ≤ xi ≤ 1} (19)
exploration, as seen in F15, F19 and F20; i=1
• Different groups contribute better solutions to the where X is the proportion weight vector, E(R(X)) and
search in optimizing different problems. For exam- V (R(X)) are the expected return and variance of the whole
ple, the biggest contributor in F06 or F12 (uni-modal portfolio respectively, xi is the weight of the initial capital

VOLUME 9, 2021 19983

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

FIGURE 13. The Analysis of Search Step Sizes of Global and Local Search Groups.

TABLE 15. The Statistics of the Search Behavior of the AMPO on the Representative Functions.

that will be allocated in the ith asset, ri is the return of the ith included. As a matter of fact, short selling is allowed in
asset, n is the total number of assets, and σij stands for the real-world markets, where investors can sell stocks that they
covariance of the returns of the ith and jth assets. do not hold and must repurchase them later.
The portfolio based on the above model is restricted. When Portfolio optimization is fundamentally a multi-objective
the constrained condition 0 ≤ xi ≤ 1 is removed, it means optimization problem. One feasible approach is to transfer the
that the weights can be negative. Such a portfolio becomes problem into a single-objective optimization problem. There
unrestricted, and both longing and shorting on stocks are are two transfer methods: the first one is to select the return

19984 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

or risk as the objective function to be optimized while the TABLE 16. A Concise Description of the Portfolio Datasets.
rest are defined as constraints; an alternative method is to
establish only one objective function by assigning weights
to multiple objectives. In the first conventional approach,
the mean-variance efficient frontier of the portfolio can be
constructed for decision-making. This thesis seeks to opti-
mize the Sharpe Ratio (SR) of the portfolio [74], which
belongs to the above second transfer approach. The SR com-
bines the information from the mean and variance of an asset.
It is a risk-adjusted measure of the investment return. Until In the experiment, such values are calculated through all
now, the SR has become a critical tool in the modern financial recorded prices for each dataset. More specifically, for each
industry to evaluate the performance of various investment stock, the average of the weekly returns is computed as
portfolios. the expected return of the stock. The daily values of the
The definition of the SR is shown in (20). yields of the U.S. and the U.K. 5-year treasuries during
Pn the time interval that the dataset involved are averaged
E(R(X)) − Rf xi ri − Rf
SR = = Pn i=1Pn (20) to act as the risk-free rates for the U.S. and U.K. mar-
V (R(X)) i=1 j=1 xi xj σij kets, respectively. These risk-free rates (yearly) are 4.20%,
where SR is the SR of the portfolio, Rf is a risk-free rate, 5.68%, 2.42%, 2.94%, 2.40% and 2.43% for DowJones,
the representations of X, xi , σij and ri are same with those FF49Industries, NASDAQ100, FTSE100, SP500 and NAS-
in (19). DAQComp, respectively. By convention, the SR is converted
Since the AMPO and other comparative algorithms are into the annualized SR for better understanding and compar-
designated for handling minimization problems, the problem ison, as shown in (23).
should be changed to the minimization problem as given E(R(X)) − Rf /52 √
in (21). Annualized SR = × 52 (23)
V (R(X))
1 where 52 stands for the number of weeks in a year.
min fitness = (21)
SR Apart from longing on stocks (restricted portfolio),
s.t. SR = 10−10 if SR ≤ 0 we study a real-world scenario in which short-selling is
As investment returns probably are zero or even negative allowed (unrestricted portfolio), i.e. xi can be negative, which
in the financial market, a tiny number 10−10 is assigned to the enlarges the searching space of the problem.
SR for dealing with this case. In this part, the experiment involves three representative
To avoid handling the equality constraint, the problem is algorithms that perform well in the benchmark function test to
converted to the unconstrained form as shown below. verify their effectiveness in tackling these practical problems.
They are the WOA, LS-cnEPSin and MOFOA. Among them,
xi
xi0 = Pn (22) the excellent performance of WOA in portfolio optimization
i=1 |xi | is also reported by [47]. In addition, referring to [76], [77],
In order to perform more accurate and realistic simulations we select three other competitive algorithms that demon-
on the real-world markets, both restricted and unrestricted strate their excellent performance in optimizing the same
scenarios of the portfolio are studied in this work. problem in the literature, namely, Adaptive Particle Swarm
Optimization (APSO) [44], Fireworks Algorithm (FA) [45]
B. THE EXPERIMENTAL SETUP and Harmony Search Algorithm (HSA) [46]. Due to a large
In this part of the experiment, the real-world datasets recently dimension of some datasets, the MaxFEs is set to 5 × 105 for
released by [75] for portfolio optimization are used as our each algorithm to achieve a trade-off between computational
experimental targets. The datasets consist of six major stock cost and performance. Other user-controlled parameters of
indexes: Dow Jones Industrial Average (DowJones), Nasdaq all algorithms are the same as those used in the benchmark
100 (NASDAQ100), Financial Times Stock Exchange 100 function test.
(FTSE100), S&P 500(SP500), Fama and French 49 Indus-
try (FF49Industries) and NASDAQ Composite (NAS- C. SIMULATION RESULTS AND DISCUSSION
DAQComp). The datasets contain cleaned weekly return The simulation results of the portfolio optimization are shown
values that have been adjusted for dividends and stock splits. in Table 17 and Table 18, where the mean values and standard
Table 16 shows the details of such datasets, where the max- deviations of the optimized annualized SR in 30 runs are pre-
imum number of stocks is from 28 up to 1, 203 so that the sented. Also, the comparison of computational cost measured
scalability of the metaheuristic can be well tested in solving in CPU seconds is provided there.
this real-world problem as well. From the tables, we can observe that the AMPO gener-
Following the previous practice, the information of mean ates the best SR over all the competing algorithms in 11
and covariance is acquired from such weekly historical data. out of 12 portfolios. Notably, the AMPO algorithm delivers

VOLUME 9, 2021 19985

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

TABLE 17. The Comparative Results of the Portfolio Optimization between AMPO and Selected Algorithms (Restricted).

TABLE 18. The Comparative Results of the Portfolio Optimization between AMPO and Selected Algorithms (Unrestricted).

TABLE 19. The Statistical Test Results of the Portfolio Optimization in p-value.

much better values than the second-best values in optimiz- its performance on the small-scale restricted portfolios,
ing large-scale unrestricted portfolios on the SP500 (3.0476 i.e. DowJones, FF49Industries, NASDAQ100 and FTSE10,
versus 2.8831) and NASDAQComp (4.7972 versus 3.4231). is only next to the AMPO, but it gets much worse
The average ranks are respectively 1.08, 2.17, 3.50, results in constructing restricted portfolios for the large-scale
3.67, 5.00, 5.75 and 6.83 for the AMPO, LS-cnEpSin, SP500 and NASDAQComp datasets. On the other hand,
HSA, MOFOA, FA, WOA and APSO algorithms. For the LS-cnEpSin algorithm produces the contrary phe-
the second-best optimizer, namely LS-cnEpSin algorithm, nomenon for unrestricted portfolios. These findings suggest

19986 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

that the adaptivity of LS-cnEpSin is in question. Also, behavior of the AMPO are investigated. Lastly, the AMPO
the LS-cnEpSin becomes much computationally expensive is applied to solve the challenging portfolio optimization
when processing large-scale problems. This is associated problem with different numbers of assets from ranging 28
with the time-consuming calculation that is discussed in up to 1, 203. The result shows that the AMPO can achieve
Section III-F. Here the AMPO is 3, even 10 times faster than robust performance in constructing these portfolios over other
the LS-cnEpSin. algorithms.
Concerning the MOFOA optimizer, it produces low-quality The drawbacks of the AMPO are summarized as follows.
solutions for such small-scale problems but much better First, the number of user-controlled parameters is larger than
results for the restricted NASDAQComp portfolio, in which those of the existing optimization approaches like the GA
it gets the best value 1.3195 among all algorithms. and PSO. In addition, the algorithm can only handle uncon-
In regard to the WOA algorithm that achieves a satisfac- strained optimization problems at the moment.
tory performance in the benchmark function test, it fails to Concerning some future work, making the parameters
produce good results here, which implies it contains limited of the AMPO as self-adaptive is worth exploring. Besides,
adaptivity and effectiveness of dealing with these practical a more thorough investigation should be conducted on apply-
problems. ing the AMPO to solve constrained optimization problems.
Another inferential statistics test based on the Wilcoxon Furthermore, it is interesting to apply the AMPO to different
signed-rank test is performed in addition to the descriptive real-world applications for comparison with more recently
statistics. The alternative hypothesis is that AMPO outper- proposed metaheuristics like the HHO and SMA. Last but not
forms the competing algorithm in terms of solution qual- least, the AMPO has a great potential to be extended to tackle
ity when resolving these tasks. When the p-value is less multi-objective optimization problems.
than 0.05, the alternative hypothesis is accepted. As shown The source code of the AMPO is released at https://
in Table 19, almost all the p values are much smaller than github.com/rayzxli/AMPO.
0.05, even most are rounded to 0. Regarding the only two
exceptions, a few extra tests still cannot distinguish the per- REFERENCES
formance between the AMPO and the compared ones. Such [1] S. R. Young, D. C. Rose, T. P. Karnowski, S.-H. Lim, and R. M. Patton,
outputs have demonstrated that the AMPO is on the whole ‘‘Optimizing deep learning hyper-parameters through an evolutionary
algorithm,’’ in Proc. Workshop Mach. Learn. High-Perform. Comput. Env-
good at tackling these portfolio problems compared to all iron., Nov. 2015, pp. 1–5.
the compared algorithms. Also, the excellent scalability of [2] A. Tharwat, A. E. Hassanien, and B. E. Elnaghi, ‘‘A BA-based algorithm
the AMPO has been verified in coping with these practical for parameter optimization of support vector machine,’’ Pattern Recognit.
Lett., vol. 93, pp. 13–22, Jul. 2017.
problems. [3] F. Friedrichs and C. Igel, ‘‘Evolutionary tuning of multiple SVM parame-
ters,’’ Neurocomputing, vol. 64, pp. 107–117, Mar. 2005.
VII. CONCLUSION [4] Y. Nalçakan and T. Ensari, ‘‘Decision of neural networks hyperparameters
with a population-based algorithm,’’ in Proc. Int. Conf. Mach. Learn.,
This paper proposes a competitive metaheuristic optimiza- Optim., Data Sci. Volterra, Italy: Springer, Sep. 2018, pp. 276–281.
tion algorithm, namely the AMPO, for tackling challenging [5] V. K. Ojha, A. Abraham, and V. Snášel, ‘‘Metaheuristic design of feedfor-
continuous optimization problems. The algorithm is carefully ward neural networks: A review of two decades of research,’’ Eng. Appl.
Artif. Intell., vol. 60, pp. 97–116, Apr. 2017.
designed with different operations to diversify the search [6] K. T. Lwin, R. Qu, and B. L. MacCarthy, ‘‘Mean-VaR portfolio opti-
strategies so as to improve its optimization capability. The mization: A nonparametric approach,’’ Eur. J. Oper. Res., vol. 260, no. 2,
AMPO powered by the multi-population design has an excel- pp. 751–766, Jul. 2017.
[7] K. Metaxiotis and K. Liagkouras, ‘‘Multiobjective evolutionary algorithms
lent adaptivity when compared with the existing metaheuris- for portfolio management: A comprehensive literature review,’’ Expert
tics. The algorithm can dynamically allocate its search power Syst. Appl., vol. 39, no. 14, pp. 11685–11698, Oct. 2012.
to different sub-populations that conduct local search or [8] S. N. Skinner and H. Zare-Behtash, ‘‘State-of-the-art in aerodynamic
shape optimisation methods,’’ Appl. Soft Comput., vol. 62, pp. 933–962,
global search during the optimization process for different Jan. 2018.
problems. [9] O. Bozorg-Haddad, M. Solgi, and H. A. Loáiciga, Meta-Heuristic and Evo-
In this work, the AMPO is critically evaluated on a total of lutionary Algorithms for Engineering Optimization. Hoboken, NJ, USA:
Wiley, 2017.
28 well-known benchmark functions covering a broad range
[10] V. Pillac, M. Gendreau, C. Guéret, and A. L. Medaglia, ‘‘A review of
of characteristics of optimization problems. All the obtained dynamic vehicle routing problems,’’ Eur. J. Oper. Res., vol. 225, no. 1,
results are carefully compared and analyzed with those of pp. 1–11, 2013.
nine state-of-the-art optimization algorithms, including the [11] Z. Huang, X. Lu, and H. Duan, ‘‘A task operation model for resource
allocation optimization in business process management,’’ IEEE Trans.
recent SI approaches, the IEEE CEC winning algorithms Syst., Man, Cybern. A, Syst. Humans, vol. 42, no. 5, pp. 1256–1270,
and the latest developed multi-population and hybrid opti- Sep. 2012.
mization algorithms. The results demonstrate the outstanding [12] X.-S. Yang, Nature-Inspired Metaheuristic Algorithms. Frome, U.K.:
Luniver Press, 2008. [Online]. Available: https://dl.acm.org/doi/book/
performance of our proposed algorithm in terms of solution 10.5555/1628847
fitness, convergence rate, scalability, stability and computa- [13] X. Yu and M. Gen, Introduction to Evolutionary Algorithms. London, U.K.:
tional cost. In particular, the AMPO shows a unique poten- Springer-Verlag, 2010.
[14] P. A. Vikhar, ‘‘Evolutionary algorithms: A critical review and its future
tial for high-dimensional continuous optimization problems. prospects,’’ in Proc. Int. Conf. Global Trends Signal Process., Inf. Comput.
Additionally, the parameter sensitivity analysis and search Commun. (ICGTSPICC), Dec. 2016, pp. 261–265.

VOLUME 9, 2021 19987

 Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

[15] H. R. Maier, S. Razavi, Z. Kapelan, L. S. Matott, J. Kasprzyk, and [42] S. Li, H. Chen, M. Wang, A. A. Heidari, and S. Mirjalili, ‘‘Slime mould
B. A. Tolson, ‘‘Introductory overview: Optimization using evolutionary algorithm: A new method for stochastic optimization,’’ Future Gener.
algorithms and other metaheuristics,’’ Environ. Model. Softw., vol. 114, Comput. Syst., vol. 111, pp. 300–323, Oct. 2020.
pp. 195–213, Apr. 2019. [43] K. Sörensen, ‘‘Metaheuristics—The metaphor exposed,’’ Int. Trans. Oper.
[16] R. S. Parpinelli and H. S. Lopes, ‘‘New inspirations in swarm intelligence: Res., vol. 22, no. 1, pp. 3–18, 2015.
A survey,’’ Int. J. Bio-Inspired Comput., vol. 3, no. 1, pp. 1–16, Jan. 2011. [44] H. Zhu, Y. Wang, K. Wang, and Y. Chen, ‘‘Particle swarm optimization
[17] J. Kennedy, ‘‘Swarm intelligence,’’ in Handbook of Nature-Inspired (PSO) for the constrained portfolio optimization problem,’’ Expert Syst.
and Innovative Computing: Integrating Classical Models With Emerging Appl., vol. 38, no. 8, pp. 10161–10169, Aug. 2011.
Technologies, A. Y. Zomaya, Ed. Boston, MA, USA: Springer, 2006, [45] N. Bacanin and M. Tuba, ‘‘Fireworks algorithm applied to constrained
pp. 187–219, doi: 10.1007/0-387-27705-6_6. portfolio optimization problem,’’ in Proc. IEEE Congr. Evol. Comput.
[18] X.-S. Yang, Z. Cui, R. Xiao, A. H. Gandomi, and M. Karamanoglu, Swarm (CEC), May 2015, pp. 1242–1249.
Intelligence and Bio-Inspired Computation: Theory and Applications. [46] K. H. Lai, W. J. Siow, and A. A. M. N. Kaw, ‘‘Sharpe ratio-based portfolio
Oxford, U.K.: Elsevier, 2013. [Online]. Available: https://www.science optimization using harmony search algorithm,’’ Commun. Comput. Appl.
direct.com/book/9780124051638/swarm-intelligence-and-bio-inspired- Math., vol. 1, no. 1, Mar. 2019.
computation?via=ihub= [47] Q. H. Zhai, T. Ye, M. X. Huang, S. L. Feng, and H. Li, ‘‘Whale optimization
[19] P. J. M. Van Laarhoven and E. H. L. Aarts, Simulated Annealing: Theory algorithm for multiconstraint second-order stochastic dominance portfolio
and Applications. Dordrecht, The Netherlands: Springer, 1987. [Online]. optimization,’’ Comput. Intell. Neurosci., vol. 2020, pp. 1–19, Aug. 2020.
Available: https://www.springer.com/gp/book/9789027725134 [48] J. E. Beasley, ‘‘OR-library: Distributing test problems by electronic mail,’’
[20] N. H. Siddique and H. Adeli, Nature-Inspired Computing: Physics and J. Oper. Res. Soc., vol. 41, no. 11, pp. 1069–1072, Nov. 1990.
Chemistry-Based Algorithms. Boca Raton, FL, USA: CRC Press, 2017. [49] A. E. Eiben and C. A. Schippers, ‘‘On evolutionary exploration and
[21] A. Y. S. Lam and V. O. K. Li, ‘‘Chemical-reaction-inspired metaheuristic exploitation,’’ Fundamenta Informaticae, vol. 35, nos. 1–4, pp. 35–50,
for optimization,’’ IEEE Trans. Evol. Comput., vol. 14, no. 3, pp. 381–399, 1998.
Jun. 2010. [50] M. Črepinšek, S.-H. Liu, and M. Mernik, ‘‘Exploration and exploitation
[22] H. Abedinpourshotorban, S. Mariyam Shamsuddin, Z. Beheshti, and
in evolutionary algorithms: A survey,’’ ACM Comput. Surv., vol. 45, no. 3,
D. N. A. Jawawi, ‘‘Electromagnetic field optimization: A physics-inspired
pp. 1–33, Jun. 2013.
Metaheuristic optimization algorithm,’’ Swarm Evol. Comput., vol. 26,
[51] H. Ma, S. Shen, M. Yu, Z. Yang, M. Fei, and H. Zhou, ‘‘Multi-population
pp. 8–22, Feb. 2016.
techniques in nature inspired optimization algorithms: A comprehensive
[23] X.-S. Yang, ‘‘Harmony search as a metaheuristic algorithm,’’ in
survey,’’ Swarm Evol. Comput., vol. 44, pp. 365–387, Feb. 2019.
Music-Inspired Harmony Search Algorithm: Theory and Applications,
[52] X. Xia, L. Gui, and Z.-H. Zhan, ‘‘A multi-swarm particle swarm optimiza-
Z. W. Geem, Ed. Berlin, Germany: Springer, 2009, pp. 1–14.
[24] S. J. Mousavirad and H. Ebrahimpour-Komleh, ‘‘Human mental search: tion algorithm based on dynamical topology and purposeful detecting,’’
A new population-based Metaheuristic optimization algorithm,’’ Int. J. Appl. Soft Comput., vol. 67, pp. 126–140, Jun. 2018.
Speech Technol., vol. 47, no. 3, pp. 850–887, Oct. 2017. [53] G. Wu, R. Mallipeddi, P. N. Suganthan, R. Wang, and H. Chen, ‘‘Differ-
[25] J. Zhang, M. Xiao, L. Gao, and Q. Pan, ‘‘Queuing search algorithm: ential evolution with multi-population based ensemble of mutation strate-
A novel Metaheuristic algorithm for solving engineering optimization gies,’’ Inf. Sci., vol. 329, pp. 329–345, Feb. 2016.
problems,’’ Appl. Math. Model., vol. 63, pp. 464–490, Nov. 2018. [54] H. Chen, S. Li, A. Asghar Heidari, P. Wang, J. Li, Y. Yang, M. Wang, and
[26] J. H. Holland, Adaptation in Natural and Artificial Systems: An Intro- C. Huang, ‘‘Efficient multi-population outpost fruit fly-driven optimizers:
ductory Analysis With Applications to Biology, Control, and Artificial Framework and advances in support vector machines,’’ Expert Syst. Appl.,
Intelligence. Cambridge, MA, USA: MIT Press, 1992. vol. 142, Mar. 2020, Art. no. 112999.
[27] R. Storn and K. Price, ‘‘Differential evolution—A simple and efficient [55] Z. Li and V. Tam, ‘‘A novel meta-heuristic optimization algorithm inspired
heuristic for global optimization over continuous spaces,’’ J. Global by the spread of viruses,’’ 2020, arXiv:2006.06282. [Online]. Available:
Optim., vol. 11, no. 4, pp. 341–359, 1997. http://arxiv.org/abs/2006.06282
[28] H. G. Beyer and H. P. Schwefel, ‘‘Evolution strategies—A comprehensive [56] S. Das and P. N. Suganthan, ‘‘Differential evolution: A survey of the state-
introduction,’’ Natural Comput., vol. 1, no. 1, pp. 3–52, 2002. of-the-art,’’ IEEE Trans. Evol. Comput., vol. 15, no. 1, pp. 4–31, Feb. 2011.
[29] J. Kennedy and R. Eberhart, ‘‘Particle swarm optimization,’’ in Proc. IEEE [57] A. W. Mohamed, H. Z. Sabry, and T. Abd-Elaziz, ‘‘Real parameter opti-
ICNN, vol. 4. Nov./Dec. 1995, pp. 1942–1948. mization by an effective differential evolution algorithm,’’ Egyptian Infor-
[30] M. Dorigo, M. Birattari, and T. Stutzle, ‘‘Ant colony optimization,’’ IEEE mat. J., vol. 14, no. 1, pp. 37–53, Mar. 2013.
Comput. Intell. Mag., vol. 1, no. 4, pp. 28–39, Nov. 2006. [58] Y. Zhuang, K. Gao, X. Miu, L. Han, and X. Gong, ‘‘Infrared and visual
[31] D. Karaboga, B. Gorkemli, C. Ozturk, and N. Karaboga, ‘‘A comprehen- image registration based on mutual information with a combined particle
sive survey: Artificial bee colony (ABC) algorithm and applications,’’ Artif. swarm optimization—Powell search algorithm,’’ Optik, vol. 127, no. 1,
Intell. Rev., vol. 42, no. 1, pp. 21–57, Jun. 2014. pp. 188–191, Jan. 2016.
[32] S. Mirjalili, S. M. Mirjalili, and A. Lewis, ‘‘Grey wolf optimizer,’’ Adv. [59] Y. del Valle, G. K. Venayagamoorthy, S. Mohagheghi, J.-C. Hernandez,
Eng. Softw., vol. 69, pp. 46–61, Mar. 2014. and R. G. Harley, ‘‘Particle swarm optimization: Basic concepts, variants
[33] J. J. Q. Yu and V. O. K. Li, ‘‘A social spider algorithm for global optimiza- and applications in power systems,’’ IEEE Trans. Evol. Comput., vol. 12,
tion,’’ Appl. Soft Comput., vol. 30, pp. 614–627, May 2015. no. 2, pp. 171–195, Apr. 2008.
[34] S. Mirjalili and A. Lewis, ‘‘The whale optimization algorithm,’’ Adv. Eng.
[60] T. T. Ngo, A. Sadollah, and J. H. Kim, ‘‘A cooperative particle swarm opti-
Softw., vol. 95, pp. 51–67, May 2016.
mizer with stochastic movements for computationally expensive numerical
[35] S. Saremi, S. Mirjalili, and A. Lewis, ‘‘Grasshopper optimisation algo-
optimization problems,’’ J. Comput. Sci., vol. 13, pp. 68–82, Mar. 2016.
rithm: Theory and application,’’ Adv. Eng. Softw., vol. 105, pp. 30–47,
[61] S. Mostafa Bozorgi and S. Yazdani, ‘‘IWOA: An improved whale opti-
Mar. 2017.
[36] G.-G. Wang, S. Deb, and L. dos S. Coelho, ‘‘Earthworm optimisation mization algorithm for optimization problems,’’ J. Comput. Design Eng.,
algorithm: A bio-inspired metaheuristic algorithm for global optimisation vol. 6, no. 3, pp. 243–259, Jul. 2019.
problems,’’ Int. J. Bio-Inspired Comput., vol. 12, no. 1, pp. 1–22, Jan. 2018. [62] M. N. Ab Wahab, S. Nefti-Meziani, and A. Atyabi, ‘‘A comprehensive
[37] G.-G. Wang, ‘‘Moth search algorithm: A bio-inspired Metaheuristic algo- review of swarm optimization algorithms,’’ PLoS ONE, vol. 10, no. 5,
rithm for global optimization problems,’’ Memetic Comput., vol. 10, no. 2, May 2015, Art. no. e0122827.
pp. 151–164, Jun. 2018. [63] J. J. Liang, B. Y. Qu, and P. N. Suganthan, ‘‘Problem definitions and eval-
[38] R. Salgotra and U. Singh, ‘‘The naked mole-rat algorithm,’’ Neural Com- uation criteria for the CEC 2014 special session and competition on single
put. Appl., vol. 31, no. 12, pp. 8837–8857, Dec. 2019. objective real-parameter numerical optimization,’’ Comput. Intell. Lab.,
[39] G. G. Wang, S. Deb, and Z. Cui, ‘‘Monarch butterfly optimization,’’ Neural Zhengzhou Univ., Zhengzhou, China, Nanyang Technol. Univ., Singapore,
Comput. Appl., vol. 31, no. 7, pp. 1995–2014, 2019. Tech. Rep., 2013, vol. 635, p. 490.
[40] A. A. Heidari, S. Mirjalili, H. Faris, I. Aljarah, M. Mafarja, and H. Chen, [64] M. Jamil and X.-S. Yang, ‘‘A literature survey of benchmark functions for
‘‘Harris hawks optimization: Algorithm and applications,’’ Future Gener. global optimization problems,’’ Int. J. Math. Model. Numer. Optim., vol. 4,
Comput. Syst., vol. 97, pp. 849–872, Aug. 2019. no. 2, p. 44, 2013.
[41] S. Shadravan, H. R. Naji, and V. K. Bardsiri, ‘‘The sailfish optimizer: [65] J. Derrac, S. García, D. Molina, and F. Herrera, ‘‘A practical tutorial on
A novel nature-inspired Metaheuristic algorithm for solving constrained the use of nonparametric statistical tests as a methodology for comparing
engineering optimization problems,’’ Eng. Appl. Artif. Intell., vol. 80, evolutionary and swarm intelligence algorithms,’’ Swarm Evol. Comput.,
pp. 20–34, Apr. 2019. vol. 1, no. 1, pp. 3–18, Mar. 2011.

19988 VOLUME 9, 2021

Z. Li et al.: Adaptive Multi-Population Optimization Algorithm for Global Continuous Optimization

[66] D. Molina, A. LaTorre, and F. Herrera, ‘‘An insight into bio-inspired and VINCENT TAM (Senior Member, IEEE) received
evolutionary algorithms for global optimization: Review, analysis, and the Ph.D. degree in computer science from the
lessons learnt over a decade of competitions,’’ Cognit. Comput., vol. 10, University of Melbourne, in 1998. He is cur-
no. 4, pp. 517–544, Aug. 2018. rently a Principal Lecturer and also Honorary
[67] R. Tanabe and A. Fukunaga, ‘‘Success-history based parameter adaptation Associate Professor with the Department of Elec-
for differential evolution,’’ in Proc. IEEE Congr. Evol. Comput., Jun. 2013, trical and Electronic Engineering, The Univer-
pp. 71–78.
sity of Hong Kong. He has over 150 interna-
[68] R. Tanabe and A. S. Fukunaga, ‘‘Improving the search performance of
tionally refereed publications, including 10 book
SHADE using linear population size reduction,’’ in Proc. IEEE Congr.
Evol. Comput. (CEC), Jul. 2014, pp. 1658–1665. chapters. His main research interests include big
[69] N. H. Awad, M. Z. Ali, and P. N. Suganthan, ‘‘Ensemble sinusoidal data analytics, computational intelligence, cloud
differential covariance matrix adaptation with Euclidean neighborhood computing, machine learning, and information visualization. Besides,
for solving CEC2017 benchmark problems,’’ in Proc. IEEE Congr. Evol. he served as the Chairman of the IEEE (HK) Computational Intelligence
Comput. (CEC), Jun. 2017, pp. 372–379. Chapter (2014–2017).
[70] D. Molina, A. LaTorre, and F. Herrera, ‘‘SHADE with iterative local search
for large-scale global optimization,’’ in Proc. IEEE Congr. Evol. Comput.
(CEC), Jul. 2018, pp. 1–8.
[71] I. B. Aydilek, ‘‘A hybrid firefly and particle swarm optimization algorithm
for computationally expensive numerical problems,’’ Appl. Soft Comput.,
vol. 66, pp. 232–249, May 2018.
[72] Z. Li, V. Tam, and L. K. Yeung, ‘‘A study on parameter sensitivity analysis
of the virus spread optimization,’’ in Proc. IEEE Symp. Ser. Comput. Intell.
(SSCI), 2020, pp. 1535–1542.
[73] A. Stuart and H. M. Markowitz, ‘‘Portfolio selection: Efficient diversifica-
tion of investments,’’ OR, vol. 10, no. 4, p. 253, Dec. 1959.
[74] W. F. Sharpe, ‘‘Mutual fund performance,’’ J. Bus., vol. 39, no. 1,
pp. 119–138, Jan. 1966.
[75] R. Bruni, F. Cesarone, A. Scozzari, and F. Tardella, ‘‘Real-world datasets
for portfolio selection and solutions of some stochastic dominance portfo-
lio models,’’ Data Brief, vol. 8, pp. 858–862, Sep. 2016.
[76] J. Doering, R. Kizys, A. A. Juan, À. Fitó, and O. Polat, ‘‘Metaheuristics for
rich portfolio optimisation and risk management: Current state and future
trends,’’ Oper. Res. Perspect., vol. 6, 2019, Art. no. 100121.
[77] D. A. Milhomem and M. J. P. Dantas, ‘‘Analysis of new approaches used
in portfolio optimization: A systematic literature review,’’ Production,
vol. 30, Jul. 2020.

ZHIXI LI (Graduate Student Member, IEEE) is

currently pursuing the Ph.D. degree with the
Department of Electrical and Electronic Engineer- LAWRENCE K. YEUNG (Senior Member, IEEE)
ing, The University of Hong Kong. was born in 1969. He received the B.Eng.
He was a Research Staff Member at the and Ph.D. degrees in information engineering
Hong Kong Applied Science and Technology from The Chinese University of Hong Kong,
Research Institute. He has solid experience in in 1992 and 1995, respectively. He joined the
academic research as well as industrial applica- Department of Electrical and Electronic Engineer-
tion development. His research interests include ing, The University of Hong Kong, in July 2000,
machine learning, computational finance, and where he is currently a Professor. His research
computational intelligence. He is a Graduate Student Member of the Com- interests include next-generation internet, packet
putational and Financial Econometrics Network with several international switch/router design, all-optical networks, and
refereed publications and academic awards from universities and academic wireless data networks.
organizations.

VOLUME 9, 2021 19989