73

CHAPTER 5
PARTICLE SWARM OPTIMIZATION FOR MINING
ASSOCIATION RULES

5.1

GENERAL
This chapter focuses on ARM using PSO methodology. Modifications

were made in the PSO velocity update for avoiding premature convergence. In
this regard, chaotic behavior and dynamic neighborhood selection were
introduced in the basic PSO method. The methodology and experimental
results of the new PSO variants are also presented in this chapter. The
avoidance of premature convergence is attempted by adjusting the control
parameters: inertia weight and acceleration coefficients. Two different adaption
mechanisms namely, data-independent and data-dependent mechanisms are
also dealt in this chapter for mining ARs.
5.2

PSO FOR ARM

GA when applied for ARM generates ARs with better PA. But, the time

taken for mining the rules is more, and the reproduction operator at times
produced offspring crowding towards local optima.
To overcome the above demerits, PSO was taken up for mining ARs.
PSO shares many similarities with evolutionary computation techniques such
as GA. However unlike GA, PSO has no evolution operators such as crossover
and mutation and also possess the advantage of storing the previous history of
particles. PSO has proved to be competitive with GA in several tasks, mainly in
optimization areas.
The initial ideas on particle swarms of Kennedy (a social psychologist)
and Eberhart (an electrical engineer) were essentially aimed at producing
computational intelligence by exploiting simple analogues of social interaction,
rather than purely individual cognitive abilities. The first simulation (Kennedy
and Eberhart 1995) was influenced by Heppner and Grenanders work

74

(Heppner and Grenander 1990) and involved analogues of bird flocks searching
for corn. Soon this soon was developed into a powerful optimization method
PSO (Kennedy and Eberhart 1995; Eberhart and Kennedy 1995; Eberhart et al.
1996). The concept of ARM using PSO is discussed in the following section.
5.2.1 Methodology
PSO is initialized with a group of random particles (solutions) and then
searches for optimum value by updating particles in successive generations. In
each iteration, all the particles are updated by following two "best" values:
pBest and gBest. The movement of the particle in PSO towards the global
optima (target) is shown in figure 5.1 below.

Generation N

.
Figure 5.1 Particles Movement towards the target
PSO is a new evolutionary algorithm, which simulates the coordinated
motion in flocks of birds. Sousa, Silva, and Neves (2004), proposed the use of
PSO for data mining. PSO can achieve the rule discovery process. The rule
representation in PSO uses the Michigan approach. PSO needs fewer particles
than GA to obtain the same results.

75

The flowchart of the PSO algorithm for mining ARs is shown in

Figure 5.2. Each record in the dataset with its attributes is represented as a
particle in PSO. During iteration process, the velocity and position of all
particles are updated based on velocity and position update equations (3.1) and
(3.2), respectively.
Start
Initialize particles with
random position
For each particles
position (p) evaluate
fitness

If then
pbest= p

fitness(p) > fitness

(pbest)

Set best of pBests as

Update particles
velocity and position

Terminate?

Stop: Giving gBest,

optimal solution

Figure 5.2 Flowchart of PSO for ARM

Binary encoding is applied for representing each particle. The particles
in this population are called initial particles. Initially the velocity and position
of all particles are randomly set, within predefined range.
To reduce the possibility of particles flying out of the problem space,
Eberhart et al. (1996) put forward a clamping scheme that limited the speed of
each particle to a range [ -vmax, vmax] with vmax usually being somewhere

76

between 0.1 and 1.0 times the maximum position of the particle. In this study
vmax is set as the maximum value of the particles (xi).
Whilst experimenting with the standard algorithm, Shi and Eberhart
(1998), noted that without the velocity memory

v old
i ,

the first part of Eqn.

(3.1), the swarm would simply contract to the global best solution found within
the initial swarm boundary (providing a local search). Conversely, with the
velocity memory, the swarm will behave in the opposite sense, expanding to
provide a global search. In order to achieve the balance between exploration
and exploitation, a modified PSO incorporating an inertia weight ( , was
introduced thus:
old
v new
i =v i +c 1 rand () ( pBestxi ) + c2 rand ()( gBest xi )

(5.1)

The initial experimentation (Carlisle and Dozier 2000; Trelea 2003)

suggested that a value between 0.8 and 1.2 provided good results, although in
later work (Eberhart and Shi 2000) they indicate that the value is typically set
to 0.9 (reducing the stepwise movement of each particle, allowing greater
initial exploration) reducing linearly to 0.4 (speeding convergence to the global
optimum), during an optimization run.
To study the impact of inertia weight on PSO for ARs mining the inertia
weight is varied and the accuracy achieved by ARM was analyzed.
5.2.2 Experimental Results and Discussion
The objective of this study is to analyze the performance of PSO for
mining ARs. PSO models without and with inertia weights were applied for
ARM. ARs are mined from the five datasets described in section 3.5. The

77

parameters of the PSO model without inertia weight for ARM is given in Table
5.1.
Table 5.1 Initial Parameter Setting of PSO for ARM
Parameter Name
Population Size

Value
Lenses

: 20

Habermans Survival : 300

Car Evaluation

: 1000

Postoperative Care

: 50

Zoo

: 100

No. of Generations

100

C1, C 2

vmax

Maximum (xi)

Each methodology was run 10 times on the dataset chosen. The PA

achieved for the five datasets is presented in Table 5.2. The same is compared
with the simple GA methodology described in Chapter 4.
Table 5.2 Predictive Accuracy Comparison of PSO and GA
for ARM

Dataset Name
Lenses
Habermans Survival
Car evaluation
Post Operative Patient
Zoo

GA
85
87
81
74
81

PSO
92.8
94.4
91.6
83.33
95.45

78

The PA obtained through PSO methodology

is

better

than

the

performance of simple GA for all the five datasets. The simplicity of PSO with
minimum function evaluations generates better ARs when compared to GA.

The execution time (CPU) is the time taken for mining ARs by PSO
methodology until the completion of specified number of generations. The
execution time results obtained for GA and PSO are presented in Figure 5.3.
140
120
100
80
60
40
20

Execution Time (s)

ti
o

va

PS O

lu

H
a

va

rv
i

Z
's

rm
a

ti
v
ra
e
p
O
st

GA

ti
e

Le

Pa

se

Datasets

Figure 5.3 Execution Time comparison of PSO and GA for ARM

In terms of computational effectiveness PSO is found to be marginally
faster than GA. PSO shares many similar properties with GAs. Both methods
begin with a group of randomly initialized population, evaluate their population
based on fitness function. However, the main difference between PSO and GA
is that PSO does not have the genetic operators as crossover and mutation. In
PSO, only the best particle passes information to others and hence, the
computational capability of PSO is marginally better than GA.
As the efficiency of PSO in terms of PA and execution time have been
found to be better than GA, the qualitative measures defined in chapter 3 are
determined for the five datasets. The results are given in Table 5.3.

79

Table 5.3 Qualitative Measures of ARM with PSO

Lenses
Laplace
Lift
Conviction
Leverage

0.52
2
Infinity
0.0625

Habermans
Car
Survival
Evaluation
0.5
0.5
2.08
1.05
Infinity
Infinity
0.006
0.0015

Postoperativ
e Patient
0.5
1.279
Infinity
0.007

Zoo
0.5
6.314
Infinity
0.016

The Laplace measure, away from the value 1, signifies that the
antecedent values are dependent on the consequent and hence the rules
generated are of importance. The conviction measure which is infinity for all
datasets show that the rules generated are interesting. The Leverage measure
being far away from 1 again insists on the interestingness of the rules
generated. The measure of Lift for all the datasets is greater than1 signifies that
the dependency of the consequent on the antecedent. Thus the rules generated
by PSO are of importance based on the Lift measure.
In the velocity update function (Eqn 3.1) the velocity of the particle in
previous generation is added as such, to the current velocity. This makes an
impact on the previous particles movement on the new velocity. Thus, the
exploration ability of the particle gets restricted. To reduce the impact of earlier
velocity on current velocity, the inertia weight parameter is added as given in
Equation (6.1). The inertia weight impart only portion of earlier velocity into
new one. The inertia weight is varied and the PA achieved by the generated
rules from the five datasets is given in Figure 5.4 below.

80

100

90

Lens es

Haberma's S urvival

Car Evaluation

Po-opert Care

80

Predictive Accuracy %

70
Z oo

w
=
0.

w
=
0.

w
=
0.

w
=
0.

w
=
0.

w
=
0.

w
=

0.

w
=
0.

60

Inertia Weight

Figure 5.4 Impact of Inertia Weight on Predictive

Accuracy for ARM
From the above figure it is inferred that for the inertia weight ( in
the range [0.4, 0.9], the PA achieved is optimum. When inertia weight is
between 0.2 and 0.3 the PA achieved is inconsistent. The PA achieved by the
weighted PSO for ARM is plotted in comparison with PSO without inertia
weight, as shown in Figure 5.5.
The PA of weighted PSO (WPSO) has enhanced, for all the five datasets
in comparison with PSO. The accuracy enhancement achieved is varying from
1.27% to 14% for the datasets chosen. An increase in PA of 14% is achieved for
the postoperative care dataset. Marginal enhance of 1.27% of PA is noted for
the zoo dataset. A better balance between exploration and exploitation is
obtained by the addition inertia weight and hence, better performance by the
WPSO.

81

100
90
80
70
60
50
40
30
20
Predictive Accurcay (%)
10
PS O
0

Po
s

O
pe

ra

tiv
e

Pa
t

Le
n

se

ie
n

WPS O

Datasets

Figure 5.5 Comparison of PSO with and without

Inertia Weight for ARM
5.3

CHAOTIC PSO FOR ARM

The canonical PSO has a tendency to get struck at local optima and

thereby leading to premature convergence when applied for solving practical

problems. To improve the global searching capability and escape from the local
optima, chaos is introduced in PSO (Alatas et al. 2007). Chaos is a
deterministic dynamic system, very sensitive and dependent on its initial
conditions and parameters. In this section chaotic behavior introduced in PSO
is presented to maintain the diversity of population and to enhance the
performance of ARM process.
5.3.1 Chaotism in PSO
To improve the global searching capability by escaping from the local
solutions, sequences generated from different Zaslavskii Map (Zhang and
Huang 2004) substitute random numbers for different parameters of PSO,
where it is necessary to make a random-based choice. Coupling such behavior
with PSO and complex dynamics, can improve the quality of results in some
optimization problems and also that chaos may be a desired process. It has been

82

empirically shown that introducing chaotic maps with ergodicity, irregularity

and the stochastic property in PSO, the global convergence has been improved
(Alatas et al., 2007).
The common method of generating chaotic behavior is based on
Zaslavskii map (Zhang and Huang, 2004). This representation of map involves
many variables. Setting right values for all the variables involved, increases the
complexity of the system. And erroneous values might bring down the accuracy
of the system involved. Logistic map and tent map also have the most
frequently used chaotic behavior. The drawback of these maps is that the range
of values generated by both maps after some iteration becomes fixed to a
particular range. To overcome this defect the tent map undisturbed by the
logistic map (Kong et al., 2010) is introduced as the chaotic behavior. A new
chaotic map model is proposed with the following equation, based on the tent
and logistic maps.
uk +1=4u k ( 1uk ) 0 u k 1

(5.2)
1/ 1.001( 2v k + 0.001u k+1)0 v k 0.5
1v
( k)+ 0.001uk +1
2 0.5< v k 1

1.001
1/
v k+1 =

The initial value of u0 and v0 are set to 0.1. The slight tuning of initial
values of u0 and v0 creates wide range of values with good distribution.
Therefore, the chaotic operator chaotic_operator(k) = vk is designed to generate
different chaotic operators by tuning u0 and v0. The value u0 is set to two
different values for generating the chaotic operators 1 and 2.

83

The velocity updation equation based on chaotic PSO is given in

Equation 6.3.
old
v new
i =v i +c 1C h aoti c operator1 ( lbest x i ) +c 2 ( 5.3 )C h aotic operator2 ( gbest x i )

In the above equation, the random number generator is replaced with

chaotic_operator1 and chaotic_operator2.

Chaotic behavior is replaced by

random number to create a new population with diversity. When diversity

exists, the exploitation capability of PSO increases. Thus, the global search is
performed effectively and the performance also increases.
5.3.2 Experimental Results and Discussion
The chaotic PSO (CPSO) was tested on the bench mark datasets
discussed earlier in chapter 3, for performance analysis. The number of
generations was set to 100. The acceleration coefficients c1 and c2 were both set
to 2. The initial velocity was set as 0 for all the particles and the value for V max
was set as 1. Each methodology was run 10 times on the chosen dataset.
The PA achieved by CPSO for ARM from the five datasets is plotted in
Figure 5.6. The PA achieved by PSO is compared with the obtained results.

84

105
100
95
90
85
Predictive Accuracy (%)

80

PS O

CPS O
o
o
Z

C
rt
e

-o

va
E

r
a
C

ti
o
a
lu

u
S
s

re

l
va

rv
i

se
n
Le
n
H
a

rm
a

75

Datasets

Figure 5.6 Comparison of Predictive Accuracy

of CPSO and PSO for ARM
Chaotic PSO generated ARs with enhanced PA, in comparison to
PSO for all the five datasets. The convergence rate (generation at which
maximum PA achieved) of the CPSO for mining ARs from the five datasets
chosen for analysis is presented in the Figure 5.7.

85

100
90
80
70
60
50
40
Genetration No.

PSO

30

CPSO

20
10

o
o

a
C
rt

re

n
o
ti
a
P

-o

lu
r
a

H
a

rm
a

Ev
a

Le

se

rv
iv
a

Datasets

Figure 5.7 Convergence rate of CPSO for ARM

86

It could be noted form the Figure 5.7 that the PSO methodology
generated ARs with maximum accuracy at the earlier stages of iteration (10-40)
for the five datasets. To avoid premature convergence chaotic behavior was
introduced in PSO. This chaotic behavior resulted in diversification of
particles, thereby evading the premature convergence. Thus the convergence
has shifted to later iterations in CPSO (30-70).
The chaotic operators could be changed by altering the initial values in
chaotic operator function. The balance between exploration and exploitation is
maintained by CPSO methodology.

5.4

DYNAMIC NEIGHBORHOOD SELECTION IN PSO FOR ARM

The problem of not reaching the optimal solution and deviation from

optimal solution space are addressed via gBest and pBest values respectively in
PSO. The global best propagates information at the fastest, in the population
dealing with exploration, while, the local best using a ring structure speeds up
the system balancing the exploration.
In published literature efforts have been directd to enhance the
performance of PSO. In Gregarious PSO (Pasupuleti and Battiti, 2006) the
social knowledge of the particle is used for discovery in the search space. If
particles are trapped in the local optimum, a stochastic velocity vector thereby
self sets the parameters. In Dynamic neighborhood PSO (Lu and Chen, 2008)
instead of using the current gBest, another parameter Neighborhood Best
(Nbest) is utilized. This term is the best particle among the current particles
neighbors in a specified neighborhood.
Fixing up the best position for particles after velocity updation by using
Euclidean distance helps in generating the best particles (Kuo et al. 2011). The
problem of getting struck at local optimum and hence premature convergence is
overcome by self adaptive PSO (Lu et al. 2010), where, the diversity of

87

population is maintained. This copes up with the deception of multiple local

optima and reduces computational complexity. An adaptive chaotic PSO
(Cehng at al. 2011) enhances the global searching capability and local
searching capability by introducing chaotic operators based on Logistic map
and tent map. In addition an adaptive search strategy which optimizes
continuous parameters is employed.
To avoid premature convergence and enhance the accuracy the
Neighborhood selection in PSO (NPSO) was proposed replacing the particle
best concept by local best.
5.4.1 Dynamic Neighborhood Selection in PSO
The balance between exploration and exploitation in PSO is the main
issue when applied for solving complex problems. The velocity update
equation plays a major role in enhancing the performance of the PSO. To
maintain the diversity of the particles and enhance the performance of the PSO,
the concept of adapting the local best particle among the neighbors is proposed
for mining ARs. The proposed work is to review the PSO for mining ARs with
dynamic neighborhood selection.
From the study carried out on the five datasets, PSO proves as effective
as GA in mining ARs. In terms of computational efficiency, PSO is marginally
faster than GA. The pBest and gBest values tend to pass the information
between populations more effectively than the reproduction operators in GA.
The neighborhood best called local best (lBest) selection is as follows;
Calculate the distance of the current particle from other particles by
equation (5.4).
x i= ( x ix gbest )

(5.4)

88

Find the nearest m particles as the neighbor of the current particle

based on distance calculated
Choose the local optimum lBest among the neighborhood in terms
of fitness values
The flowchart of the proposed algorithm is given in Figure 5.8. The
number of neighborhood particles m is set to 2. Velocity and position updation
of particles are based on equation (3.1) and (3.2). The velocity updation is
restricted to maximum velocity Vmax, set to the maximum of x i.
termination condition is set as fixed number of generations.

Star
t
K =1 ,Initialize
xi(k), vi(k)
Compute f(xi(k))
Reorder the
particles
Generate
neighborhoods
I =1
Determine best

i = i +1

particles in the
neighborhood of i
Compute xi(K+1)
Compute (f(xi(K+1))
Update previous
best if necessary
Y
N

i
N

k
K

Sto
p

K = K+1
y

The

89

Figure 5.8 Flowchart for Association Rule Mining with Dynamic

Neighborhood Selection in PSO

The velocity update function of NPSO is as defined in equation (5.5)

and the position update function is adopted using Eqn. (3.2).
old
v new
id =v id +c 1 rand () ( lBestx id ) + c2 rand ()(gBestx id )

(5.5)

The interestingness of discovered ARs is an important and active area

within data mining research. The measure of interestingness varies from
application to application and from expert to expert. Each interestingness
measure produces different results, and experts have different opinions of what
constitutes a good rule. The interestingness measure for a rule is taken from
relative confidence and is as follows:
interestingness ( k ) =

support ( x y )support ( x ) support ( y )

support ( x ) (1support ( y ))

(5.6)
Where, k is the rule; x the antecedent part of the rule and y the consequent part
of the rule k.
5.4.2 Experimental Results and Discussion
The effectiveness of the proposed dynamic neighborhood selection in
PSO methodology for ARM was tested on the five datasets selected for
analysis. The parameter setting of the proposed NPSO methodology for ARM
is listed in Table 5.4.

Table 5.4 Default Parameter Setting of NPSO for ARM

Parameter Name
Parameter Value
Population Size
Lenses
: 20

90

Initial Velocity
c1
c2
Vmax
No of Iterations
No. of Runs

Habermans Survival
Car Evaluation
Postoperative Care
Zoo
0
2
2
1
100
10

: 300
: 1000
: 50
: 100

The maximum accuracy achieved from repeated runs is recorded as the

PA for each case. The interestingness is calculated from the corresponding run.
The PA achieved was compared with PSO for the same datasets and the highest
PA achieved for multiple runs is plotted in Figure 5.9.

100

95

90

85

Predictive Accuracy(%)
PSO

NPSO

80

o
o
Z

a
C

-o

rt

a
lu

re

n
ti
o

va
E
r
a

rm

va

Le

rv
i

se

75

Datasets

Figure 5.9 Predictive Accuracy Comparison for Dynamic

Neighborhood selection in PSO

91

The PA achieved for all the five datasets chosen was better compared to
PSO methodology. A maximum enhancement of 11.18% is achieved for the
postoperative care dataset.
The interestingness or relative confidence measures of the mined rule
are shown in Table 5.5. The ARs mined with dynamic neighborhood selection
PSO is with good interestingness measure, indicating the importance of the
mined rules.
Table 5.5 Measure of Interestingness for Dynamic Neighborhood
selection PSO
Dataset

Lens

Interestingness
Value

Car

Habermans Postoperative

Evaluation

Survival

Patient

0.73

0.8

0.78

0.82

Zoo
0.76

Experts using evolutionary algorithms observe that the time complexity

of PSO is less, when compared to GA. Premature convergence may also result
in reduced execution time. The scope of this work is to avoid premature
convergence. The concept of local best based on neighborhood particles,
instead of the individual best particles, focuses on this issue.
The convergence rate of NPSO for mining ARs is given in Table 5.6.
Table 5.6 Comparison of Convergence rate of
NPSO with PSO for ARM
Dataset

Lens

PSO
NPSO
(Generation (Generation
No.)
No.)
10

60

92

Car Evaluation

20

60

Habermans
Survival

10

60

Postoperative
Patient

50

60

Zoo

20

60

The PSO methodology produces maximum results at the earlier iteration

stages itself (10-50), whereas, the neighborhood selection in PSO extends the
convergence rate to iterations more than 50, thus, avoiding premature
convergence.
The selection of local best particles based on neighbors (lBest),
replacing the particles own best (pBest), enhances the accuracy of the rules
mined. The concept of local best (lBest) based on neighborhood selection in
fitness space instead of other measures helps in maintaining the diversity of
local points optimally, thus striking a balance between premature convergence
and diversification of particles in problem space.
5.5 MOTIVATION FOR ADAPTIVE TECHNIQUES
While solving problems with PSO, its properties affect the performance.
The properties of PSO depend on the parameter setting and hence users need to
find the opt value for the parameters to optimize the performance. The
interactions between the parameters have a complex behavior and so each
parameter value will have a different effect depending on the value set for
others. Without prior knowledge of the problem, parameter setting is difficult
and time consuming. The two major ways of parameter setting are through
parameter tuning and parameter control (Eiben and Smith, 2003). Parameter
tuning is the commonly practiced approach that amounts to finding appropriate
values for the parameters, before running the algorithm. Parameter control
steadily modifies the control parameter values during the run. This could be

93

achieved either through one of: deterministic, adaptive or self-adaptive

techniques (Eiben et al. 2007).
Deterministic parameter control is effected through a deterministic rule
that modifies the strategy parameter without any feedback. This method
becomes unreliable for most problems because the parameter adjustments must
rely on the status of the problem at current time. In self-adaptive approach, the
parameters to be controlled are encoded into the candidate solution which may
result in deadlock. Good solution depends on finding the good setting of
parameters and obtaining the good setting of parameters depends on finding the
good solution. Moreover, extra bits are required to store these strategy
parameters, so the dimensionality of the search space increases. Thus, the
corresponding search space becomes larger and hence the complexity of the
problem increases. In this context, adaptive method is the better solution.
The parameters of PSO and their role in optimization process are
presented in Table 5.7.
Table 5.7 Parameters of PSO and their roles
Parameter
Inertia weight

Parameter Role
( Controls the impact of the velocity
history into the new velocity

Acceleration

Maintains the diversity of swarm

Coefficient c1
Acceleration

Convergence towards the global

Coefficient c2

optima

The research attempts on dynamically adapting the parameter of PSO to

enhance the efficiency of ARM based on: data independent and data dependent
techniques as discussed in the following sections.
5.6

DATA INEPENDENT ADAPTION IN PSO

94

The velocity update equation plays a major role in enhancing the

performance of the PSO. However, similar to other evolutionary computation
algorithms, the PSO is also a population-based iterative algorithm. Hence, it
can easily get trapped in the local optima when solving complex multimodal
problems (Liang et al., 2006). These weaknesses have restricted wider
applications of the PSO (Engelbrecht 2006).
To balance the global search and local search, inertia weight () was
introduced. It can be a positive constant or even a positive linear or nonlinear
function of time (Shi and Eberhart 2002). Inertia weight plays a key role in the
process of providing balance between exploration and exploitation process. The
inertia weight determines the contribution rate of a particles previous velocity
to its velocity at the current time step. Eberhart and Shi (2001), proposed a
random inertia weight strategy and experimentally found that this strategy
increases the convergence of PSO in early iterations of the algorithm. In globallocal best inertia weight (Arumugam and Rao 2006), the inertia weight is based
on the function of local best and global best of the particles, in each generation.
It neither takes a constant value nor a linearly decreasing time-varying value.
Using the merits of chaotic optimization, Chaotic Inertia Weight has been
proposed by Feng et al. (2010).
Gao et al. (2008), proposed a new PSO algorithm which combined the
logarithm decreasing inertia weight with chaos mutation operator. Adaptive
parameter control strategies can be developed based on the identified
evolutionary state and by making use of existing research results on inertia
weight (Shi and Eberhart 1999; Shi and Eberhart 2001). Some use a selfadaptive method by encoding the parameters into the particles and optimizing
them together with the position during run time (Tripathi et al. 2007;
Yamaguchi and Yasuda 2006).

95

To maintain effective balance between exploration and exploitation, the

inertia weight is adapted during evolution independent of the dataset involved.
The inertia weight is made adaptive by two different techniques.

5.6.1 Methodology
The original PSO has pretty good convergence ability, but also suffers
the demerit of premature convergence, due to the loss of diversity. Improving
the exploration ability of PSO has been an active research topic in recent years.
Thus, the proposed algorithm introduces the concept of self adaptation as the
primary key to tune the two basic rules velocity and position. By improving the
inertia weight formulae in PSO the diversity of population could be achieved.
The basic PSO, as resented by Eberhart and Kennedy (1995), has no inertia
weight. In 1998, first time Shi and Eberhart presented the concept of constant
inertia weight.
By looking at equation (3.1) more closely, it can be seen that the
maximum velocity allowed actually serves as a constraint that controls the
maximum global exploration ability the PSO can have. By setting a too small
maximum for the velocity allowed, the maximum global exploration ability is
limited, and hence, PSO will always favor a local search, no matter what the
inertia weight is. PSO can have a large range of exploration ability, by setting
optimum value for inertia weight. Since the maximum velocity allowed affects
global exploration ability indirectly, and the inertia weight affects it directly, it
is better to control global exploration ability through inertia weight only. A way
to achieve this is by allowing inertia weight to control exploration ability. Thus,
the inertia weight is made to change automatically (self adaptive). Three self
adaptive inertia weights methods namely: Self Adaptive PSO1 (SAPSO1), Self
Adaptive PSO2 (SAPSO2) and Self Adaptive chaotic PSO (SACPSO) are
introduced for mining ARs in this work.

96

In order to linearly decrease the inertia weight as iteration progresses the

inertia weight is made adaptive through the equation (5.7) in SAPSO1.
= max( max min )

g
G

(5.7)

where, and are the maximum and minimum inertia weights, g is the
generation index and G is the predefined maximum number of generation.
In SAPSO2 the inertia weight adaptation is made to depend upon the
values from previous generation so as to linearly decrease its value with
increasing iterations, as given by equation (5.8).

( t+ 1 )= ( t )

( maxmin )
G

(5.8)

Where, (+1) is the inertia weight for the current generation, () inertia
weight for the previous generation, and are the maximum and
minimum inertia weights and G is the predefined maximum number of
generation.
The value of and are set as 0.9 and 0.4 respectively.
The values are arrived from section 5.2 (Figure 5.4) where the
PA of ARM arrived is optimum, when values are within the range [0.4, 0.9].
The velocity update equation specified in Eqn. (6.1) is applied for
mining ARs. The position update is made as given in equation 3.2. For both
SAPSO1 and SAPSO2 methods, the inertia weight is adjusted during evolution
based on equation (5.7) and (5.8).
In the proposed SACPSO methodology, the inertia weight adaption is
made in chaotic PSO for ARM. The velocity update equation of chaotic PSO
(Eqn. 5.3) is altered by adding the inertia weight as given in Equation (5.9).

97
old
v new
i =v i +c 1Chaotic operator1 ( lbest x i ) + c 2Chaotic operator2 ( gbest x i ) (5.9)

Here, the inertia weight adaptation is performed based on the equation

(5.7), for mining ARs.
5.6.2 Experimental Results and Discussion
ARM is performed on the five datasets using the three adaptive
methodologies discussed in the previous section. The initial parameter setting
for the experiment is listed in Table 5.8.
Table 5.8 Default Parameter Setting of Adaptive
Methodologies for ARM

Parameter
Name
Population Size

Initial Velocity
c1 , c2
Vmax

Value
Lenses
Habermans Survival
Car Evaluation
Postoperative Care
Zoo
0
2

: 20
: 300
: 1000
: 50
: 100

1
0.4
0.9

ARs are mined with the three methodologies; SAPSO1, SAPSO2 and
SACPSO for the five datasets. The PA achieved by the three methods over
generations is plotted individually for the five datasets as given in Figure 5.10
(a-e).

98

100
95
90
85
80
75

Predictive Accuarcy (%)

PS O

S APS O1 70

S APS O2

S ACPS O

65
60
55
50

10

20

40

60

80

100

No. of Iterations

(a)

Lenses Dataset
100

95

90

Predictive Accuracy (%)

PS O

85

S APS O1

S APS O2

S ACPS O

80

75

70

10

20

40

60

No. of Iterations

(b) Habermans Survival Dataset

80

100

99

92

82

72
Predictive Accuracy (%)
PS O

S APS O1
62

S APS O2

S ACPS O

52

42

10

20

40

60

80

100

No. of Iterations

(c) Car Evaluation Dataset

100

90

80

Predictive Accuracy (%)

PS O

70

S APS O1

S APS O2

S ACPS O

60

50

40

10

20

40

60

No. of Iterations

(d) Postoperative Patient Dataset

80

100

100

100
99
98
97
96
Predictive Accuracy (%)
PS O

95

S APS O1 94

S APS O2

S ACPS O

93
92
91
90

10

20

40

60

80

100

No. of Iterations

(e) Post Operative Patient Dataset

Figure 5.10 Comparison of Predictive Accuracy of three Adaptive
Methodologies with PSO for ARM
From Figure 5.10 a to e it is observed that the three adaptation
techniques: SAPSO1, SAPSO2 and SACPSO generated ARs with better PA
than PSO methodology. SAPSO1 methodology performs marginally better
when compared to other two methodologies except for the Habermans
Survival dataset, where SACPSOs PA is better. The performance of SAPSO2
is consistent for all the five datasets throughout the evolution process.
Introduction of adaptation in inertia weight enhances the accuracy of the
system considerably. The inertia weight controls the impact of previous flying
experience, which is utilized to keep the balance between exploration and
exploitation. The adaptation of the control parameters result in enhancement of
the accuracy thereby increasing the ability of PSO for mining ARs.
5.7

DATA DEPENDENT ADAPTION IN PSO

The literature on adaptive strategies in GA and PSO focuses on the

mechanisms of adaptation over generation either decreasing or increasing the

101

parameter values, independent of the data involved to enhance the

performance. As ARM is mainly based on relationship or correlation between
items in a dataset, data dependent strategy of adaptation could generate better
associated rules both qualitatively and quantitatively. The performance of the
adaptive strategy for mining ARs could be enhanced, if the parameter
adjustments are done depending on: i) the data involved and ii) fitness values
used for evolution through generations. The proposed ARM based on data
dependent adaptation of control parameters in PSO, is discussed in the
following section.
5.7.1

Methodology
The roles of the parameters are significant in the performance of the

methodologies. PSO is mainly conducted by three key parameters important for

the speed, convergence and efficiency of the algorithm (Yao et al. 1999): the
inertia weight ()

and two positive acceleration coefficients (c 1 and c2).

Inertia weight controls the impact of the velocity history into the new velocity.
Acceleration parameters are typically two positive constants, called the
cognitive parameter c1 and social parameter c2.
Inertia weight plays a key role in the process of providing balance
between exploration and exploitation process. The linearly decreasing inertia
weight over time enhances the efficiency and performance of PSO (Xin et al.,
2009). Suitable fine-tuning of cognitive and social parameters c 1 and c2 may
result in faster convergence of the algorithm and alleviate the risk of settling in
one of the local minima. To adapt the parameters of PSO in the proposed
adaptive PSO methodology for ARM, parameter tuning was performed based
on the evolutionary state to which the data is fits. The acceleration coefficients
are adapted based on the dataset from which ARs are mined. The inertia weight
is adjusted based on the fitness value of the individuals. The pseudo code for
the Adaptive PSO is given below:

102

/* Ns: size of the swarm, C: maximum number of iterations, Of : the final output*/
i.

t = 0, randomly initialize S0,

Initialize xi, i, i {1, . . .,Ns} /* xi : the ith particle */

Initialize vi, i, i {1, . . .,Ns} /* vi : the velocity of the ith particle*/
Initialize 1, c1(1), c2(1)
/* : Inertia weight, c1, c2 :
Acceleration
Coefficients */
Pbi xi, i, i {1, . . .,Ns} /*
Pbi : the personal best of the ith
particle */
Gb xi
/* Gb : the global best particle */

ii. for t = 1 to C,
for i = 1 to Ns
F(xi) = confidence(xi )

/* C : total no of iterations */
log (support (xi)

(length(x) + 1)

/* F(xi) : Fitness of xi */
If ( F(xi) < F(Pbi))
Pbi xi

/* Update particle best */

Gb min(Pb1, Pb2, , PbNs)

/* Update global best */

adjust parameters((t), c1(t), c2(t)) /* Adaptive Adjustment */

vi(t) = (t)vi (t-1)+ c1(t)r1(Pbi xi) + c2(t) r2(Gb xi )
/* Velocity Updation
r1, r2 : random values*/
xi (t)= xi(t-1) + vi(t)
/* Position Updation */

A(t) non dominated(S(t) A(t))

iii. Of A(t) and stop

/* Updating the Archive*/

/* Of : Output*/

Adjust parameters ((t), c1(t), c2(t)) in the above pseudo code is achieved
through adaptive mechanism newly proposed. The proposed approach based on
estimation of evolutionary state distributes the evolution of data into four
states, namely, Convergence, Exploration, Exploitation and Jumping out.

103

Estimation of Evolutionary State (EES)

During the PSO process, the population distribution characteristics vary
not only with the generation number, but also with the evolutionary state. For
example, at an early stage, the particles may be scattered in various areas, and
hence, the population distribution is dispersive. As the evolutionary process
goes on, the particles would cluster together and converge to a local or globally
optimal area. Hence, the population distribution information would differ from
the early stage. The estimation of evolutionary state detects the population
distribution information and locates the region in which the particle lies.
Based on the search behaviors and the population distribution
characteristics of the PSO, estimation of evolution state is done as follows
a. The distance between particles is calculated using the Euclidean distance
measure for each particle i using the Equation (5.10).
1
d i=
N1

(x jx i )2

j=1, j i

(5.10)

Where, N is the population size, xi and xj are the ith and jth particles in
the population, respectively.
b. Calculate the evolutionary state estimator (e), defined as

e=

d gd min
d max d min

(5.11)

dg

d max

where,
d min

is the distance measure of the gBest particle,

are the maximum and minimum distance measures respectively,

from step a, above.

c. Record the evolutionary (e) factor for 100 generations for each dataset
individually.

104

d. Classify the estimator (e), to the state it belongs: Exploration,

Exploitation, Convergence, Jumping out, for the datasets based on the
evolutionary states.
The evolutionary state estimator e over hundred generations is plotted
for the zoo dataset as shown in Figure 5.11. The evolution states are estimated
based on the e values. The state transition is different for the five datasets and
is nondeterministic and fuzzy. Based on the evolution state estimator values,
classification is done for determining the intervals at which the class transition
occurs.
60

50

40

Evaluationestimator (e)

30

20

10

9
4

1
3

Iteration No.

Figure 5.11Evolutionary State Information Robustly

Revealed by e at Run Time for Zoo Dataset
The intervals arrived for the five datasets based on EES is given in Table
5.9.
Table 5.9 Classification into States by Evolutionary Factor

105

Datasets
EES State
Convergence

0.0 0.3

Car
Habermans Postoperativ
Evaluatio
Zoo
Survival
e Patient
n
0.0- 0.15 0.0 - 0.4
0.0 - 0.5
0.0- 0.15

Exploitation

0.1- 0.4

0.15- 0.25 0.3-0.7

0.2- 0.6

0.1- 0.35

Exploration

0.2- 0.7

0.1- 0.3

0.6- 0.9

0.4-0.8

0.2- 0.4

Jumping out

0.6-1

0.3-1

0.8-1

0.7-1

0.3-1

Lenses

The change of state reflected as per the PSO sequence is Convergence

Exploitation

Exploration

Jumping Out

Convergence.

The formulation of numerical implementation of the classification for

the Habermans Survival dataset is as follows:
Case (a) Exploration: A medium to large value of e represents exploration,
whose membership function is defined as:

0,e <0.6
5 e3, 0.6< e 0.7
exploration ( e )=
1, 0.7<e 0.8
10 e+ 90.8< e 0.9
0,0.9< e 1

(5.12a)

Case (b)Exploitation: A shrunk value of e represents exploitation whose

membership function is defined as:

0,0< e 0.3
10 e3,0.3< e 0.4
exploitation ( e ) =
1, 0.4< e 0.6
5 e+ 4 0.6< e 0.7
0, 0.7<e 1

(5.12b)

Case (c)Convergence: A minimal value of e represents convergence, whose

membership function is defined as:

106

1, 0<e 0.3
convergen ce ( e )= 5 e+ 2.5 0.3< e 0.4
0, 0.4< e 1

(5.12c)

Case (d)Jumping Out: When PSO is jumping out of a local optimum, the
global best particle is distinctively away from the swarming cluster. Hence, the
largest value of f reflects S4, whose membership function is, thus, defined as:

0, 0< e 0.8
jumping out ( e )= 5 e+5 0.8< e 0.9
1, 0.9< e 1

(5.12d)

Therefore, at a transitional period, two memberships will be activated,

and e can be classified to either state. The singleton method of defuzzification
is adopted. The distribution of all particles in PSO, based on Euclidean distance
calculated is illustrated in Figure 5.12.

d g d pi

, Exploration

Convergence
Particle

d g d pi

, Exploitation,

d g d pi

, Jumping

Out
- Best particle of the swarm

Figure 5.12 Population Distribution in PSO based on Evolutionary factor e

107

Based on the estimation into states using evolutionary estimation factor

e and fitness values, the adaptation of acceleration coefficients and inertia
weight is performed.
Adaptive Control of Acceleration Coefficients
The

acceleration

coefficients

are

made

adaptive

through

the

classification of evolutionary states. Parameter c1 represents the selfcognition that pulls the particle to its own historical best position, helping in
exploration of local niches and maintaining the diversity of the swarm.
Parameter c2 represents the social influence that pushes the swarm to
converge to the current globally best region, helping with fast convergence.
Both c1 and c2 are initially set to 2 (Song and Gu 2004; Mendes et al. 2004),
based on the published literature. The acceleration coefficients are adaptively
altered during evolution, according to the evolutionary state, with strategies
developed. The strategy to adopt for the four states is given in Table 5.10. The
values for and

are empirically arrived for the datasets taken up for study.

Table 5.10 Control Strategies for c1 and c2

State/Acceleratio

c1

c2

n Coefficient
Exploration
Exploitation

Increase by
Increase by

Decrease by
Decrease by

Convergence

Increase by

Increase by

Jumping out

Decrease by

Increase by

The increase or decrease in the values of c 1 and c2 arrived in Table 5.10 is

discussed below.
Exploration: During exploration, particles should be allowed to explore as
many optimal regions as possible. This avoids crowding over single optima,

108

probably the local optima, and explores the target thoroughly. Increase in the
value of c1 and decrease in c2 facilitate this process.
Exploitation: In this state, based on the historical best positions of each
particle, they group towards those points. The local information of the particle
aids this process. A slight increase in c1 advances the search around particle best
(pBest) positions. At the same time the slight decrease in c 2 avoids the
deception of local optima, as the final global position has yet to be explored.
Convergence: In this state, the swarm identifies the global optima. All the
other particles, in the swarm should lead towards the global optima region. The
slight increase in the value of c2 helps this process. To fasten up the process of
convergence, a slight increase in the value of c1 is adopted.
Jumping Out: The global best (gBest) particle move away from the local
optima towards global optima, taking it away from the crowding cluster. Once
any particle in the swarm reaches this region, then all particles are to follow the
same pattern rapidly. A large c2 along with a relatively small c1 value helps to
obtain this goal.
Bounds for Acceleration Coefficient
Adjustments on the acceleration coefficients should be minimum so as
to maintain the balance between values c1 and c2. Hence, the maximum increase
or decrease between two generations is in the range [0.02, 0.1]. The value of

is set as 0.02 based on trials and the value for is set as 0.06 similarly. c 1

and c2 values are clamped in the interval [1.5, 2.5] in order to maintain the
diversity of the swarm and fast convergence (Carlisle and Dozier 2001). The
sum of the acceleration coefficients is limited to 4.0 (Zhan et al. 2007), when
the sum exceeds this limit then both c1 and c2 are normalized based on equation
(5.13).

109

c i=

ci
4.0, i=1,2.
c 1+c 2

(5.13)

The adaptation behavior of the acceleration coefficients c 1 and c2 through

change of states, in estimation of evolution state for Zoo dataset is shown in
Figure 5.13.

2.4
2.3
2.2
2.1
2
Coefficient Value
C1

1.9
1.8

C2

C1

C2

1.7
1.6
61

51
56

46

36
41

26
31

16
21

6
11

1.5
Generation Number

Figure 5.13Adaptation of Acceleration Coefficients through EES

for

Zoo Dataset
Inertia Weight Adaptation

The inertia weight controls the impact of previous flying experience,

which is utilized to keep the balance between exploration and exploitation. The
particle adjusts its trajectory according to its best experience and enjoys the
information of its neighbors. In addition, the inertia weight is also an important
convergence factor; smaller the inertia weight, faster the convergence of PSO.
A linear decrease in inertia weight gradually may swerve the particle from their

110

global optima (Chatterjee and Siarry 2006). Hence, a nonlinear adaptation of

inertia weight as proposed in the given Equation (5.14), is considered as the
solution to the above issue. The global best particle is derived based on the
fitness value of the particles in the swarm. The proposed methodology for
adopting the inertia weight is based on the fitness values exhibited by the
particles.

max , f f avg
( f max f )( max min)
max
, f avg /2< f f avg
=
f max f min
( max min)
min +i
, f < f avg /2
G

(5.14)

where, f, fmin and favg are the fitness values, which are decided by the current
particles in the swarm. The minimum fitness value of all particles in swarm is
defined as fmin, while the mean value is selected as f avg. The adaptation of inertia
weight for zoo dataset over generations is given in Figure 5.14.

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1

Iterations

99

85
92

71
78

57
64

43
50

29
36

8
15
22

0
1

Inertia Weight

111

Figure 5.14 Inertia Weight Adaptation for Zoo dataset

From figure it can be concluded that in general, the change in inertia
weight decreases linearly. In detail, it is adapted based on the fitness
information dynamically. The value of inertia weight swings around 0.461
frequently for all generations. With the above value of inertia weight, the
convergence ability in the search space, to avoid the risk of being trapped into
local minima, and improve the diversity of PSO.
5.7.2 Experimental Results and Discussion
The objective of the adaptive PSO is to enhance the performance of
ARM by making the adaption dependent on dataset used. The proposed
methodology was validated using the bench mark datasets chosen for validating
the experiment.
Five datasets were used for analyzing the performance of APSO and
comparing with that of simple PSO. The parameter setting for the APSO
methodology is given in the Table 5.11.
Table 5.11 Parameter Setting for APSO
Parameter Name
Population Size

No. of Generations
0

pm

Parameter Value
Lenses
Habermans Survival
Car Evaluation
Postoperative Care
Zoo
100
0.3

pc

0.3
0.25

c1 , c2
max

2
0.9

min

0.4

: 20
: 300
: 1000
: 50
: 100

112

The scope of this study on ARM using adaptive PSO is to

Mine ARs with adaptive PSO with multiobjective, namely, accuracy
and significance of the rules generated;
Compare the performance with simple

PSO

(non

adaptive

methodologies);
Analyze the performance over generations and
Convergence analysis of APSO in terms of execution time
The APSO methodology was applied on the five datasets to mine ARs
and the PA was recorded. The results are compared with that of simple GA
discussed in chapter 4 and PSO discussed earlier in this chapter.
The APSO methodology for mining ARs adapt the acceleration
coefficients based on the estimation of evolutionary state. Based on the fitness
values the inertia weight is adapted. Then, the velocity modification is based on
the state in which the particle lies. This acts as a balance between exploration
and exploitation, thus escaping from premature convergence. The PA of the
ARs mined is improved through the evolution process.
The PA for the adaptive PSO methodology over 100 generations, is
shown in Figure 5.15. The adaptive PSOs performance on Car Evaluation
dataset and Zoo data set is consistent. The performance on Habermans survival
dataset and Post operative patient dataset, is maximum at the end of evolution.
This avoids premature convergence at initial stages. For the Lenses dataset
where the dataset size is small, global search space is maintained effectively
making the convergence possible, even at early iterations.
The Laplace, Conviction, Leverage and Lift measures of the five
datasets for the adaptive PSO is recorded as given in Table 5.12.

113

100
99
98
97
Predictive Accuracy (%)

Car
Haberman

96

Lens

95

Postop
Zoo

94
93
Iteration No.

Figure 5.15 Predictive Accuracy of Adaptive PSO over hundred

Generations

Table 5.12 Multiobjective Measures of APSO

Lenses

Laplace
Convictio
n
Leverage
Lift

0.5294

Haberman's
Survival

Car
Evaluatio
n

Postoperativ
e Patient

Zoo

0.501608

0.502488

0.5028

0.5024

Infinity

Infinity

Infinity

Infinity

Infinity

0.026
1.548

0.003039
2.0458

0.002394
1.9875

0.0301
2.2478

0.0249
2.4558

When the Laplace measure is away from 1, it indicates that the

antecedent values are dependent on the consequent values, and hence the rules
generated are of significant. The conviction measure which is infinity for all
datasets show that the rules generated are interesting. The Leverage measure
being far away from 1 again insists on the interestingness of the rules

114

generated. The measure of Lift for all the datasets, greater than1 signifies the
dependency of the consequent on the antecedent. The rules generated by
Adaptive PSO are of importance based on the Lift measure.
The PA achieved from the adaptive PSO methodology is compared with
that of simple PSO, as shown in Figure 5.16. The Adaptive PSO performs
better than the simple PSO for all the five datasets, when applied for ARM.
100
95
90
85
80
Predictive Accuracy (%)

75
PSO

70

APSO

65

Dataset

Figure 5.16 Predictive Accuracy comparison of APSO with PSO

The number of rules generated by APSO with PA less than 0.05
percentage of the maximum PA arrived, is plotted for mining ARs against the
simple PSO methodology for the datasets in Figure 5.17. In terms of number of
rules generated adaptive PSO performs better for Hagermans Survival, Zoo
and car evaluation datasets. The number of rules generated for lens and
postoperative patient dataset is marginally better than that of simple PSO. As
the number of instances in both datasets is small, the increase in the number of
rules is found to be minimum.

115

35
30
25
20
No. of Rules Generated 15
PSO

APSO
10
5
0

Datasets

Figure 5.17Comparison of No. of Rules Generated by APSO and PSO

The execution time of APSO for mining ARs from the datasets in
comparison with PSO methodology is given in Figure 5.18. This is the time
taken by the system to generate ARs (CPU time). The number of iterations
when the PA is at maximum, is taken as point at which the execution time is
recorded. The execution time of the APSO for mining ARs is more, when
compared to PSO. The adaption mechanism of acceleration coefficients based
on evolution factor, and inertia weight adaptation increases the complexity of
the algorithm, resulting in higher execution time when compared to PSO.

116

10000
9000
8000
7000
6000
5000
4000
Execution Time (ms)
3000
2000
1000
0

9634

6532

6123
4586
3425

2552
1841
1243

PSO
APSO

84156

Datasets

Figure 5.18 Comparison of Execution Time of APSO and PSO

When number of instances in the dataset is less then increase in

execution time is found to be marginal. Thus, the difference in the execution
time for mining rules for the Lenses, Postoperative Patient and Zoo datasets,
are minimum for APSO than PSO. For the Habermans survival dataset with
moderate number of instances, there is a noticeable increase in execution time,
whereas, for the car evaluation dataset, the difference in execution time is large.
The consistent performance of the APSO in terms of PA throughout evolution
and the PA achieved from APSO for mining ARs, balances the difference in the
execution time.
The major drawback of the simple PSO is its premature convergence,
where, the particle fixes some local optima as the target (global search area),
where all particles converge locally. One of the objectives of Adaptive PSO is
to avoid the premature convergence, thereby striking a balance between
exploration and exploitation. The iteration number at which PA is high is
plotted for both APSO and PSO methodologies, for mining ARs for the datasets
used in Figure 5.19.

117

100
90
80
70
60
50
40

Iteration Number

30
PS O20

APS O

10
0

Datasets

Figure 5.19 Comparison of Convergence of APSO and PSO

The performance of APSO is compared with SAPSO1 and SAPSO2 as
given in Figure 5.20 and it is noted that the data dependent methodology of
mining ARs performs better than data independent methodologies in terms of
PA.
100
99.5
99
98.5

Predictive Accuracy %

98

S APS O2
S APS O1

97.5

APS O

Datasets

Figure 5.20 Comparison of Convergence of APSO and SAPSO

118

The analysis of the performance of the five datasets through the adaptive
methods of PSO has been compared with simple PSO. The Adaptive PSO
generated ARs with better PA and qualitative measures. To test the significance
of the proposed APSO, it is compared with Ant Colony Optimization (ACO) (a
similar population based search method) for mining rules. Four more datasets
from UCI repository for which ARM using ACO has been carried out, have
been chosen from published literature. The PA of the mined rules using APSO
for these datasets was recorded. The comparative results obtained by AGA,
APSO and ACO are given in Table 5.13.
Table 5.13 Predictive Accuracy Comparison APSO with AGA and ACO
Dataset
Lenses
Habermans
Survival
Car
Evaluation
Postoperativ

Adaptive Adaptive

Lale et

Parpinelli

Liu et Lale et

et al. 2002 al. 2003 al. 2008

-

GA
87.5

PSO
98.1

al. 2010
-

68

99.6

96

99.95

e Care
Zoo
Iris
Nursery
Tic Tac Toe
Wisconsin

92

99.54

91
96
97.30
88.36

99.72
98.86
99.12
98.76

98
97.24
-

75.57

76.58

98.67
97.16
97.59

Breast

92.54

98.75

96.97

94.32

Cancer
Note : - Data not available in study
The adaptive PSO methodology generates ARs with better PA when
compared to other methodologies analyzed. The adaptive PSO methodology
when applied for mining ARs generates rules with enhanced PA, increase in the
number of rules generated, along with better rule set measures. The increase in
the execution time of APSO when compared to simple PSO methodology is

119

also minimum taking into account the complexity of the methodology and the
increase in accuracy achieved over PSO.
The balance in setting the global search space between exploration and
exploitation is also attained, while mining ARs with APSO, for all the datasets
used. This is noted from the shift in iteration number of APSO when compared
to simple PSO, where highest PA is noted. Adopting the acceleration
coefficients c1 (cognitive factor) and c2 (social factor) through EES
methodology, makes the adjustments in fixing up the global search space
effectively with the help of local optima. The fitness value of the particles is
used in adapting inertia weight. This helps in better balance between
exploration and exploitation when particles fly through the search space.
The comparison of PA for mining ARs is between the methods proposed
based on PSO in given in Table 5.14. From the table it is clear that the APSO
methodology performs better in terms of PA among the other methods for
ARM. Thus the data-dependent adaptation of control parameters maintains the
diversity of the population effectively, thereby resulting in optimal PA.
Table 5.14 Predictive Accuracy Comparison of PSO based Methods
PSO
Lenses
Car Evaluation
Habermans Survival
Po-opert Care
Zoo

CPSO

NPSO

WPS

SAPSO

SAPSO

SACPS

APSO

83.57

87.5

93.1

97.91

97.91

97.91

97.82

98.1

97.61

99.86

97.1

99.93

99.93

99.92

99.91

99.95

92.86

96.15

92

99.48

99.29

99.2

99.74

99.6

83.33

92.86

94.5

99.29

99.18

99.47

98.61

99.54

95.45

94.44

96.5

96.67

97.92

99.09

98.69

99.72

120

5.8

SUMMARY

PSO algorithm with and without inertia weight was applied for mining
ARs. Based on the results two new proposals for ARM were made namely;
Chaotic

PSO and Dynamic

Neighborhood PSO.

These

proposed

methodologies generated ARs with better PA and quantitative measures. In data

independent adaptation three different methodologies are proposed for mining
ARs. Among SAPSO1, SAPSO2 and SACPSO the SAPSO1 performs better in
terms of PA of the AR mined. The proposed APSO methodology is dependent
on data for adjusting the control parameters, and the generated ARs with better
PA and in terms of number of rules generated.