Novel Decomposition-Based Multi-objective Evolutionary Algorithm Using Reinforcement Learning Adaptive Operator Selection (MOEA/D-QL)

Abstract

It is known that applying adaptive operator selection (AOS) techniques can improve the search process in the space of candidate solutions of a multi-objective evolutionary algorithm. The AOS consists of two main tasks; the first assigns the weights or credits to each available operator, and the second selects the best operator. For the first time in this chapter, a reinforcement learning technique (Q-learning) is used to perform an adaptive selection of operators in the MOEA/D algorithm, which we call MOEA/D-QL. The objective of Q-learning is to learn a series of rules that tell an agent what action to take under certain circumstances; that is, an agent seeks to execute the actions that give it the most ac-cumulated reward. In this case, an action corresponds to a variation operator. Four variants of the Differential Evolution operator have been used for this work. In addition, two states are also used: the first state, $S_{0}$, corresponds to a child solution that enters front 0, and the second state, $S_{1}$, corresponds to solutions that do not enter front 0. MOEA/D-QL algorithm has been validated by comparing it with two state-of-the-art multi-objective algorithms, MOEA/D and a version of MOEA/D that uses an AOS based on Thompson's dynamic sampling called MOEA/D-DYTS. Fifteen multi-objective reference problems with 2 and 3 objectives were used as instances. Three metrics have been applied: hypervolume, generalized dispersion, and inverse generational distance. The non-parametric tests Wilcoxon signed-rank and Friedman were applied with a significance level of 5%, where it is observed that the MOEA/D-QL algorithm is superior in the hypervolume and inverted generational distance metrics.