Curious: Searching for Unknown Regions of Space with a Subpopulation-based Algorithm

Intrinsic motivation and novelty search are promising approaches to deal with plateaus, deceptive functions and other exploration problems where using only the main objective function is insufficient. However, it is not clear until now how and if intrinsic motivation (novelty search) can improve single objective algorithms in general. The hurdle is that using multi-objective algorithms to deal with single-objective problems adds an unnecessary overhead such as the search for non-dominated solutions. Here, we propose the Curious algorithm which is the first multi-objective algorithm focused on solving single-objective problems. Curious uses two subpopulations algorithms. One subpopulation is dedicated for improving objective function values and another one is added to search for unknown regions of space based on objective prediction errors. By using a differential evolution operator, genes from individuals in all subpopulations are mixed. In this way, the promising regions (solutions with high fitness) and unknown regions (solutions with high prediction error) are searched simultaneously. Because of thus realized strong yet well controlled novelty search, the algorithm possesses powerful exploration ability and outperforms usual single population based algorithms such as differential evolution. Thus, it demonstrates that the addition of intrinsic motivation is promising and should improve further single objective algorithms in general.