On the Learning Properties of Dueling DDQN in Parameter Control for Evolutionary and Swarm-based Algorithms

Lacerda M, Buarque de Lima Neto F, Amorim Neto H, Kuchen H, Ludermir T

Abstract

This work is intended to assess the learning capability of an agent implemented with a Dueling Double Deep Q-Network in the problem of parameter control for Evolutionary and Swarm-based algorithms. The objective is to build a general parameter control method for these algorithms, that can be used for any Population Based Algorithm (PBA) to solve any numerical optimization problem, implemented for any computing platform, and is able to choose a good sequence of parameter values for the PBA, given a time budget constraint. For the experiments, an implementation of the Particle Swarm Optimization for CUDA devices was chosen as the PBA and a set of well-known highly complex numerical minimization problems were used for the benchmark. The experiments showed that the agent is clearly able to evolve from a completely random decision policy to a fitness-minimization-oriented policy for most of the functions.