Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
This topic contains 0 replies, has 1 voice, and was last updated by arXiv 11 months, 3 weeks ago.

Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
Deep artificial neural networks (DNNs) are typically trained via gradientbased learning algorithms, namely backpropagation. Evolution strategies (ES) can rival backpropbased algorithms such as Qlearning and policy gradients on challenging deep reinforcement learning (RL) problems. However, ES can be considered a gradientbased algorithm because it performs stochastic gradient descent via an operation similar to a finitedifference approximation of the gradient. That raises the question of whether nongradientbased evolutionary algorithms can work at DNN scales. Here we demonstrate they can: we evolve the weights of a DNN with a simple, gradientfree, populationbased genetic algorithm (GA) and it performs well on hard deep RL problems, including Atari and humanoid locomotion. The Deep GA successfully evolves networks with over four million free parameters, the largest neural networks ever evolved with a traditional evolutionary algorithm. These results (1) expand our sense of the scale at which GAs can operate, (2) suggest intriguingly that in some cases following the gradient is not the best choice for optimizing performance, and (3) make immediately available the multitude of techniques that have been developed in the neuroevolution community to improve performance on RL problems. To demonstrate the latter, we show that combining DNNs with novelty search, which was designed to encourage exploration on tasks with deceptive or sparse reward functions, can solve a highdimensional problem on which rewardmaximizing algorithms (e.g. DQN, A3C, ES, and the GA) fail. Additionally, the Deep GA parallelizes better than ES, A3C, and DQN, and enables a stateoftheart compact encoding technique that can represent millionparameter DNNs in thousands of bytes.
Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning
by Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O. Stanley, Jeff Clune
https://arxiv.org/pdf/1712.06567v1.pdf
You must be logged in to reply to this topic.