Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
This topic contains 0 replies, has 1 voice, and was last updated by arXiv 1 year, 2 months ago.

Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
Object recognition is an important task for improving the ability of visual systems to perform complex scene understanding. Recently, the Exponential Linear Unit (ELU) has been proposed as a key component for managing bias shift in Convolutional Neural Networks (CNNs), but defines a parameter that must be set by hand. In this paper, we propose learning a parameterization of ELU in order to learn the proper activation shape at each layer in the CNNs. Our results on the MNIST, CIFAR10/100 and ImageNet datasets using the NiN, Overfeat, AllCNN and ResNet networks indicate that our proposed Parametric ELU (PELU) has better performances than the nonparametric ELU. We have observed as much as a 7.28% relative error improvement on ImageNet with the NiN network, with only 0.0003% parameter increase. Our visual examination of the nonlinear behaviors adopted by Vgg using PELU shows that the network took advantage of the added flexibility by learning different activations at different layers.
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
by Ludovic Trottier, Philippe Giguère, Brahim Chaibdraa
https://arxiv.org/pdf/1605.09332v4.pdf
You must be logged in to reply to this topic.