This report introduces a novel algorithm to learn the width of non-linear activation functions (of arbitrary analytical form) in layered networks. The algorithm is based on a steepest gradient-descent technique, and relies on the inductive proof of a theorem that involves the novel concept of expansion function of the activation associated to a given unit of the neural net. Experimental results obtained in a speaker nomalization task with a mixture of Multilayer Perceptron show a dramatic improvement of performance with respect to the standard Back-Propagation training
Scheda prodotto non validato
Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte di FBK.
Titolo: | Learning the Width of Activations in Neural Networks |
Autori: | |
Data di pubblicazione: | 1996 |
Abstract: | This report introduces a novel algorithm to learn the width of non-linear activation functions (of arbitrary analytical form) in layered networks. The algorithm is based on a steepest gradient-descent technique, and relies on the inductive proof of a theorem that involves the novel concept of expansion function of the activation associated to a given unit of the neural net. Experimental results obtained in a speaker nomalization task with a mixture of Multilayer Perceptron show a dramatic improvement of performance with respect to the standard Back-Propagation training |
Handle: | http://hdl.handle.net/11582/1296 |
Appare nelle tipologie: | 5.12 Altro |