Descorrelacao de dados e aceleracao da aprendizagem em redes neuronais

The back propagation algorithm is the most popular supervised training method for neural networks. In its original version, this algorithm corresponds to a simple gradient method applied to the minimization of a cost functional. However, the large number of parameters and the highly nonlinear nature of the cost functional, typical in neural network applications, often imply too slow learning processes. This work aims at contributing to the development of faster training methods for neural networks. This problem is considered using two complementary approaches. In the first part, we propose to include, in the back propagation algorithm, independent learning rate parameters for each synaptic weight, which are adapted according to the sign changes of the partial derivatives of the cost functional. This method is a simple way to exploit the 2"n"d order information of the diagonal of the Hessian matrix, avoiding the explicit computation of this matrix. In the second part of this thesis, we consider the possibility of further improving the performance of back propagation by performing a data ortho normalization at each layer of a multilayer network. For the ortho normalization process, we propose to integrate in conventional multilayer architectures additional linear layers, trained by a novel decorrelation algorithm. This unsupervised training algorithm exhibits an isotropic behaviour relative to the input data distribution, a property which is not present in other distributed decorrelation algorithmsAvailable from Fundacao para a Ciencia e a Tecnologia, Servico de Informacao e Documentacao, Av. D. Carlos I, 126, 1200 Lisboa / FCT - Fundação para o Ciência e a TecnologiaSIGLEPTPortuga