Heterogeneous neural networks and the leader2 algorithm

This paper is the final document written to gather the impressions and conclusions which we have come to during the development of this master thesis. In this research project you will find the description of a new kind of artificial neural network, Heterogeneous Neural Network 2 (HNN2), which can be seen as a general abstraction of the Radial Basis Function network. The model of neuron used is an improved version of the one presented by Belanche [1] and the neural network is initialized using a clustering algorithm, Leader2, developed at [2]. We will explain the way we have followed to get this artificial neural network that works allways with understandable information, uses the concept of similarity and allows users to improve the algorithm results taking advantage of expert information. The basic Heterogeneous Neural Network (HNN) is also known as Similarity Neural Network (SNN), by the importance of the similarity measures inside this method. The basic idea is that a combination of similarity functions, comparing variables independently, is more capable of catching better the singularity of an heterogeneous data set than other methods which require previous data transformation. Each variable has its own characteristics, which is information that can be used by the expert that knows it to choose its most suitable similarity function, taking advantage of all the information he has. If this is done for each variable, we will be working probably with a similarity measure that understands better the data. Missing values are also a relevant characteristic of heterogeneous data, so we have to learn to deal with them. All these ideas are applied to HNN and Leader2, joint to several improvements performed to the neural network, like regularization or Alternate Optimization, in order to fit better the data but avoiding overfitting. This is why we have called it Heterogeneous Neural Network 2 (HNN2). This document is divided in several chapters. Initially, we will give an in-depth description of the problem which we want to solve. In the second chapter, State of the art, you will get a wide perspective of how was the field in which this project has been developed before we started. Then, there is a description of the used methodology, where you can find the main decisions and the development itself, followed by the explanation of the experimental settings done to test the HNN2. Their results are commented and evaluated in the next chapter, and next some conclusions are inferred. Finally, you will find the references used in the research and several annexes with additional relevant information. But in first term, before starting the description of the problem and in the way of making the reading easier, it is necessary to provide you some vocabulary to know exactly the meaning we have given to several key words. Next, in the same terms, you will find the most used symbols with their description