Evolving Neural Networks That Are Both Modular and Regular: HyperNeat Plus the Connection Cost Technique

One of humanity’s grand scientific challenges is to create ar-tificially intelligent robots that rival natural animals in intel-ligence and agility. A key enabler of such animal complexity is the fact that animal brains are structurally organized in that they exhibit modularity and regularity, amongst other attributes. Modularity is the localization of function within an encapsulated unit. Regularity refers to the compressibil-ity of the information describing a structure, and typically involves symmetries and repetition. These properties im-prove evolvability, but they rarely emerge in evolutionary algorithms without specific techniques to encourage them. It has been shown that (1) modularity can be evolved in neural networks by adding a cost for neural connections and, sepa-rately, (2) that the HyperNEAT algorithm produces neural networks with complex, functional regularities. In this pa-per we show that adding the connection cost technique to HyperNEAT produces neural networks that are significantly more modular, regular, and higher performing than Hyper-NEAT without a connection cost, even when compared to a variant of HyperNEAT that was specifically designed to en-courage modularity. Our results represent a stepping stone towards the goal of producing artificial neural networks that share key organizational properties with the brains of natu-ral animals