Generation of art works using deep neural networks

Master in Artificial Intelligence, Facultat d’Informàtica de Barcelona (FIB), Facultat de Matemàtiques (UB), Escola Tècnica Superior d’Enginyeria (URV), Universitat Politècnica de Catalunya (UPC), Universitat de Barcelona (UB), Universitat Rovira i Virgili (URV).Art creation has always been on of the most popular and difficult challenges of Artificial Intelligence. This work uses state of the art Artificial Intelligence techniques to create artistic images. First the concept of what art is is analysed along its history. It is related to its capacity to show a conceptual and subjective representation of reality that is able to stimulate our minds. The representative power of Deep Neural Networks is used to create, from photographs of reality, artworks. Along the Art history, from paleolithic to modern art, Art has been used to communicate to other humans (or gods) ideas in human minds. From ancient times, Art studies the relations that already exist in nature, like the golden ratio, and applies them to its works to provide a pleasant feeling to observers. Art finds the ways to express ideas as they are stored in minds, not like perfect reproductions of reality. These ideas are affected by subjective point of view, preconceptions, emotions or the artist’s whole understanding of the world. The artwork is able to communicate author’s ideas to other humans. In modern times Art moved away from politics, religion and society and started looking for its own reasons. The concept of aesthetic is born as the philosophy of the art. Kant states that beauty is not a property of an artwork or natural phenomenon, but is instead a consciousness of the pleasure that attends the ’free play’ of the imagination and the understanding. From different attitudes, the artists expected to interpret the reality creating feelings of vitality, pleasure, imagination and reflection, giving birth to different styles or artistic movements. Some philosophers and artists call nowadays the Death of Art period, since everything can be considered Art. The process of creating an artwork starts with the perception of reality. In our brain and mind a representation of reality is coded in the form of neural impulses and ideas. These are combined with previous state of mind, ideas and emotions. The imagination employs the same circuits used by perception to create a mental representation of an artwork that produces in the mind the stimulus that the artist wants to communicate. During the creation of the artwork the artist keeps modifying it until it matches the imaginary image, until the artwork produces in the artist the desired effect. The algorithm presented emulates, up to a certain point, this process. Deep Neural Networks haveproved its capacity to learn to build a representation of an image useful for complex tasks like image classification. Their training on real world examples provides them capacity of >understanding> real images somehow similar to ours. The process of creating an artwork starts with the encoding of the images presented in the form of features that correspond to the activations of the pre-trained Very Deep Neural Network at different layers. A starting image is provided as a basis for the creation of the artwork. It is also fed to the same Neural Network to build its feature representation. These feature representations are used as they are, combined in more elaborate features, or modified. These combinations and modifications are defined in the form of losses on the features. The gradients of the errors are back-propagated through the Neural Network until the image layer. Optimisation is performed on the image to obtain a new image that matches better the desired representation in the activations of the neurons. This process is repeated until the images matches perfectly the desired representation, it can not evolve to a better solution or the user stops it. The different errors used correspond to different models that pretend to focus on different features of the images and different effects. Here are presented: Content model, represents the same image that is fed and tries to make the generated image to be a reconstruction of it. Texture model: represents the texture ofthe image presented; is based on combinations of features that use to be present in same place. Maximum Activation: maximises the sum of the activations of the neurons. Minimum activation: minimises the sum of activations. The last two have a version where the features to maximise or minimise need to be present too in an auxiliary image. All of them can be combined working on the same framework having as an input an arbitrary number of images represented at different layers. Relative importance of different models is controlled using sets of weights for each model, each image and each layer. The number of possibilities that this technique offers is unlimited. Several experiments are done showing the effects of each model and the main parameters of them. Also combinations of models are tested. The results are interpreted from the perspective of what do they represent and their analogy to human processes and the practical use of the algorithm for art creation as well. Up to this point the algorithm is able to produce artistic works. Nevertheless the images produced usually have some noise or small artifacts. To provide a painting finish they are processed with a special Convolutional Neural Network. This convolutional neural network was originally designed for super-resolution. It is trained using low-res inputs and high-res outputs. For this work I am using a set of weights trained with Japanese anime images. They encode a plain colours effect while keeping sharp edges, quite similar to the style of painting artworks. This work and some of the artworks produced were presented with Cadi Art Natura and Espai d’Art La Duana to a group of artists. They recognised the artistic quality of the artworks and admitted that never would think that they were made by a computer. They explained interpretations and similarities with other styles. They considered this an interesting tool for their work and suggested uses and possibilities that I did not consider before. There is a world of possibilities to explore in the use of Deep Neural Networks for Art creation.Peer Reviewe