Deep Transfer Learning for Art Classification Problems

peer reviewedIn this paper we investigate whether Deep Convolutional Neural Net- works (DCNNs), which have obtained state of the art results on the ImageNet challenge, are able to perform equally well on three different art classification problems. In particular, we assess whether it is beneficial to fine tune the net- works instead of just using them as off the shelf feature extractors for a sepa- rately trained softmax classifier. Our experiments show how the first approach yields significantly better results and allows the DCNNs to develop new selective attention mechanisms over the images, which provide powerful insights about which pixel regions allow the networks successfully tackle the proposed classi- fication challenges. Furthermore, we also show how DCNNs, which have been fine tuned on a large artistic collection, outperform the same architectures which are pre-trained on the ImageNet dataset only, when it comes to the classification of heritage objects from a different dataset.INSIGHT: Intelligent neural systems as integrated heritage tool