On the trade-off between feature extraction and fine tuning in transfer learning

Transfer learning is the default solution when using deep learning in image-related tasks, like image classification. When a model has been trained in a large and varied enough dataset, it allows to reuse the visual features that it has learnt for tasks that may have limited training data or environments with limited computational resources. In this work we perform an experimental study on feature extraction and fine-tuning, the two most common transfer learning approaches for image classification. We evaluate the trade-offs of performing a hyperparameter search and the subsequent task with both approaches, in relation to performance, environmental footprint, computational and human involved resources. This work shows the cases in which feature extraction or fine tuning are preferable and proposes a series of recommendations of use for transfer learning, with respect to the aforementioned metrics