Mid-level Deep Pattern Mining∗

Mid-level visual element discovery aims to find clusters of image patches that are both representative and discrimi-native. In this work, we study this problem from the prospec-tive of pattern mining while relying on the recently popular-ized Convolutional Neural Networks (CNNs). Specifically, we find that for an image patch, activation extracted from the fully-connected layer of CNNs have two appealing prop-erties which enable its seamless integration with pattern mining. Patterns are then discovered from a large number of CNN activations of image patches through the well-known association rule mining. When we retrieve and visualize image patches with the same pattern (See Fig. 1), surpris-ingly, they are not only visually similar but also semanti-cally consistent. We apply our approach to scene and ob-ject classification tasks, and demonstrate that our approach outperforms all previous works on mid-level visual element discovery by a sizeable margin with far fewer elements be-ing used. Our approach also outperforms or matches recent works using CNN for these tasks