On Vectorization of Deep Convolutional Neural Networks for Vision Tasks

We recently have witnessed many ground-breaking re-sults in machine learning and computer vision, gen-erated by using deep convolutional neural networks (CNN). While the success mainly stems from the large volume of training data and the deep network architec-tures, the vector processing hardware (e.g. GPU) undis-putedly plays a vital role in modern CNN implemen-tations to support massive computation. Though much attention was paid in the extent literature to understand the algorithmic side of deep CNN, little research was dedicated to the vectorization for scaling up CNNs. In this paper, we studied the vectorization process of key building blocks in deep CNNs, in order to better under-stand and facilitate parallel implementation. Key steps in training and testing deep CNNs are abstracted as ma-trix and vector operators, upon which parallelism can be easily achieved. We developed and compared six imple-mentations with various degrees of vectorization with which we illustrated the impact of vectorization on the speed of model training and testing. Besides, a unified CNN framework for both high-level and low-level vi-sion tasks is provided, along with a vectorized Mat-lab implementation with state-of-the-art speed perfor-mance