Optimization Methods for Convolutional Sparse Coding

Abstract Sparse and convolutional constraints form a natural prior for many optimization problems that arise from physical processes. Detecting motifs in speech and musical passages, super-resolving images, com-pressing videos, and reconstructing harmonic motions can all leverage redundancies introduced by convo-lution. Solving problems involving sparse and convo-lutional constraints remains a difficult computational problem, however. In this paper we present an overview of convolutional sparse coding in a consistent frame-work. The objective involves iteratively optimizing a convolutional least-squares term for the basis functions, followed by an L1-regularized least squares term for the sparse coefficients. We discuss a range of optimization methods for solving the convolutional sparse coding ob-jective, and the properties that make each method suit-able for different applications. In particular, we concen-trate on computational complexity, speed to conver-gence, memory usage, and the effect of implied bound-ary conditions. We present a broad suite of examples covering different signal and application domains to il-lustrate the general applicability of convolutional sparse coding, and the efficacy of the available optimization methods