1Optimality of Operator-Like Wavelets for Representing Sparse AR(1) Processes

Abstract — It is known that the Karhunen-Loève transform (KLT) of Gaussian first-order auto-regressive (AR(1)) processes results in sinusoidal basis functions. The same sinusoidal bases come out of the independent-component analysis (ICA) and actually correspond to processes with completely independent samples. In this paper, we relax the Gaussian hypothesis and study how orthogonal transforms decouple symmetric-alpha-stable (SαS) AR(1) processes. The Gaussian case is not sparse and corresponds to α = 2, while 0 < α < 2 yields processes with sparse linear-prediction error. In the presence of sparsity, we show that operator-like wavelet bases do outperform the sinusoidal ones. Also, we observe that, for processes with very sparse increments (0 < α ≤ 1), the operator-like wavelet basis is indistinguishable from the ICA solution obtained through nu-merical optimization. We consider two criteria for independence. The first is the Kullback-Leibler divergence between the joint probability density function (pdf) of the original signal and the product of the marginals in the transformed domain. The second is a divergence between the joint pdf of the original signal and the product of the marginals in the transformed domain, which is based on Stein’s formula for the mean-square estimation error in additive Gaussian noise. Our framework then offers a unified view that encompasses the discrete cosine transform (known to be asymptotically optimal for α = 2) and Haar-like wavelets (for which we achieve optimality for 0 < α ≤ 1). Index Terms — Operator-like wavelets, independent-component analysis, auto-regressive processes, stable distributions