Sparse basis selection, ica, and majorization: towards a unified perspective

Sparse solutions to the linear inverse problem Ax = y and the determination of an environmentally adapted overcomplete dictio-nary (the columns of A) depend upon the choice of a “regulariz-ing function ” d(x) in several recently proposed procedures. We discuss the interpretation of d(x) within a Bayesian framework, and the desirable properties that “good ” (i.e., sparsity ensuring) regularizing functions, d(x) might have. These properties are: Schur-concavity (d(x) is consistent with majorization); concav-ity (d(x) has sparse minima); parameterizability (d(x) is drawn from a large, parameterizable class); and factorizability of the gra-dient of d(x) in a certain manner. The last property (which nat-urally leads one to consider separable regularizing functions) al-lows d(x) to be efficiently minimized subject to Ax = y using an Affine Scaling Transformation (AST)-like algorithm “adapted ” t