FOCUSS-based dictionary learning algorithms

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maxi-mum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log-priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as ‘concepts, ’ ‘features ’ or ‘words ’ capable of succinct expression of events encountered in the environ-ment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial ‘25 words or less’), but not necessarily as succinct as one entry. To learn an environmentally-adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS, an affine scaling transformation (AST)-like sparse signal representation algorithm recently developed at UCSD, and an update of the dictionary using these sparse representations.