phil schniter: EM-turbo-GAMP matlab code

EM-turbo-GAMP matlab code

The following links point to matlab code for sparse reconstruction (i.e., compressive sensing) of a single measurement vector (SMV) or multiple measurement vectors (MMV). All of these techniques build on the Generalized Approximate Message Passing (GAMP) algorithm, developed by Sundeep Rangan for the case of known i.i.d signal and noise priors. The code below extends GAMP to the practical case of unknown non-i.i.d priors by automatically learning the signal and noise prior parameters (while simultaneously reconstructing the signal) using an Expectation Maximization (EM) approach, and by incorporating structured sparsity using the turbo-GAMP approach, where the hyperparameters behind the structured-sparsity are also learned using EM.

matlab code

EM-NN-GAMP: Very simple interface; based on i.i.d non-negative Bernoulli-Gaussian-mixture signal prior and noise prior that is either i.i.d zero-mean Gaussian or i.i.d Laplacian.
EM-GM-GAMP: Very simple interface; based on i.i.d Bernoulli-Gaussian-mixture signal prior and i.i.d zero-mean Gaussian noise prior.
EM-BG-GAMP: Very simple interface; based on simpler i.i.d Bernoulli-Gaussian signal prior and i.i.d zero-mean Gaussian noise prior.
EM-turbo-GAMP: Object-oriented programming environment that handles many forms of structured-sparse inference.

references

First papers:
- J. P. Vila and P. Schniter, ``An Empirical-Bayes Approach to Compressive Sensing via Approximate Message Passing [pdf],'' presented at the Duke Workshop on Sensing and Analysis of High-Dimensional Data (SAHD) (Durham, North Carolina), July 2011.
- P. Schniter, ``Turbo Reconstruction of Structured Sparse Signals [pdf with typos corrected],'' Proc. Conf. on Information Sciences and Systems, (Princeton, NJ), Mar. 2010. [slides]
Bilinear versions:
- J. T. Parker, P. Schniter, and V. Cevher, ``Bilinear Generalized Approximate Message Passing---Part I: Derivation [pdf] [arxiv].'' [Matlab] IEEE Transactions on Signal Processing, vol. 62, no. 22, pp. 5839-5853, Nov. 2014.
- J. T. Parker, P. Schniter, and V. Cevher, ``Bilinear Generalized Approximate Message Passing---Part II: Applications [pdf] [arxiv].'' [Matlab] IEEE Transactions on Signal Processing, vol. 62, no. 22, pp. 5854-5867, Nov. 2014.
Gaussian-mixture and non-negativity:
- J. Vila and P. Schniter, ``An Empirical-Bayes Approach to Recovering Linearly Constrained Non-Negative Sparse Signals [pdf] [arxiv],'' IEEE Transactions on Signal Processing, vol. 62, no. 18, pp. 4689-4703, Sep. 2014.
- J. P. Vila and P. Schniter, ``Expectation-Maximization Gaussian-Mixture Approximate Message Passing [pdf] [arxiv],'' IEEE Transactions on Signal Processing, vol. 61, no. 19, pp. 4658-4672, Oct. 2013.
Structured sparsity:
- J. Ziniel, S. Rangan, and P. Schniter, ``A Generalized Framework for Learning and Recovery of Structured Sparse Signals [pdf],'' Statistical Signal Processing Workshop, (Ann Arbor, MI), Aug. 2012.
- J. Ziniel and P. Schniter, ``Dynamic Compressive Sensing of Time-Varying Signals via Approximate Message Passing [pdf][arxiv].'' IEEE Transactions on Signal Processing, vol. 61, no. 21, pp. 5270-5284, Nov. 2013.
- J. Ziniel and P. Schniter, ``Efficient High-Dimensional Inference in the Multiple Measurement Vector Problem [pdf] [arxiv],'' IEEE Transactions on Signal Processing, vol. 61, no. 2, pp. 340-354, Jan. 2013.
Classification:
- J. Ziniel, P. Schniter, and P. Sederberg, ``Binary Linear Classification and Feature Selection via Generalized Approximate Message Passing [arxiv].'' to appear in IEEE Transactions on Signal Processing.
Bilinear, non-negative, Gaussian-mixture, and structured-sparse:
- J. Vila, P. Schniter, and J. Meola, ``Hyperspectral Unmixing via Turbo Bilinear Approximate Message Passing [arxiv].''

background literature

Generalized Approximate Message Passing (GAMP) is a powerful method to estimate an unknown i.i.d non-Gaussian vector from a (known) linear transformation of that vector observed through i.i.d probabilistic measurement channels. The algorithm was recently proposed by Sundeep Rangan, building on the Relaxed Belief Propagation (RBP) algorithm of Dongning Guo and Chih-Chun Wang, as well as on the AMP algorithm proposed by David Donoho, Arian Maleki, and Andrea Montanari and rigorously analyzed by Mohsen Bayati and Andrea Montanari.
- S. Rangan, ``Generalized Approximate Message Passing for Estimation with Random Linear Mixing [arxiv],'' posted on arXiv 25 Oct. 2010.
- M. Bayati and A. Montanari, ``The dynamics of message passing on dense graphs, with applications to compressed sensing [IEEExplore],'' IEEE Transactions on Information Theory, vol. 57, no. 2, pp. 764-785, Feb. 2011.
- D. Donoho, A. Maleki, and A. Montanari, ``Message Passing Algorithms for Compressed Sensing [arXiv],'' Proc. Natl. Acad. Sci., 2009.
- D. Guo and C.-C. Wang, ``Large sparse linear systems observed via arbitrary channels: A decoupling principle [IEEExplore],'' Proc. IEEE International Symposium on Information Theory, (Nice, France), June 2007.
turbo-(G)AMP is a framework that extends (G)AMP to problems with non-identical non-independent (e.g., "structured sparse") signals and non-identical non-independent probabilistic measurement channels.
- P. Schniter, ``Turbo-AMP: A Graphical-Models Approach to Compressive Inference [slides],'' presented at Stanford University (Dec. 2010) and MIT Lincoln Labs (June 2012).
- P. Schniter, ``Turbo Reconstruction of Structured Sparse Signals [pdf with typos corrected],'' Proc. Conf. on Information Sciences and Systems, (Princeton, NJ), Mar. 2010. [slides]
- S. Som, L. C. Potter, and P. Schniter, ``On Approximate Message Passing for Reconstruction of Non-Uniformly Sparse Signals [pdf],'' Proc. National Aerospace and Electronics Conf., (Dayton, OH), July 2010.

back to phil's home page...

(last updated )