Foundations and Trends® in Signal Processing > Vol 4 > Issue 3

Theory and Use of the EM Algorithm

By Maya R. Gupta, Department of Electrical Engineering, University of Washington, USA, gupta@ee.washington.edu | Yihua Chen, Department of Electrical Engineering, University of Washington, USA, yhchen@ee.washington.edu

 
Suggested Citation
Maya R. Gupta and Yihua Chen (2011), "Theory and Use of the EM Algorithm", Foundations and Trends® in Signal Processing: Vol. 4: No. 3, pp 223-296. http://dx.doi.org/10.1561/2000000034

Publication Date: 15 Apr 2011
© 2011 M. R. Gupta and Y. Chen
 
Subjects
Randomized algorithms in signal processing
 

Free Preview:

Download extract

Share

Download article
In this article:
1 The Expectation-Maximization Method 
2 Analysis of EM 
3 Learning Mixtures 
4 More EM Examples 
5 EM Variants 
6 Conclusions and Some Historical Notes 
Acknowledgments 
References 

Abstract

This introduction to the expectation–maximization (EM) algorithm provides an intuitive and mathematically rigorous understanding of EM. Two of the most popular applications of EM are described in detail: estimating Gaussian mixture models (GMMs), and estimating hidden Markov models (HMMs). EM solutions are also derived for learning an optimal mixture of fixed models, for estimating the parameters of a compound Dirichlet distribution, and for dis-entangling superimposed signals. Practical issues that arise in the use of EM are discussed, as well as variants of the algorithm that help deal with these challenges.

DOI:10.1561/2000000034
ISBN: 978-1-60198-430-2
80 pp. $65.00
Buy book (pb)
 
ISBN: 978-1-60198-431-9
80 pp. $100.00
Buy E-book (.pdf)
Table of contents:
1: The Expectation-Maximization Method
2: Analysis of EM
3: Learning Mixtures
4: More EM Examples
5: EM Variants
6: Conclusions and Some Historical Notes
Acknowledgements
References

Theory and Use of the EM Algorithm

Theory and Use of the EM Algorithm introduces the expectation-maximization (EM) algorithm and provides an intuitive and mathematically rigorous understanding of this method. It describes in detail two of the most popular applications of EM: estimating Gaussian mixture models (GMMs), and estimating hidden Markov models (HMMs). It also covers the use of EM for learning an optimal mixture of fixed models, for estimating the parameters of a compound Dirichlet distribution, and for disentangling superimposed signals. It discusses problems that arise in practice with EM, and variants of the algorithm that help deal with these challenges. Theory and Use of the EM Algorithm is designed to be useful to both the EM novice and the experienced EM user looking to better understand the method and its use.

 
SIG-034