Foundations and Trends® in Machine Learning > Vol 10 > Issue 1-2

Kernel Mean Embedding of Distributions: A Review and Beyond

Krikamol Muandet, Mahidol University, Thailand and Max Planck Institute for Intelligent Systems, Germany, krikamol.mua@mahidol.ac.th Kenji Fukumizu, Institute of Statistical Mathematics, Japan, fukumizu@ism.ac.jp Bharath Sriperumbudur, Pennsylvania State University, USA, bks18@psu.edu Bernhard Schölkopf, Max Planck Institute for Intelligent Systems, Germany, bs@tuebingen.mpg.de
 
Suggested Citation
Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur and Bernhard Schölkopf (2017), "Kernel Mean Embedding of Distributions: A Review and Beyond", Foundations and Trends® in Machine Learning: Vol. 10: No. 1-2, pp 1-141. http://dx.doi.org/10.1561/2200000060

Published: 28 Jun 2017
© 2017 K. Muandet, K. Fukumizu, B. Sriperumbudur and B. Schölkopf
 
Subjects
Kernel methods,  Nonparametric methods,  Statistical learning theory,  Statistical/machine learning,  Pattern recognition and learning,  Information theory and statistics,  Learning and statistical methods,  Filtering, Estimation, Identification
 

Free Preview:

Article Help

Share

Download article
In this article:
1. Introduction
2. Background
3. Hilbert Space Embedding of Marginal Distributions
4. Hilbert Space Embedding of Conditional Distributions
5. Relationships between KME and Other Methods
6. Future Directions
7. Conclusions
Acknowledgements
References

Abstract

A Hilbert space embedding of a distribution—in short, a kernel mean embedding—has recently emerged as a powerful tool for machine learning and statistical inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original “feature map” common to support vector machines (SVMs) and other kernel methods. In addition to the classical applications of kernel methods, the kernel mean embedding has found novel applications in fields ranging from probabilistic modeling to statistical inference, causal discovery, and deep learning. This survey aims to give a comprehensive review of existing work and recent advances in this research area, and to discuss challenging issues and open problems that could potentially lead to new research directions. The survey begins with a brief introduction to the RKHS and positive definite kernels which forms the backbone of this survey, followed by a thorough discussion of the Hilbert space embedding of marginal distributions, theoretical guarantees, and a review of its applications. The embedding of distributions enables us to apply RKHS methods to probability measures which prompts a wide range of applications such as kernel two-sample testing, independent testing, and learning on distributional data. Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications. The conditional mean embedding enables us to perform sum, product, and Bayes’ rules—which are ubiquitous in graphical model, probabilistic inference, and reinforcement learning— in a non-parametric way using this new representation of distributions. We then discuss relationships between this framework and other related areas. Lastly, we give some suggestions on future research directions.

DOI:10.1561/2200000060
ISBN: 978-1-68083-288-4
154 pp. $99.00
Buy book
 
ISBN: 978-1-68083-289-1
154 pp. $260.00
Buy E-book
Table of contents:
1. Introduction
2. Background
3. Hilbert Space Embedding of Marginal Distributions
4. Hilbert Space Embedding of Conditional Distributions
5. Relationships between KME and Other Methods
6. Future Directions
7. Conclusions
Acknowledgements
References

Kernel Mean Embedding of Distributions: A Review and Beyond

A Hilbert space embedding of a distribution—in short, a kernel mean embedding—has recently emerged as a powerful tool for machine learning and statistical inference. The basic idea behind this framework is to map distributions into a reproducing kernel Hilbert space (RKHS) in which the whole arsenal of kernel methods can be extended to probability measures. It can be viewed as a generalization of the original “feature map” common to support vector machines (SVMs) and other kernel methods. In addition to the classical applications of kernel methods, the kernel mean embedding has found novel applications in fields ranging from probabilistic modeling to statistical inference, causal discovery, and deep learning.

Kernel Mean Embedding of Distributions: A Review and Beyond provides a comprehensive review of existing work and recent advances in this research area, and to discuss some of the most challenging issues and open problems that could potentially lead to new research directions. The targeted audience includes graduate students and researchers in machine learning and statistics who are interested in the theory and applications of kernel mean embeddings.

 
MAL-060