Foundations and Trends® in Signal Processing >
Vol 8 > Issue 1–2

Jonathan H. Manton and Pierre-Olivier Amblard (2015), "A Primer on Reproducing Kernel Hilbert Spaces", Foundations and Trends® in Signal Processing: Vol. 8: No. 1–2, pp 1-126. http://dx.doi.org/10.1561/2000000050

© 2015 J. H. Manton and P.-O. Amblard

Adaptive control and signal processing, Kernel methods, Classification and prediction, Adaptive signal processing, Statistical signal processing, Filtering, Estimation, Identification

Reproducing Kernel Hilbert Spaces

Download article
**In this article:**

1. Introduction

2. Finite-dimensional RKHSs

3. Function Spaces

4. Infinite-dimensional RKHSs

5. Geometry by Design

6. Applications to Linear Equations and Optimisation

7. Applications to Stochastic Processes

8. Embeddings of Random Realisations

9. Applications of Embeddings

References

Reproducing kernel Hilbert spaces are elucidated without assuming prior familiarity with Hilbert spaces. Compared with extant pedagogic material, greater care is placed on motivating the definition of reproducing kernel Hilbert spaces and explaining when and why these spaces are efficacious. The novel viewpoint is that reproducing kernel Hilbert space theory studies extrinsic geometry, associating with each geometric configuration a canonical overdetermined coordinate system. This coordinate system varies continuously with changing geometric configurations, making it well-suited for studying problems whose solutions also vary continuously with changing geometry. This primer can also serve as an introduction to infinite-dimensional linear algebra because reproducing kernel Hilbert spaces have more properties in common with Euclidean spaces than do more general Hilbert spaces.

1. Introduction

2. Finite-dimensional RKHSs

3. Function Spaces

4. Infinite-dimensional RKHSs

5. Geometry by Design

6. Applications to Linear Equations and Optimisation

7. Applications to Stochastic Processes

8. Embeddings of Random Realisations

9. Applications of Embeddings

References

Hilbert space theory is an invaluable mathematical tool in numerous signal processing and systems theory applications. Hilbert spaces satisfying certain additional properties are known as Reproducing Kernel Hilbert Spaces (RKHSs).

This primer gives a gentle and novel introduction to RKHS theory. It also presents several classical applications. It concludes by focusing on recent developments in the machine learning literature concerning embeddings of random variables. Parenthetical remarks are used to provide greater technical detail, which some readers may welcome, but they may be ignored without compromising the cohesion of the primer. Proofs are there for those wishing to gain experience at working with RKHSs; simple proofs are preferred to short, clever, but otherwise uninformative proofs. Italicised comments appearing in proofs provide intuition or orientation or both.

*A Primer on Reproducing Kernel Hilbert Spaces* empowers readers to recognize when and how RKHS theory can profit them in their own work.