Foundations and Trends® in Signal Processing > Vol 6 > Issue 1

Bivariate Markov Processes and Their Estimation

By Yariv Ephraim, George Mason University, USA, yephraim@gmu.edu | Brian L. Mark, George Mason University, USA, bmark@gmu.edu

 
Suggested Citation
Yariv Ephraim and Brian L. Mark (2013), "Bivariate Markov Processes and Their Estimation", Foundations and TrendsĀ® in Signal Processing: Vol. 6: No. 1, pp 1-95. http://dx.doi.org/10.1561/2000000043

Publication Date: 26 Jun 2013
© 2013 Y. Ephraim and B. L. Mark
 
Subjects
Statistical signal processing
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction 
2. Preliminaries 
3. Likelihood Function of Observable Process 
4. Recursive Non-Causal Estimation 
5. Recursive Causal Estimation 
6. Maximum Likelihood Parameter Estimation 
7. Recursive Parameter Estimation 
8. Discrete-Time Bivariate Markov Chains 
9. Hidden Bivariate Markov Chains 
10. Underlying Diffusion Processes 
11. Selected Applications 
12. Concluding Remarks 
Acknowledgments 
References 

Abstract

A bivariate Markov process comprises a pair of random processes which are jointly Markov. One of the two processes in that pair is observable while the other plays the role of an underlying process. We are interested in three classes of bivariate Markov processes. In the first and major class of interest, the underlying and observable processes are continuous-time with finite alphabet; in the second class, they are discrete-time with finite alphabet; and in the third class, the underlying process is continuous-time with uncountably infinite alphabet, and the observable process is continuous-time with countably or uncountably infinite alphabet. We refer to processes in the first two classes as bivariate Markov chains. Important examples of continuoustime bivariate Markov chains include the Markov modulated Poisson process, and the batch Markovian arrival process. A hidden Markov model with finite alphabet is an example of a discrete-time bivariate Markov chain. In the third class we have diffusion processes observed in Brownian motion, and diffusion processes modulating the rate of a Poisson process. Bivariate Markov processes play central roles in the theory and applications of estimation, control, queuing, biomedical engineering, and reliability. We review properties of bivariate Markov processes, recursive estimation of their statistics, and recursive and iterative parameter estimation.

DOI:10.1561/2000000043
ISBN: 978-1-60198-674-0
100 pp. $75.00
Buy book (pb)
 
ISBN: 978-1-60198-675-7
100 pp. $115.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. Preliminaries
3. Likelihood Function of Observable Process
4. Recursive Non-Causal Estimation
5. Recursive Causal Estimation
6. Maximum Likelihood Parameter Estimation
7. Recursive Parameter Estimation
8. Discrete-Time Bivariate Markov Chains
9. Hidden Bivariate Markov Chains
10. Underlying Diffusion Processes
11. Selected Applications
12. Concluding Remarks
Acknowledgments
References

Bivariate Markov Processes and Their Estimation

Bivariate Markov processes play a central role in the theory and applications of estimation, control, queuing, biomedical engineering, and reliability. Bivariate Markov Processes and Their Estimation presents some of the fundamentals of the theory of bivariate Markov processes, and reviews the various parameters and signal estimation approaches that are associated with these Markov processes. It reviews both causal and non-causal estimation of some statistics of the bivariate Markov processes. In addition, it covers off-line as well as on-line recursive parameter estimation approaches.

Bivariate Markov Processes and Their Estimation is an ideal springboard for researchers and students who are interested in pursuing the study of this interesting family of processes. While proofs are generally omitted, an interested reader should be able to implement the estimation algorithms for bivariate Markov chains directly from the text. The material should be accessible to the signal processing community, although it requires some familiarity with Markov chains and the intricacies of the theory of hidden Markov models.

 
SIG-043