8385 - APPLIED PROBABILITY MODELS AND MARKOV PROCESSES
MM-LS - AFC-LS - CLAPI-LS - CLEFIN-LS - CLELI-LS - DES-LS - CLG-LS - M-LS - IM-LS - ACME-LS - EMIT-LS
Course taught in English
Go to class group/s: 31
Markov chains are the simplest mathematical models for random phenomenon evolving in time. Their simple structure makes it possible to say a great deal about their behaviour. At the same time, the class of Markov chains is rich enough to serve in many applications. This makes Markov chains the first and most important examples of random processes.
Indeed, the whole of the mathematical study of random processes can be regarded as a generalization in one way or another of the theory of Markov chains.
The course is an account of the elementary theory of Markov chains, with applications.
Probability and Measure
Probability spaces and Expectations; Monotone convergence and Fubini's theorem
Discrete-time Markov chains
Definitions and basic properties; Class structure; Hitting time and absorption probabilities; Strong Markov property; Recurrence and transience; Recurrence and transience of random walks; Invariant distributions; Convergence to equilibrium; Time reversal; Ergodic theorem
Continuous-time Markov chains
Q-matrices and their exponential; Continuous-time random processes; Some properties of the exponential distribution; Poisson processes; Jump chain in and holding times; Explosion; Forward and backward equations; Non-minimal chains; Continuous-time properties
Martingales; Brownian motion
Markov chains in Finance; Markov chains in Biology; Applications in Genetics; Queues and queuing networks; Markov chains in resource management; Markov chains and decision processes; Markov chains Monte Carlo
The exam consists of an essay on a topic chosen by the student, that is discussed with the professor.
- J.R. NORRIS, Markov Chains, Cambridge University Press, 1998