Course 2008-2009 a.y.

8385 - APPLIED PROBABILITY MODELS AND MARKOV PROCESSES


MM-LS - AFC-LS - CLAPI-LS - CLEFIN-LS - CLELI-LS - DES-LS - CLG-LS - M-LS - IM-LS - ACME-LS - EMIT-LS

Department of Decision Sciences

Course taught in English

Go to class group/s: 31
MM-LS (6 credits - I sem. - AI) - AFC-LS (6 credits - I sem. - AI) - CLAPI-LS (6 credits - I sem. - AI) - CLEFIN-LS (6 credits - I sem. - AI) - CLELI-LS (6 credits - I sem. - AI) - DES-LS (6 credits - I sem. - AI) - CLG-LS (6 credits - I sem. - AI) - M-LS (6 credits - I sem. - AI) - IM-LS (6 credits - I sem. - AI) - ACME-LS (6 credits - I sem. - AI) - EMIT-LS (6 credits - I sem. - AI)
Course Director:
PIETRO MULIERE

Classes: 31 (I sem.)
Instructors:
Class 31: PIETRO MULIERE


Course Objectives

Markov chains are the simplest mathematical models for random phenomenon evolving in time. Their simple structure makes it possible to say a great deal about their behaviour. At the same time, the class of Markov chains is rich enough to serve in many applications. This makes Markov chains the first and most important examples of random processes.
Indeed, the whole of the mathematical study of random processes can be regarded as a generalization in one way or another of the theory of Markov chains.
The course is an account of the elementary theory of Markov chains, with applications.


Course Content Summary

  • Probability and Measure
    Probability spaces and Expectations; Monotone convergence and Fubini's theorem
  • Discrete-time Markov chains
    Definitions and basic properties; Class structure; Hitting time and absorption probabilities; Strong Markov property; Recurrence and transience; Recurrence and transience of random walks; Invariant distributions; Convergence to equilibrium; Time reversal; Ergodic theorem
  • Continuous-time Markov chains
    Q-matrices and their exponential; Continuous-time random processes; Some properties of the exponential distribution; Poisson processes; Jump chain in and holding times; Explosion; Forward and backward equations; Non-minimal chains; Continuous-time properties
  • Further Theory
  • Martingales; Brownian motion
  • Applications
    Markov chains in Finance; Markov chains in Biology; Applications in Genetics; Queues and queuing networks; Markov chains in resource management; Markov chains and decision processes; Markov chains Monte Carlo

Detailed Description of Assessment Methods

The  exam consists of an essay on a topic chosen by the student, that is discussed with the professor.


Textbooks

  • J.R. NORRIS, Markov Chains, Cambridge University Press, 1998
Exam textbooks & Online Articles (check availability at the Library)
Last change 03/04/2008 13:39