Course 2005-2006 a.y.

8230 - QUANTITATIVE METHODS FOR SOCIAL SCIENCES


GM-LS - MM-LS - OSI-LS - AFC-LS - CLAPI-LS - CLEFIN-LS - CLELI-LS - CLEACC-LS - DES-LS - CLEMIT-LS - CLG-LS

Department of Decision Sciences

Course taught in English

Go to class group/s: 31
GM-LS (6 credits - I sem. - AI) - MM-LS (6 credits - I sem. - AI) - OSI-LS (6 credits - I sem. - AI) - AFC-LS (6 credits - I sem. - AI) - CLAPI-LS (6 credits - I sem. - AI) - CLEFIN-LS (6 credits - I sem. - AI) - CLELI-LS (6 credits - I sem. - AI) - CLEACC-LS (6 credits - I sem. - AI) - DES-LS (6 credits - I sem. - AI) - CLEMIT-LS (6 credits - I sem. - AI) - CLG-LS (6 credits - I sem. - AI)
Course Director:
PIETRO MULIERE

Classes: 31 (I sem.)
Instructors:
Class 31: PIETRO MULIERE


Course Objectives

Markov chains are the simplest mathematical models for random phenomenon evolving in time. Their simple structure makes it possible to say a great deal about their behaviour. At the same time, the class of Markov chains is rich enough to serve in many applications. This makes Markov chains the first and most important examples of random processes.
Indeed, the whole of the mathematical study of random processes can be regarded as a generalization in one way or another of the theory of Markov chains.
The course is an account of the elementary theory of Markov chains, with applications.


Course Content Summary

  • Probability and Measure
    Probability spaces and Expectations; Monotone convergence and Fubini's theorem
  • Discrete-time Markov chains
    Definitions and basic properties; Class structure; Hitting time and absorption probabilities; Strong Markov property; Recurrence and transience; Recurrence and transience of random walks; Invariant distributions; Convergence to equilibrium; Time reversal; Ergodic theorem
  • Continuous-time Markov chains
    Q-matrices and their exponential; Continuous-time random processes; Some properties of the exponential distribution; Poisson processes; Jump chain in and holding times; Explosion; Forward and backward equations; Non-minimal chains; Continuous-time properties
  • Further Theory
  • Martingales; Brownian motion
  • Applications
    Markov chains in Finance; Markov chains in Biology; Applications in Genetics; Queues and queuing networks; Markov chains in resource management; Markov chains  decision processes; Markov chains Monte Carlo

Detailed Description of Assessment Methods

The  exam will consist of an essay on a topic chosen by the student, that will be discussed with the professor.


Textbooks

  • D.R. COX, H.D. MILLER, The Theory of Stochastic Processes, Chapman and Hall, 1980.
  • E. CINLAR, Introduction to Stochastic Processes, Englewood Cliffs, Prentice Hall, 1975.
  • M. JOSIFESCU, Finite Markov Processes and their Applications, J. Wiley, 1980.
  • S. KARLIN, H.M. TAYLOR, A First course in Stochastic processes, Academic Press, 1975.
Exam textbooks & Online Articles (check availability at the Library)
Last change 28/04/2005 00:00