Course 2019-2020 a.y.


Department of Decision Sciences

Course taught in English
Go to class group/s: 31
CLEAM (6 credits - II sem. - OP  |  SECS-S/01) - CLEF (6 credits - II sem. - OP  |  SECS-S/01) - CLEACC (6 credits - II sem. - OP  |  SECS-S/01) - BESS-CLES (6 credits - II sem. - OP  |  SECS-S/01) - WBB (6 credits - II sem. - OP  |  SECS-S/01) - BIEF (6 credits - II sem. - OP  |  SECS-S/01) - BIEM (6 credits - II sem. - OP  |  SECS-S/01) - BIG (6 credits - II sem. - OP  |  SECS-S/01) - BEMACS (6 credits - II sem. - OP  |  SECS-S/01)
Course Director:

Classes: 31 (II sem.)

Suggested background knowledge

Solid knowledge of calculus and of basic probability theory (e.g. probability distributions and random variables). Also, some knowledge of basic programming tools (such as R) can be helpful.

Mission & Content Summary


Stochastic processes are the natural tool to model real-world phenomena involving randomness and uncertainty. They offer a power mathematical framework to analyze complex problems in a variety of applied areas, ranging from business and industry to economics, finance, social sciences, biology and computer science. Moreover, the use of stochastic processes (and related probabilistic techniques) to build advanced statistical models is central to the ongoing data science revolution. The aim of the course it to provide students with a basic understanding of the probabilistic models and techniques underlying the most widely used classes of stochastic processes. The main focus is on modeling aspects, which are completed by a description of some popular algorithms for simulation. Mathematical concepts are integrated with real-world applications and examples and illustrated through simulations. At the end of the course, students will have bridged the gap between their elementary probability skills and the knowledge required to understand and use basic models based on stochastic processes.


  • Conditional probabilities and conditional expectations.
  • Introduction to stochastic processes and Markov chains.
  • Discrete-time Markov chains: Chapman-Kolmogorov equation, Classification of states, Limiting properties, Applications (e.g. stochastic models, sequential testing, website ranking).
  • Introduction to Stochastic Simulation, Simulation techniques and Monte Carlo methods.
  • Markov Chain Monte Carlo algorithms, Computational applications.
  • Counting processes and the Poisson process, Continuous-time stochastic processes, Examples and modeling applications.

Intended Learning Outcomes (ILO)


At the end of the course student will be able to...
  • Understand the connection between simple random variables and more elaborate and realistic stochastic processes.
  • Formulate and analyse probabilistic models based on Markov chains and counting processes.
  • Understand the principles of stochastic simulation and Monte Carlo algorithms.


At the end of the course student will be able to...
  • Translate real-world situations involving randomness and uncertainty into probabilistic models.
  • Exploit the tools of probability theory to answer relevant questions, such as: what is the equilibrium distribution of a specific stochastic system? What is the probability of a certain outcome?
  • Use software to simulate the evolution of probabilistic systems. Design simple Monte Carlo algorithms for computational purposes.
  • Apply stochastic processes to model and solve real-world problems commonly occurring in diverse fields, such as business, industry, economics, finance and data science.

Teaching methods

  • Face-to-face lectures
  • Exercises (exercises, database, software etc.)


The main learning activity of this course consists in face-to-face lectures. Lectures integrate discussions of theoretical notions, such as mathematical definitions, with examples and in-class solution of exercises. The aim of the examples and exercises shown in class is two folded:

  • First to illustrate and visualize the theoretical notions with simple examples.
  • Secondly to describe real-world applications (both modeling and computational) of the probabilistic tools.

In addition, the R statistical software is used in class to display computer simulations of the stochastic models under consideration. A set of exercises are provided for students to practice and get familiar with the lectures’ content.  

Assessment methods

  Continuous assessment Partial exams General exam
  • Written individual exam (traditional/online)


The assessment for all students, both attending and non-attending, consists in a written exam. The aim of the exam is to verify both the students’ understanding of the theoretical material (e.g. ability to provide definitions and characterizations of some noteworthy stochastic process), and their capacity to apply the theory for modeling or computational purposes (e.g. ability to identify appropriate models for a given situation and use them to answer specific questions).

Teaching materials


S.M. ROSS, Introduction to probability models, Academic Press, 2014.

Last change 01/06/2019 18:04