Facebook pixel
Info
Foto sezione
Logo Bocconi

Course 2023-2024 a.y.

20876 - DEEP LEARNING AND REINFORCEMENT LEARNING

AI
Department of Computing Sciences

Course taught in English

Go to class group/s: 29

AI (8 credits - II sem. - OB  |  INF/01)
Course Director:
ANDREA CELLI

Classes: 29 (II sem.)
Instructors:
Class 29: ANDREA CELLI


Synchronous Blended: Lessons in synchronous mode in the classroom (for a maximum of one hour per credit in remote mode)

Suggested background knowledge

For an effective learning experience, it is recommended previous programming experience (ideally in Python), good knowledge of calculus and linear algebra, probability and statistics.


Mission & Content Summary
MISSION

In this course, we will investigate the algorithmic principles and foundations of two particularly exciting subfields of Machine Learning: deep learning and reinforcement learning. The first part of the course focuses on deep learning, where we develop the conceptual tools to understand what happens when a neural net is trained, and to understand the latest architectural advancements that have proven remarkably effective in practical applications. The second part of the course focuses on sequential decision making, in which an agent interacts with an unknown environment to accomplish a goal. We will start from the multi-armed bandits setting and gradually increase the complexity of the scenarios up to general reinforcement learning problems.

CONTENT SUMMARY

- Part one: optimization for neural net training, overparametrization, attention and memory, GANs, representation learning

- Part two: multi-armed bandits problems, value-based methods, policy-gradient methods, RL with function approximators, Multi-agent RL


Intended Learning Outcomes (ILO)
KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
  • Define the key features of deep learning and reinforcement learning.
  • Identify strengths and limitations of deep learning and reinforcement learning algorithms.
  • Recognize the connections between optimization and Deep learning/RL.
  • Evaluate the main trade-offs in the choice of a technique for a particular problem.
APPLYING KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
  • Formulate and solve machine learning problems using tools from deep-learning and reinforcement-learning
  • Independently extend algorithms and theories discussed in class to new problems.
  • Understand and use new tools beyond what discussed in class by reading research papers.
     

Teaching methods
  • Face-to-face lectures
  • Exercises (exercises, database, software etc.)
  • Group assignments
DETAILS
  • Face-to-face lectures focus on the theoretical, methodological, and computational aspects of the topics covered by the course. 
  • In hands-on exercise sessions students will work on their laptops to implement solutions to relevant case studies. Students will use Python and common machine learning libraries. 
  • Group assignment: each student is required to participate in a group research-based project and provide a final written report.
     

Assessment methods
  Continuous assessment Partial exams General exam
  • Written individual exam (traditional/online)
  •     x
  • Group assignment (report, exercise, presentation, project work etc.)
  •     x
    ATTENDING AND NOT ATTENDING STUDENTS

    The exam consists of a theory part and a research-based project.

    • The theory part consists in exercises and questions to be answered on paper, and it is used to assess the "knowledge and understanding" learning objectives. This contributes to 60% of the final grade.
    • The project consists in a group work exploring a research-oriented problem related to the topics of the course. The project contributes to 40% of the final grade. The project is used to assess the "applying knowledge and understanding" learning objectives.
    • In order to pass the exam, students must achieve a passing grade in both the theory part and the project part.

    Teaching materials
    ATTENDING AND NOT ATTENDING STUDENTS
    • Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016
    • R.S. Sutton, Richard, A. G. Barto,  Reinforcement learning: An introduction, 2018.
    Last change 26/05/2023 14:44