20876 - DEEP LEARNING AND REINFORCEMENT LEARNING
Department of Computing Sciences
Course taught in English
Go to class group/s: 29
Course Director:
ANDREA CELLI
ANDREA CELLI
Suggested background knowledge
For an effective learning experience, it is recommended previous programming experience (ideally in Python), good knowledge of calculus and linear algebra, probability and statistics.
Mission & Content Summary
MISSION
In this course, we will investigate the algorithmic principles and foundations of two particularly exciting subfields of Machine Learning: deep learning and reinforcement learning. The first part of the course focuses on deep learning, where we develop the conceptual tools to understand what happens when a neural net is trained, and to understand the latest architectural advancements that have proven remarkably effective in practical applications. The second part of the course focuses on sequential decision making, in which an agent interacts with an unknown environment to accomplish a goal. We will start from the multi-armed bandits setting and gradually increase the complexity of the scenarios up to general reinforcement learning problems.
CONTENT SUMMARY
- Part one: optimization for neural net training, overparametrization, attention and memory, GANs, representation learning
- Part two: multi-armed bandits problems, value-based methods, policy-gradient methods, RL with function approximators, Multi-agent RL
Intended Learning Outcomes (ILO)
KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
- Define the key features of deep learning and reinforcement learning.
- Identify strengths and limitations of deep learning and reinforcement learning algorithms.
- Recognize the connections between optimization and Deep learning/RL.
- Evaluate the main trade-offs in the choice of a technique for a particular problem.
APPLYING KNOWLEDGE AND UNDERSTANDING
At the end of the course student will be able to...
- Formulate and solve machine learning problems using tools from deep-learning and reinforcement-learning
- Independently extend algorithms and theories discussed in class to new problems.
- Understand and use new tools beyond what discussed in class by reading research papers.
Teaching methods
- Lectures
- Practical Exercises
- Collaborative Works / Assignments
DETAILS
- Face-to-face lectures focus on the theoretical, methodological, and computational aspects of the topics covered by the course.
- In hands-on exercise sessions students will work on their laptops to implement solutions to relevant case studies. Students will use Python and common machine learning libraries.
- Group assignment: each student is required to participate in a group research-based project and provide a final written report.
Assessment methods
| Continuous assessment | Partial exams | General exam | |
|---|---|---|---|
|
x | x |
ATTENDING AND NOT ATTENDING STUDENTS
The exam consists of a theory part and a research-based project.
- The theory part consists in exercises and questions to be answered on paper, and it is used to assess the "knowledge and understanding" learning objectives. This contributes to 60% of the final grade.
- The project consists in a group work exploring a research-oriented problem related to the topics of the course. The project contributes to 40% of the final grade. The project is used to assess the "applying knowledge and understanding" learning objectives.
- In order to pass the exam, students must achieve a passing grade in both the theory part and the project part.
Teaching materials
ATTENDING AND NOT ATTENDING STUDENTS
- Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016
- R.S. Sutton, Richard, A. G. Barto, Reinforcement learning: An introduction, 2018.
Last change // :