20876 - DEEP LEARNING AND REINFORCEMENT LEARNING

For an effective learning experience, it is recommended previous programming experience (ideally in Python), good knowledge of calculus and linear algebra, probability and statistics.

In this course, we will investigate the algorithmic principles and foundations of two particularly exciting subfields of Machine Learning: deep learning and reinforcement learning. The first part of the course focuses on deep learning, where we develop the conceptual tools to understand what happens when a neural net is trained, and to understand the latest architectural advancements that have proven remarkably effective in practical applications. The second part of the course focuses on sequential decision making, in which an agent interacts with an unknown environment to accomplish a goal. We will start from the multi-armed bandits setting and gradually increase the complexity of the scenarios up to general reinforcement learning problems.

- Part one: optimization for neural net training, overparametrization, attention and memory, GANs, representation learning

- Part two: multi-armed bandits problems, value-based methods, policy-gradient methods, RL with function approximators, Multi-agent RL

• Define the key features of deep learning and reinforcement learning.
• Identify strengths and limitations of deep learning and reinforcement learning algorithms.
• Recognize the connections between optimization and Deep learning/RL.
• Evaluate the main trade-offs in the choice of a technique for a particular problem.

• Formulate and solve machine learning problems using tools from deep-learning and reinforcement-learning
• Independently extend algorithms and theories discussed in class to new problems.
• Understand and use new tools beyond what discussed in class by reading research papers.
   

• Face-to-face lectures
• Exercises (exercises, database, software etc.)
• Group assignments

• Face-to-face lectures focus on the theoretical, methodological, and computational aspects of the topics covered by the course. 
• In hands-on exercise sessions students will work on their laptops to implement solutions to relevant case studies. Students will use Python and common machine learning libraries. 
• Group assignment: each student is required to participate in a group research-based project and provide a final written report.
   

The exam consists of a theory part and a research-based project.

• The theory part consists in exercises and questions to be answered on paper, and it is used to assess the "knowledge and understanding" learning objectives. This contributes to 60% of the final grade.
• The project consists in a group work exploring a research-oriented problem related to the topics of the course. The project contributes to 40% of the final grade. The project is used to assess the "applying knowledge and understanding" learning objectives.
• In order to pass the exam, students must achieve a passing grade in both the theory part and the project part.

• Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016
• R.S. Sutton, Richard, A. G. Barto,  Reinforcement learning: An introduction, 2018.