Course 2024-2025 a.y.

Part 1: Theory

• Mathematical background: multivariable calculus and linear algebra.

• Convex sets: affine and convex sets, operations that preserve convexity, separating and supporting hyperplanes.

• Convex functions: basic properties and examples, operations that preserve convexity, the conjugate function, quasiconvex functions, log-concave and log-convex functions.

• Convex optimization problems: optimization problems, convex optimization, linear optimization problems, quadratic optimization problems.

• Duality: the Lagrange dual function, the Lagrange dual problem, geometric interpretation, optimality conditions, perturbation and sensitivity analysis, examples, theorems of alternatives.
   

Part 2: Algorithms

• Unconstrained minimization: unconstrained minimization problems, descent methods, gradient descent method, steepest descent method, Newton’smethod.

• Equality constrained minimization: equality constrained minimization problems, Newton’s method with equality constraints.

• Stochastic gradient methods: noisy gradients, incremental gradient method, classification and the perceptron, empirical risk minimization, the randomized Kaczmarz method, convergence analysis.

• Carry out a formal mathematical proof.

• Master the theory of convex sets and functions.

• Formulate convex optimization problems.

• Master duality theory.

• Develop and analyze efficient algorithms for solving both unconstrained and constrained optimization problems.

• Model challenging applications from data science and machine learning as mathematical optimization problems.

• Interpret the results by performing sensitivity analysis in an applied context.

• Identify the state of the art of optimization algorithms for different types of optimization problems.

• Solve convex optimization problems.

• Lectures
• Practical Exercises

Some exercises will be solved during the lectures, with the aim of applying the main theoretical results to different problems.

The written individual exam aims at evaluating:

• the knowledge of the fundamental mathematical notions and the ability to apply these notions to the solution of simple problems and exercises;
• the ability to articulate the knowledge of mathematical notions in a conceptually and formally correct way, adequately using definitions, theorems and proofs;
• the ability to actively search for deductive ideas that are fit to prove possible links between the properties of mathematical objects;
• the ability to apply mathematical notions to the solution of more complex problems and exercises.

• Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2004. Available online at: http://www.stanford.edu/~boyd/cvxbook/

• Benjamin Recht and Stephen J. Wright, Optimization for Modern Data Analysis, 2019. Preprint available online at: https://people.eecs.berkeley.edu/~brecht/opt4ml_book/