Course Material for AI2100: Convex Optimization (Spring 2024)

Please Note that the course material is typically updated after each lecture. The topics listed for future dates are tentative and can vary.

Lecture Notes and Classroom Scribble

The classroom scribble and the lecture notes are intended for quick reference\review. Lecture notes and classroom scribble are by no means a complete source and infact are based on the suggested textbooks\references. It is highly recommended that the students also refer to the suggested textbooks\references.

1. (01 Jan 2024): Introduction

   1. Course overview

2. (02 Jan 2024): Single Variable Optimization [Lecture Notes]

   1. Overview of concepts in Single Variable Optimization

3. (05 Jan 2024): Algorithms for Single Variable Optimization [Scribble] [Lecture Notes]

   1. Overview of Search Space Reduction Techniques

   2. Introduction to Golden Section Search

4. (08 Jan 2024): Algorithms for Single Variable Optimization [Scribble] [Lecture Notes]

   1. Golden Section Search

5. (09 Jan 2024): Algorithms for Single Variable Optimization [Scribble] [Lecture Notes]

   1. The Bisection Search

6. (12 Jan 2024): Algorithms for Single Variable Optimization [Scribble] [Lecture Notes]

   1. Newton's Method for Unconstrained Single Variable Optimization

7. (16 Jan 2024): Algorithms for Single Variable Optimization [Scribble] [Lecture Notes]

   1. Secant Method for Unconstrained Single Variable Optimization

   2. Newton's method for solving Nonlinear Equations

8. (18 Jan 2024): Linear and Affine Combination [Scribble] [Lecture Notes]

   1. Linear and Affine Combination

   2. Subspace and Affine Hull

9. (22 Jan 2024): Affine and Convex Combination [Scribble] [Lecture Notes]

   1. Affine Sets and their Properties

   2. Convex Hull

10. (29 Jan 2024): Convex Sets and Convex Hull [Scribble] [Lecture Notes]

    1. Convex Sets and their properties

    2. Algorithms for finding the convex hull

11. (01 Feb 2024): Function of Multiple Variables [Scribble] [Lecture Notes]

    1. Function of Multiple Variables

    2. Linear and Affine Functions

12. (12 Feb 2024): Convex Functions [Scribble] [Lecture Notes]

    1. Convex Functions

    2. Review of Limit and Derivative of a function of single variable

13. (15 Feb 2024): Derivative of a function of single variable [Scribble] [Lecture Notes]

    1. Derivative as an existence of affine approximation

14. (12 Feb 2024): Derivative of a function of Multiple Variables [Scribble] [Lecture Notes]

    1. Derivative of a function of Multiple Variables

    2. Gradient, Hessian, and Jacobian

15. (19 Feb 2024): Directional Derivative and the Gradient [Scribble] [Lecture Notes]

    1. Approximations and the Chain Rule

    2. Directional Derivative and the Gradient

16. (20 Feb 2024): Conditions for Convexity [Scribble] [Lecture Notes]

    1. Feasible Directions

    2. First and Second Order Conditions for Convexity

17. (23 Feb 2024): Optimality Conditions [Scribble] [Lecture Notes]

    1. Taylor's Series Expansion

    2. First and Second Order Conditions for Optimality

18. (26 Feb 2024): Optimality Conditions [Scribble] [Lecture Notes]

    1. Second Order Sufficient Condition for Optimality

    2. General Framework of Optimization Algorithms

19. (27 Feb 2024): Overview of Linear Programming and Newton's Method [Scribble] [Lecture Notes]

    1. Overview of Linear Programming

    2. Newton's Method

20. (01 Mar 2024): Solving System of Nonlinear Equations [Scribble] [Lecture Notes]

    1. Quadratic Forms

    2. Newton's Method for Solving System of Nonlinear equations

21. (04 Mar 2024): State Estimation [Scribble] [Lecture Notes]

    1. Overview of State Estimation

    2. Gauss Newton Method

22. (05 Mar 2024): Descent based Approaches [Scribble] [Lecture Notes]

    1. Descent Direction

    2. Descent Direction based Approaches to Compute Optima

23. (08 Mar 2024): Gradient Descent Approach [Scribble] [Lecture Notes]

    1. Gradient Descent Approach

    2. An interesting Example

24. (11 Mar 2024): Steepest Descent Algorithm [Scribble] [Lecture Notes]

    1. Steepest Descent Approach

    2. Use of Steepest Descent to Solve System of Linear Equations

25. (13 Mar 2024): Other Descent based Approaches [Scribble] [Lecture Notes]

    1. Newton's Method as a Version of the Gradient Descent