CS6230: Optimization Methods in Machine Learning

Fall 2016

Instructor: Vineeth N Balasubramanian

Class Schedule: Mon 4:00 – 5:30 pm, Thu 2:30 – 4:00 pm

Class Location: 212 (Block-A)

TA: Adepu Ravi Sankar

OBJECTIVE: To understand the various optimization methods that underlie machine learning methods that have become so popular today in real-world applications. This course will provide an introduction to these methods, discussions on their uses, as well as provide opportunities to improve upon them as part of course projects. (This will be an interaction-driven class focusing on learning deeper insights on optimization in machine learning. Please expect the course to have mathematical rigor.)

TOPICS: Introduction to Optimization, Convex Sets, Convex Functions, Lagrange Duality, Convex Optimization Algorithms, Second-order cone models, Semi-definite programming, Semi-infinite programming, Minimax, Sublinear algorithms, Interior Point Methods, Active set, Stochastic gradient, Coordinate descent, Cutting planes method, Applications to Image/Video/Multimedia Processing

ELIGIBILITY: Students who have completed a basic machine learning course, and are interested in exploring its mathematical foundations. Instructor pre-approval is necessary to register for the course.

COURSE LECTURES:

4^th Aug 2016: Course Introduction (PDF)

REFERENCES:

1.     Sra, Suvrit, Sebastian Nowozin, and Stephen J. Wright, eds. Optimization for machine learning. Mit Press, 2012. (ISBN: 9780262016469): 

2.     Roberto Battiti, Mauro Brunato. The LION Way: Machine Learning plus Intelligent Optimization. Lionsolver, Inc. 2013.

3.     Bubeck, Sebastien. "Theory of Convex Optimization for Machine Learning." arXiv preprint arXiv:1405.4980, 2014.

OTHER USEFUL RESOURCES:

-       http://simons.berkeley.edu/talks/peter-richtarik-2013-10-23

-       Introduction to Convex Optimization in Machine Learning

-       Kristin Bennett, Emilio Parrado-Hernandez. Interplay of Optimization and Machine Learning Research, Journal of Machine Learning Research, 2006.

-       Nati Srebro, Ambuj Tewari. Stochastic Optimization for Machine Learning, Tutorial at International Conference on Machine Learning, 2010.

-       Stephen Wright. Optimization Methods in Machine Learning, Tutorial at Neural Information Processing Systems, 2010.

-       Clarkson, Kenneth L., Elad Hazan, and David P. Woodruff. Sublinear optimization for machine learning. Journal of the ACM (JACM) 59.5 (2012): 23. ()

-       Miclet, Laurent, and Antoine Cornuejols. "What is the place of Machine Learning between Pattern Recognition and Optimization?." 2008. ()

-       Submodularity in machine learning: http://submodularity.org/.

-       Parallel Coordinate Descent Methods for Big Data Optimization