CS6230:
Optimization Methods in Machine Learning
Fall 2016
Instructor: Vineeth N
Balasubramanian
Class Schedule: Mon 4:00 – 5:30
pm, Thu 2:30 – 4:00 pm
Class Location: 212
(BlockA)
TA: Adepu
Ravi Sankar
OBJECTIVE: To understand
the various optimization methods that underlie machine learning methods that
have become so popular today in realworld applications. This course will
provide an introduction to these methods, discussions on their uses, as well as
provide opportunities to improve upon them as part of course projects. (This will be an interactiondriven class
focusing on learning deeper insights on optimization in machine learning.
Please expect the course to have mathematical rigor.)
TOPICS: Introduction to Optimization, Convex Sets, Convex
Functions, Lagrange Duality, Convex Optimization Algorithms, Secondorder cone
models, Semidefinite programming, Semiinfinite programming, Minimax,
Sublinear algorithms, Interior Point Methods, Active set, Stochastic gradient,
Coordinate descent, Cutting planes method, Applications to
Image/Video/Multimedia Processing
ELIGIBILITY:
Students who have completed a basic machine learning course, and are interested
in exploring its mathematical foundations. Instructor preapproval is necessary
to register for the course.
COURSE LECTURES:
4^{th} Aug 2016: Course Introduction (PDF)
REFERENCES:
1.
Sra, Suvrit, Sebastian Nowozin, and Stephen J. Wright,
eds. Optimization
for machine learning. Mit Press, 2012. (ISBN: 9780262016469):
2.
Roberto Battiti, Mauro Brunato. The LION Way: Machine Learning plus
Intelligent Optimization. Lionsolver, Inc. 2013.
3.
Bubeck, Sebastien. "Theory of Convex
Optimization for Machine Learning." arXiv preprint arXiv:1405.4980,
2014.
OTHER USEFUL
RESOURCES:

http://simons.berkeley.edu/talks/peterrichtarik20131023

Introduction
to Convex Optimization in Machine Learning

Kristin Bennett, Emilio ParradoHernandez. Interplay
of Optimization and Machine Learning Research, Journal of Machine Learning
Research, 2006.

Nati Srebro, Ambuj Tewari. Stochastic
Optimization for Machine Learning, Tutorial at International Conference on
Machine Learning, 2010.

Stephen Wright. Optimization
Methods in Machine Learning, Tutorial at Neural Information Processing
Systems, 2010.

Clarkson, Kenneth L., Elad Hazan, and David P.
Woodruff. Sublinear
optimization for machine learning. Journal of the ACM (JACM) 59.5 (2012):
23. ()

Miclet, Laurent, and Antoine Cornuejols. "What
is the place of Machine Learning between Pattern Recognition and Optimization?."
2008. ()

Submodularity in machine learning: http://submodularity.org/.

Parallel
Coordinate Descent Methods for Big Data Optimization