This PhD level course will explore foundational and modern ideas for speeding up optimization, with a particular focus on machine learning applications. We will survey ideas from (and for) numerical linear algebra, continuous optimization, combinatorial optimization, zero-order optimization, automated machine learning, and deep learning. We will cover powerful techniques for speeding up computation, including ideas from (randomized) low rank approximation and preconditioning, stochastic and subsampling methods, automatic differentiation, lagrangian methods, hyperparameter selection methods, and more. A detailed list of topics may be found here.
For each topic, we will begin with a few foundational lectures, followed by student-guided discussion on modern research papers. The course will culminate in a final research project which will constitute a majority of the grade. Students will deliver short presentations on their projects in class in addition to written reports.
- Paper reviews should be submitted on this google form (same link for all papers)
- Slack for general questions and comments.
- Office hours are on this calendar. You can also talk with Prof. Udell after class or reach out by email at udell@cornell.edu to set up an appointment.
Students are required to attend class, with at most two absences. (Any student may attend up to four classes may also be attended remotely via Zoom, or more by prior arrangement with the instructor. Students at Cornell Tech will attend all classes on Zoom.)
Course grades have four components:
- Reviews (.3): Students will write reviews of around twelve papers that we read for class and upload them to CMT.
- Quiz (.1): Most classes will begin with a two-question quiz. The questions will be easy to answer if you read the paper. Grading on this portion of the course will be nonlinear (eg, 0 points if too few quiz questions are answered correctly).
- Present a paper (.3): Students will each lead the discussion twice (or so), possibly in teams depending on course enrollment. The student leading the discussion will prepare a 30 minute presentation using slides, two true/false or multiple choice questions on the paper, and a class activity or questions for discussion. The presentations will be graded by peer review.
- Research project (.3): The final research project should explore a new idea for speeding up optimization. Projects can be in teams of up to three students except by special advance permission from the instructor. Students will prepare an initial project proposal, midterm report, and final report on the project, and will make an in-class presentation. Research projects can (and should!) advance your PhD research.
This google doc serves as our schedule. Sign up for presentation slots on the google doc by adding your name and a link to the paper you'll present. (Make sure not to choose a paper someone else has already picked!) We may spend more or less time on a topic depending on student interest.
talks include
- quiz
- discussion questions
before the talk
- submit a PR to the class git repo with your slides
bring
- power
- dongle (for HDMI or VGA)
setup
- projector
- camera
- zoom
- share screen