Course Overview
In this course we will introduce classical methods in
pattern classification and machine learning, focusing on statistical
learning approaches for supervised and unsupervised learning problems.
We will present the theoretical and algorithmic underpinnings of these
methods. The course will consist of biweekly lectures, 6 problem sets
that contain both mathemetical and MATLAB/Octave programming exercises, one
in-class exam, and a final project.
Prerequisites
Undergraduate level training or coursework in algorithms, linear
algebra, calculus and multivariate calculus, basic probability and
statistics; an undergraduate level course in Artificial Intelligence
may be helpful but is not required. A background in programming will
also be necessary for the problem sets; specifically students are
expected to be familiar with MATLAB/Octave or learn it during the course.
Piazza Forum
We will use Piazza for class discussions. Please go to this
Piazza website to join the course forum (note: you must use a
ucla.edu email account to join the forum). We strongly encourage
students to post on this forum rather than emailing the course staff
directly (this will be more efficient for both students and staff).
Students should use Piazza to:
- Ask clarifying questions about the course material.
- Share useful resources with classmates (so long as they do not
contain homework solutions).
- Look for project partners or other students to form study groups.
- Answer questions posted by other students to solidify your own
understanding of the material.
The course Academic Integrity Policy must be followed on the message
boards at all times. Do not post or request homework solutions! Also,
please be polite.
Staff Contact Info
Instructor: Prof. Ameet Talwalkar
Office Hours: Boelter 4531F, Tuesday 10:00a - 11:00a
Email: ameet at cs.ucla.edu
Teaching Assistants:
- Nikolaos
Karianakis
Office Hours: Boelter 2432, Monday 3:30p - 4:30p; Wednesday 2:00p - 3:00p
Email: nikos.karianakis at gmail
- Amogh Param
Office Hours: Boelter 2432, Monday 11:30a - 12:30p; Friday 2:30p - 3:30p
Email: aparam at cs.ucla.edu
Textbooks
There will be no required textbooks, though we suggest the
following to help you to study:
- Machine Learning: A Probabilistic Perspective by Kevin Murphy.
- Elements of Statistical Learning by Trevor Hastie, Robert
Tibshirani and Jerome Friedman (freely available online).
We will provide suggested readings from these books in the schedule
below, using the acronyms MLAPA and ESL.
Academic Integrity Policy
Group studying and collaborating on problem sets are encouraged, as
working together is a great way to understand new material. Students
are free to discuss the homework problems with anyone under the
following conditions:
- Students must write their own solutions and understand the
solutions that they wrote down.
- Students must list the names of their collaborators (i.e., anyone
with whom the assignment was discussed).
- Students may not use old solution sets from this class or any
other class under any circumstances, unless the instructor grants
special permission.
These policies are described in more detail in the
Academic
Integrity Agreement, which all students must sign and turn in.
Students are also encouraged to read the Dean of Students'
guide to Academic Integrity.
Grading Policy
Grades will be based on the following components:
- Problem Sets (50%): There will be 6 problem sets. Each
each problem set having equal weight.
- Problem sets are due at the beginning of class on the due
date. Late submissions will not be accepted.
- Students can drop their lowest grade (i.e., only the top 5
grades will count).
- All solutions must be typed (preferably using LaTeX) and
printed out. Solutions that are not typed will be penalized 25%;
unreadable answers will not be graded.
- Solutions will be graded on both correctness and clarity. If
you cannot solve a problem completely, you will get more partial
credit by identifying the gaps in your argument than by
attempting to cover them up.
- For some assignments, it might be the case that only a subset of
the problems will be graded in detail. Students may or may not be
told which problems will be graded in advance.
- Midterm (20%): This in-class exam will cover material
from the lectures and the problem sets. In addition to accounting
for 20% of the final course grade, a midterm score below 50 may
subject you to a failing grade regardless of performance on other
graded materials.
- Final Project (30%): Projects consist of a proposal, a written report
and a poster presentation. All projects must be completed in groups
of 1-2 students. 25% of your project grade will be based on your
proposal, 25% on your poster presentation, and 50% on your final
report.
- Academic Integrity Form: Students will not earn points
on any coursework unless they sign and turn in the Academic
Integrity Agreement. This form must be turned in with the first
problem set.
Regrade requests for homework or the midterm must be made within one week after
the graded assignments have been handed out, regardless of whether you attended
class that day. Please put your request in writing, indicating the parts you
want regraded and why, and hand in your request to the TA in person. Please
keep in mind the unlikely but real possibility that a regrade may lower your
grade, in case a previously overlooked mistake is discovered.
Using MATLAB / Octave
All registered students for this course can obtain MATLAB access via
their SEASnet accounts. This applies to both Engineering and
non-Engineering students), though students who are not officially
enrolled (per the Registrar's Office) will not be able to get a SEAS
account. SEASnet accounts can be applied for by going to this link, then clicking on the "Create Account" tab
in the upper left hand corner. Student accounts must be picked up in
person at the SEASnet Help Desk. Remote Desktop is also available to
all enrolled students in Engineering. If you need assistance
please contact help@seas.ucla.edu.
Octave is freely available.
Using LaTeX
Students are strongly encouraged to use LaTeX. LaTeX makes it
simple to typeset mathematical equations, and is extremely useful for
grad students to know. Most of the academic papers you read were
written with LaTeX, and probably most of the textbooks too. Here is an excellent LaTeX tutorial and here are
instructions for installing LaTeX on your machine. Also note
that LaTeX is also installed on department-run linux machines.
Acknowledgments
This course is based on material developed by Fei Sha.
Some of the administrative content on the course website is adapted
from material from Jenn
Wortman Vaughan, Rich Korf, and Alexander Sherstov.
Tentative Schedule
Date |
Topics |
Reading |
HW |
9/24 |
Course Overview, Math Review
|
[MLAPA] 1.1-1.3, 2; see other references in Math Review slides |
|
9/29 |
Nearest Neighbors
|
[MLAPA] 1.4.1-1.4.3; [ESL] 13.3 |
HW1 released
(supplementary files) |
10/1 |
Decision Trees
|
[MLAPA] 16.2; [ESL] 9.2 |
|
10/6 |
Naive Bayes
|
[MLAPA] 3.5; [ESL] 6.6.3 |
HW1 and Integrity Form due;
HW2 released (supplementary file) |
10/8 |
Logistic Regression
|
[MLAPA] 1.4.6, 8.1-8.4; [ESL] 4.1, 4.2, 4.4 |
|
10/13 |
Gaussian and Linear
Discriminant Analysis, Multiclass Classification
|
[MLAPA] 4.2.1-4.2.5; [ESL] 4.3 |
|
10/15 |
Perceptron, Linear Regression
|
[MLAPA] 8.5.1-8.5.4, 1.4.5, 7.1-7.3, 7.5.1, 7.5.2, 7.5.4,
7.6; [ESL] 3.1-3.2 |
HW2 due;
HW3 released (supplementary file);
HW4 released |
10/20 |
Linear Regression, Nonlinear Basis Functions |
[MLAPA] 1.4.7, 1.4.8; [ESL] 7.1-7.3, 7.10
Linear Algebra Review |
Project Proposal Guideline released |
10/22 |
No Class
|
No new reading |
|
10/27 |
Overfitting / Regularization |
No new reading |
|
10/29 |
Bias-Variance tradeoff, Kernel Methods |
[MLAPA] 14.1, 14.2, 14.4.1, 14.4.3; [ESL] 5.8, 6.3, 6.7
| HW3 and HW4 due |
11/3 |
Support Vector Machines -- Geometric Intuition |
No new reading |
HW5 released (supplementary file) |
11/5 |
Kernel SVM |
No new reading |
Project Proposal due |
11/10 |
Boosting |
[MLAPA] 16.4.1-5, 16.4.8-9; [ESL] 16.1-16.2
| |
11/12 |
Neural Networks and Deep Learning |
[MLAPA] 16.5.1-6, 28; [ESL] 11.3-11.7
| HW5 due |
11/17 |
In-class Midterm
|
|
|
11/20 |
Clustering and Mixture Models |
[MLAPA] 11.1-11.3, 11.4.1-4, 11.5; [ESL] 14.3.1-9
| |
11/24 |
EM algorithm |
[MLAPA] 12.2; [ESL] 14.5.1
| HW6 released (supplemental files)
Project Report Guideline released |
12/1 |
Dimensionality Reduction and PCA |
No new reading |
|
12/3 |
Project Office Hours
|
|
HW6 can be submitted, but isn't due until 12/4 in section |
12/11 |
Poster Presentation, 3-5pm
|
|
Project Report due |