Date 
Topics 
Readings 

Introduction 

Th 8/28 
Machine Learning Foundations Overview, applications, settings 
Bishop 1 

Tu 9/2 
Probability Probability, stats 
You do NOT need to read all Probability: Bishop Chapter 2 Bishop Appendix B 

Th 9/4 
Decision Trees Construction, pruining, overfitting 

Supervised Learning: Linear Methods 

Tu 9/9 
Regression Least squares and regression 
Bishop 3 

Th 9/11 
Classification Logistic Regression 
Bishop 4 

Tu 9/16 
Naive Bayes 

Th 9/18 
Online Learning for Structured Prediction Perceptron, multiclass, structured 
Blum. OnLine Algorithms in Machine Learning. 1996 Bishop 4.1.2 Reducing Multiclass to Binary (Sections 1, 2, 3) 

Supervised Learning: NonLinear Methods 

Tu 9/23 
Support Vector Machines Maxmargin classification and optimization 
Bishop 7.1 Bishop Appendix E 

Th 9/25 
Kernel Methods Dual optmization, kernel trick 
Bishop 6.1, 6.2 

Tu 9/30 
Instance based learning Nearestneighbors 
Bishop 2.5 Mitchell 88.4 

Th 10/2 
Ensemble Methods Boosting 
Bishop 14.1,14.2,14.3 

Tu 10/7 
Deep Networks 1 Neural Network models 
Bishop 5.1,5.2,5.3,5.5 

Th 10/9 
Deep Networks 2 Deep learning 
One of the Following: Deep Learning for NLP (without Magic) ICML Deep Learning Tutorial (LeCun and Ranzato) 

Unsupervised Learning 

Tu 10/14 
Midterm Exam 

Th 10/16 
No Class Monday schedule 

Tu 10/21 
Clustering ExpectationMaximization and kmeans 
Bishop 9 

Th 10/23 
EM and Clustering 1 Gaussian mixture models 
Bishop 9 

Tu 10/28 
EM and Clustering 2 The EM Algorithm 
Bishop 9 

Graphical Models 

Th 10/30 
Graphical models 1 Bayesian networks and conditional independence 
Bishop 8.1, 8.2 

Tu 11/4 
Graphical models 2 MRFs and Exact inference 
Bishop 8.3, 8.4 

Th 11/6 
Graphical models 3 Inference 
Bishop 8.3, 8.4 

Tu 11/11 
Sequential graphical models 1 Max Sum and Max Product 
Bishop 13.1,13.2 

Th 11/13 
Sequential graphical models 2 HMMs and CRFs 

Other Topics 

Tu 11/18 
Dimensionality reduction PCA, probabilistic PCA, LDA 
Bishop 12.1,12.2,12.3 Jonathon Shlens's PCA Tutorial (more mathematically oriented) 

Th 11/20 
Current Trends in Supervised Learning 

Tu 12/2 
TBA 

Th 12/4 
Practical Machine Learning 
