
Lecture 1: introduction,
fundamental problem in ML 
slides 


Lecture 2: probability theory,
distributions, expectations. 
slides 

Video Lecture (recommended):
Introduction to Machine Learning by Iain Murray 
watch 


Lecture 3: maximum likelihood
estimation, sufficient statistics. 
slides 

Related readings: PRML sections
2.3.4 and 2.4.1 



Lecture 4:
maximum likelihood estimation, basic concepts in
information theory 
slides 

Related readings: PRML sections
2.3.4, 2.4.1, 1.6 



Lecture 5:
Bayes rule, naive Bayes, Bayesian inference for
Multinomial 
slides 

Related readings: PRML sections
2.1, 2.2 



Lecture 6:
Gaussian partitions, Bayesian inference for Gaussians 
slides 

Related readings: PRML sections
2.3.1, 2.3.2, 2.3.3, 2.3.6, 2.3.7 



Lecture 7:
Nonparametric methods, Knearest neighbor classification, linear regression 
slides 

Related readings: PRML sections
2.5, 3.1 



Lecture 8: Regularized least squares 
slides 

Related readings: PRML sections
3.1, 3.2 



Lecture 9:
Bayesian linear regression, Bayesian model comparison, discriminant function 
slides 

Related readings: PRML sections
3.3, 3.4, 4.1 



Lecture 10:
Least squares for classification, Fisherâ€™s Linear Discriminant 
slides 

Related readings: PRML sections
4.1 



Lecture 11:
Perceptron algorithm, probabilistic generative models, probabilistic discriminative models 
slides 

Related readings: PRML sections
4.2, 4.3 



Lecture 12:
Laplace approximation, BIC, Bayesian logistic
regression 
slides 

Related readings: PRML sections
4.4, 4.5 



Lecture 13:
VC Theory and largemargin classifiers 
slides 

Related readings: PRML
section 7.1 


Recommended
Reading: A tutorial on SVMs for pattern
recognition 
pdf 


Lecture 14:
Support vector machines and KKT theorem 
slides 


Lecture 15:
Nonlinear SVM, VC theory revisited, support vector regression 
slides 

Demo materials
for SVM 
svm demo 


Lecture 16:
Dimensionality Reduction (PCA and SVD) 
slides 


Lecture 17:
Latent Semantic Indexing, Latent Dirichlet
Allocation, Nonnegative Matrix Factorization 
slides 


Lecture 18:
Graphical Models (Bayesian networks) 
slides 

Related readings: PRML
section 8.1, 8.2, 8.3 



Lecture 19:
Inference on a chain, factor graphs 
slides 

Related readings: PRML
section 8.4.13 



Lecture 20:
Sumproduct and Maxsum algorithms for efficient exact inference 
slides 

Related readings: PRML
sections 8.4.45 



Lecture 21:
The Expectationmaximization Algorithm 
slides 

Related readings: PRML
sections 9.19.4 



Lecture 22: EM
for Gaussian Mixture Models 
slides 

Related readings: PRML
sections 9.19.4 



Lecture 23: Variational Bayes 
slides 

Related readings: PRML
sections 10.1 



Lecture 24:
Expectation Propogation Algorithm 
slides 

Related readings: PRML
sections 10.7 



Lecture 25: MCMC Tutorial by Iain Murray 
slides 


Lecture 26: MCMC continues 
slides 