Lectures in Statistical Learning Theory
Department of Computer Science & Automation
Indian Institute of Science
Introduction to Statistical Consistency in Machine Learning
Statistical consistency is a fundamental notion for learning algorithms
that asks the following simple question: As a learning algorithm is
supplied with more and more training data, does the prediction model
learned by it approach an ideal or optimal prediction model for the given
learning problem? The last several years have seen significant progress in
development of tools that help us understand and characterize consistency
of learning algorithms for various types of prediction problems. This
series of two 2-hour lectures will give a guided tour of recent
developments in the area, including self-contained introductions to the
central notions of loss functions and loss matrices, surrogate loss
functions, calibration, and surrogate regret bounds.
Lecture 1: Statistical consistency of learning algorithms for binary
Fri Aug 22, 2014
3:00 - 5:00 PM
Lecture 2: Statistical consistency of learning algorithms for multiclass
Mon Aug 25, 2014
3:30 - 5:30 PM
- Lecture notes from E0 370, Aug-Dec 2013 (these are scribed so somewhat rough, but should be possible to follow the main ideas):
Tutorial lecture slides, Indo-US lectures week, Jan 2014.
- Papers on proper CPE losses/proper composite surrogates:
- Papers on calibrated surrogates for binary classification:
- Peter L. Bartlett, Michael I. Jordan, and Jon D. McAuliffe.
Convexity, classification, and risk bounds.
Journal of the American Statistical Association, 101(473):138-156, 2006.
- Clayton Scott.
Calibrated asymmetric surrogate losses.
Electronic Journal of Statistics, 6:958-992, 2012.
- Nagarajan Natarajan, Inderjit S. Dhillon, Pradeep Ravikumar and Ambuj Tewari.
Learning with noisy labels.
- Papers on calibrated surrogates for multiclass 0-1 classification:
- Papers on calibrated surrogates for general multiclass losses: