VVZ API is not affiliated with ETH Zurich. Data might be outdated or incorrect. Please view the official ETHZ Vorlesungsverzeichnis for binding information.

252-0526-00L 7 Credits MSC , WBZ D-BSSE , D-INFK , D-MATH , D-MAVT , D-ITET
You're viewing possible stale or outdated data. Please check the latest semester for more up-to-date information.

Statistical Learning Theory

Lecturers & Examiners: Prof. em. Dr. Joachim M. Buhmann, Dr. Carlos Cotrini Jimenez
VVZ CR n/a

Last Updated: 2026-02-05 15:42:01

Abstract

The course covers advanced methods of statistical learning:- Variational methods and optimization.- Deterministic annealing.- Clustering for diverse types of data.- Model validation by information theory.

Objective

The course surveys recent methods of statistical learning. The fundamentals of machine learning, as presented in the courses "Introduction to Machine Learning" and "Advanced Machine Learning", are expanded from the perspective of statistical learning.

Content

- Variational methods and optimization. We consider optimization approaches for problems where the optimizer is a probability distribution. We will discuss concepts like maximum entropy, information bottleneck, and deterministic annealing. - Clustering. This is the problem of sorting data into groups without using training samples. We discuss alternative notions of "similarity" between data points and adequate optimization procedures. - Model selection and validation. This refers to the question of how complex the chosen model should be. In particular, we present an information theoretic approach for model validation. - Statistical physics models. We discuss approaches for approximately optimizing large systems, which originate in statistical physics (free energy minimization applied to spin glasses and other models). We also study sampling methods based on these models.

Resources

Lecture Notes

A draft of a script will be provided. Lecture slides will be made available.

Literature

Hastie, Tibshirani, Friedman: The Elements of Statistical Learning, Springer, 2001. L. Devroye, L. Gyorfi, and G. Lugosi: A probabilistic theory of pattern recognition. Springer, New York, 1996

Learning Materials (Links)

General Information

Language
English
Levels
MSC , WBZ
Frequency
Yearly recurring

Examination

Type
session examination
Mode
written 180 minutes
Aids
4 A4 handwritten or fontsize 12 pages (2 sheets with notes on its two sides); course script.
70% session examination, 30% project; the final grade will be calculated as weighted average of both these elements. As a compulsory continuous performance assessment task, the project must be passed on its own and has a bonus/penalty function.The practical projects are an integral part (60 hours of work, 2 credits) of the course. Participation is mandatory. Failing the project results in a failing grade for the overall course examination.Students who fail to fulfil the project requirement must de-register from the exam. Otherwise, they are not admitted to the exam and they will be treated as a no show.

Course Components

Type Title Time & Place Hours
lecture Statistical Learning Theory
  • Mon 14:00-16:00 (ER SA TZ)
  • Mon 14:15-16:00 (HG G 3)
  • Tue 17:00-18:00 (ER SA TZ)
  • Tue 17:15-18:00 (HG G 3)
3 h weekly
exercise Statistical Learning Theory
  • Mon 16:00-18:00 (ER SA TZ)
  • Mon 16:15-18:00 (HG G 3)
2 h weekly
independent project Statistical Learning Theory No time listed 1 h weekly

Offered In