VVZ API is not affiliated with ETH Zurich. Data might be outdated or incorrect. Please view the official ETHZ Vorlesungsverzeichnis for binding information.
Information Theory
Last Updated: 2026-02-05 16:38:16
Abstract
This short course on information theory will introduce fundamental concepts such as entropy, information, sufficiency, typicality, concentration and will present a range of topics from data coding, statistics, inference, decision-making and learning that relate in interesting ways to information theory.
Objective
The goal of the course is to familiarize students with the foundations of information theory and to illustrate its practical use across a wide range of applications.
Content
Part 1: Information - Entropy & Information, Sufficiency, Typicality & Concentration Part 2: Coding - Data Compression, Rate-Distortion Theory, Channel Coding Part 3: Inference - Statistical Inference, Maximum Entropy inference, Algorithmic Complexity Part 4: Decisions - Betting Games, Optimal Investment, Evolution Part 5: Learning - Memory, Auto-encoding (may be subject to change)
Resources
Lecture Notes
A script will be distributed over the course of the semester.
Literature
T. Cover, J. Thomas: Elements of Information Theory, John Wiley, 2006 (2nd Edition) D. MacKay, Information Theory, Inference and Learning Algorithms, Cambridge University Press, 2003.
Learning Materials (Links)
- Main link
- Information
General Information
- Language
- English
- Levels
- BSC
- Frequency
- Yearly recurring
Examination
- Type
- session examination
- Mode
- written 120 minutes
- Aids
- Max 4 pages (single sided) printed or handwritten notes.
Course Components
| Type | Title | Time & Place | Hours |
|---|---|---|---|
| lecture | Information Theory |
|
2 h weekly |
| exercise | Information Theory |
|
1 h weekly |
Offered In
-
-
Electives (Students may also choose courses from the Master's program in Computer Science. It is their responsibility to make sure that they meet the requirements and conditions for these courses.)
-