VVZ API is not affiliated with ETH Zurich. Data might be outdated or incorrect. Please view the official ETHZ Vorlesungsverzeichnis for binding information.

252-0055-00L 4 Credits BSC D-INFK
You're viewing possible stale or outdated data. Please check the latest semester for more up-to-date information.

Information Theory

Lecturers & Examiners: Prof. Dr. Thomas Hofmann
VVZ CR 3.5

Last Updated: 2026-02-05 16:22:56

Abstract

This short course on information theory will introduce fundamental concepts such as entropy, information, sufficiency, typicality, concentration and will present a range of topics from data coding, statistics, inference, decision-making and learning that relate in interesting ways to information theory.

Objective

The goal of the course is to familiarize students with the foundations of information theory and to illustrate its practical use across a wide range of applications.

Content

Part 1: Information - Entropy & Information, Sufficiency, Typicality & Concentration Part 2: Coding - Data Compression, Rate-Distortion Theory, Channel Coding Part 3: Inference - Statistical Inference, Maximum Entropy inference, Algorithmic Complexity Part 4: Decisions - Betting Games, Optimal Investment, Evolution Part 5: Learning - Memory, Auto-encoding (may be subject to change)

Resources

Lecture Notes

A script will be distributed over the course of the semester.

Literature

T. Cover, J. Thomas: Elements of Information Theory, John Wiley, 2006 (2nd Edition) D. MacKay, Information Theory, Inference and Learning Algorithms, Cambridge University Press, 2003.

Learning Materials (Links)

General Information

Language
English
Levels
BSC
Frequency
Yearly recurring

Examination

Type
graded semester performance
The final assessment will be a combination of 3 pen & paper tests (each weighted with 25%) and a short term paper (25%). There will be no written exam. Problem sets will prepare for the tests. The tests will be scheduled roughly in mid-March, late April and late May. The latest date to de-register from the class will be March 24.

Registration & Places

Max Places
50

Course Components

Type Title Time & Place Hours
lecture Information Theory
  • Fri 08:15-10:00 (CAB G 51)
2 h weekly
exercise Information Theory
  • Tue 13:15-14:00 (CAB G 59)
1 h weekly

Offered In

  • Computer Science Bachelor
    • Electives (Students may also choose courses from the Master's program in Computer Science. It is their responsibility to make sure that they meet the requirements and conditions for these courses.)