COMP30690 Information Theory

Academic Year 2020/2021

This course provides an introduction to the fundamentals of Information Theory for computer scientists. Shannon's Information Theory is one of the greatest intellectual achievements of the 20th century. It provides a comprehensive view on the concept of information, whereby fundamental limits to the performance of practical algorithms are established. Information Theory concerns all areas where information is processed, stored or transmitted in some way or another. Consequently, it is an essential component in the education of a computer scientist.

Show/hide contentOpenClose All

Curricular information is subject to change

Learning Outcomes:

- Understand the general relevance of Shannon's Information Theory in the Information Age.
- Review essential probability theory.
- Become acquainted with fundamental information-theoretical concepts such as entropy, mutual information, relative entropy: Jensen's inequality, log-sum inequality, data-processing inequality, sufficient statistics, Fano's inequality.
- Understand the centrality of the asymptotic equipartition property in Information Theory: typical set.
- Understand the fundamentals of data compression: Kraft inequality, optimal codes, Huffman codes, Shannon-Fano-Elias coding.
- Understand the concept of channel capacity: symmetric channels, channel coding theorem, elementary channel coding techniques (repetition, Hamming codes).
- Acquire the basic insights into the connection between Information Theory and statistics.

Indicative Module Content:

(see above)

Student Effort Hours: 
Student Effort Type Hours
Lectures

24
Tutorial

16
Autonomous Student Learning

80
Total

120
Approaches to Teaching and Learning:
Reflective learning
Enquiry & problem-based learning
 
Requirements, Exclusions and Recommendations
Learning Requirements:

Working knowledge of basic calculus and algebra.

Learning Recommendations:

Knowledge of probability theory would be helpful, although the course is self-contained in this respect.


Module Requisites and Incompatibles
Not applicable to this module.
 
Assessment Strategy  
Description Timing Open Book Exam Component Scale Must Pass Component % of Final Grade
Assignment: Three sets of exercises and questions to be completed individually by the students. Throughout the Trimester n/a Alternative linear conversion grade scale 40% No

75
Multiple Choice Questionnaire: Midterm online examination. Unspecified n/a Alternative linear conversion grade scale 40% No

25

Carry forward of passed components
Yes
 
Resit In Terminal Exam
Spring Yes - 2 Hour
Please see Student Jargon Buster for more information about remediation types and timing. 
Feedback Strategy/Strategies

• Feedback individually to students, post-assessment

How will my Feedback be Delivered?

Not yet recorded.