EE6317 Course Information

Each student will be expected to

• watch lecture videos (2-3 hour videos per week)
• attend discussion sessions (1-2 times per week, in regular class hours) and solve short in-class quizzes
• participate in class and on piazza
• solve homework assignments, roughly 2 (collaboration is fine as long as it is explicitly declared, but you have to write the solutions in your own words)
• present a paper/do a project

• Slot A: Mondays 09:00-10:00, Wednesdays 11:00-12:00 and Thursdays 10:00-11:00
• Class venue: Online

Primarily, the material will be drawn from

• Lecture notes and slides, uploaded here and piazza.
• Elements of Information Theory, (EIT) Thomas M Cover and Joy A Thomas, second edition, Wiley inc. (Amazon link). Limited number of copies available in the university library. While the first edition of the book has all the material we need for the course, the second edition has a much expanded problem set.

Other references:

• "A mathematical theory of communication", Claude Shannon, Bell systems Tech Journal, 1948. The classic paper that started this field.
• Modern Coding Theory, Tom Richardson and Ruediger Urbanke, Cambridge University Press, 2008. The bible for LDPC codes.
• Information theory, inference, and learning algorithms, David MacKay (soft copy available for free on the author's website). A very nice book, but gives a very different flavour that what will be covered in the course. Suggested for additional reading.
• A student's guide to coding and information theory, Stefan Moser (amazon link).
• Information theory: coding theorems for discrete memoryless systems, Imre Csiszar and Janos Korner. An advanced textbook. Recommended reading after covering this course as well as EE6317.

Light reading:

• The information: A history, a theory, a flood James Gleick
• A mind at play: How Claude Shannon invented the information age Jimmy Soni and Rob Goodman. A biography of Claude Shannon.

Recap of basic concepts in information theory; Channels and error correcting codes; Discrete memoryless channels; Shannon's channel coding theorem; Typical sets and the asymptotic equipartition property; Joint and conditionally typical sets; Proof of the channel coding theorem; Linear codes; Linear codes achieve capacity; Low-density parity-check (LDPC) codes; Belief propagation decoding of LDPC codes; Polar codes; Polar codes achieve capacity of binary-input memoryless symmetric channels;

You may work in groups of at most 2 people each. Specific topics will be updated here. Good sources for the latest work on channel coding can be found in the IEEE Transactions on Information Theory, IEEE Transactions on Communications, various IEEE conferences including the annual IEEE International Symposium on Information Theory (ISIT), Information Theory Workshop (ITW). Several good preprints are made available online on the Arxiv preprint server.

The following are representative papers. You should go through papers citing/cited by the following as well.

• Turbo codes, the first class of codes that could practically get close to capacity.
• Concatenated codes, another very interesting approach that lets you approach capacity but for very large blocklengths.
• Expander codes are a family of codes that have asymptotically large minimum distance, but do not achieve capacity.
• Spatially coupled LDPC codes are also known to achieve capacity. Other references: ref 1, ref2
• Channel coding in 5G
• List decoding of polar codes
• How to construct polar codes
• Reed Muller codes achieve capacity of the BEC
• Finite blocklength information theory, the paper that was a recent breakthrough.
• Advanced topics on channel coding from these lecture notes, or these are also acceptable.
• Lattice codes achieve the capacity of the AWGN channel
• Physical layer network coding was a hot topic, and lattice codes were a contributing factor
• Coding for DNA storage
• People have used neural networks for channel coding
• Rate splitting for multiple access channels
• Codes for the binary deletion channel