The Mathematical Theory of Information, and Applications (Version 2.0)

These lecture notes introduce some basic concepts from Shannon’s information theory, such as (conditional) Shannon entropy, mutual information, and Rényi entropy, as well as a number of basic results involving these notions. Subsequently, well-known bounds on perfectly secure encryption, source coding (i.e. data compression), and reliable communication over unreliable channels are discussed. We also cover and prove the elegant privacy amplification theorem. This provides a means to mod out the adversary’s partial information and to distill a highly secret key. It is a key result in theoretical cryptography, and a primary starting point for the very active subarea of unconditional security.