This text is devoted to the theory of probabilistic information measures and their application to coding theorems for information sources and noisy channels. The eventual goal is a general development of Shannon's mathematical theory of communication, but much of the space is devoted to the tools and methods required to prove the Shannon coding theorems. These tools form an area common to ergodic theory and information theory and compromise several quantitative notions of the information in random variables, random processes and dynamical systems. Examples are entropy, mutual information and discrimination or relative entropy, along with the limiting normalized versions of these quantities such as entropy rate and information rate. Much of the book is concerned with their properties, especially the long term asymptotic behaviour of sample information and expected information.
- ISBN10 3540973710
- ISBN13 9783540973713
- Publish Date September 1990 (first published 4 September 1990)
- Publish Status Out of Print
- Out of Print 23 March 2008
- Publish Country DE
- Publisher Springer-Verlag Berlin and Heidelberg GmbH & Co. KG
- Imprint Springer-Verlag Berlin and Heidelberg GmbH & Co. K
- Format Hardcover
- Pages 355
- Language English