Neri Merhav (2010), "Statistical Physics and Information Theory", Foundations and Trends® in Communications and Information Theory: Vol. 6: No. 1–2, pp 1-212. http://dx.doi.org/10.1561/0100000052

© 2010 N. Merhav

Download article
**In this article:**

1. Introduction

2. Basic Background in Statistical Physics

3. Physical Interpretations of Information Measures

4. Analysis Tools and Asymptotic Methods

5. Interacting Particles and Phase Transitions

6. The Random Energy Model and Random Coding

7. Extensions of the REM

8. Summary and Outlook

Acknowledgments

References

This monograph is based on lecture notes of a graduate course, which focuses on the relations between information theory and statistical physics. The course was delivered at the Technion during the Spring of 2010 for the first time, and its target audience consists of EE graduate students in the area of communications and information theory, as well as graduate students in Physics who have basic background in information theory. Strong emphasis is given to the analogy and parallelism between information theory and statistical physics, as well as to the insights, the analysis tools and techniques that can be borrowed from statistical physics and 'imported' to certain problem areas in information theory. This is a research trend that has been very active in the last few decades, and the hope is that by exposing the students to the meeting points between these two disciplines, their background and perspective may be expanded and enhanced. This monograph is substantially revised and expanded relative to an earlier version posted in arXiv (1006.1565v1 [cs.iT]).

1. Introduction

2. Basic Background in Statistical Physics

3. Physical Interpretations of Information Measures

4. Analysis Tools and Asymptotic Methods

5. Interacting Particles and Phase Transitions

6. The Random Energy Model and Random Coding

7. Extensions of the REM

8. Summary and Outlook

Acknowledgments

References

*Statistical Physics and Information Theory* focuses on some of the relationships and the interplay between information
theory and statistical physics - a branch of physics that deals with many-particle systems using probabilistic and statistical methods in the
microscopic level. The author interlaces the physics and the information-theoretic subjects with each other, rather than giving them in two
continuous, separate parts. This makes the relations between information theory and statistical physics more apparent. He also shows that, not
only are the relations between information theory and statistical physics interesting academically in their own right, but moreover, they prove
useful and beneficial in that they provide information-theorists with new insights and mathematical tools to deal with information-theoretic
problems. These mathematical tools sometimes prove a lot more efficient than traditional tools used in information theory, and they may give
either simpler expressions for performance analysis, or improved bounds, or both. The author provides examples of the techniques and insights.
One example is the use of integrals in the complex plane and the saddle-point method. Another example is the analysis technique of error
exponents, which stems from the random energy model, along with its insights about phase transitions.

*Statistical Physics and Information Theory* highlights to the reader techniques that have been been used in one branch of
science which can be applied effectively in another. The point is that it is not the physics itself that may be useful, it is the way in which
physicists use mathematical tools. This will bring new insights to all students and researchers in the field of information theory and
communications.