Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.

CRC Press eBooksMay 15, 20260 citations

Relative Entropy and Mutual Information in Gaussian Statistical Field Theory

Markus Schröfl, Stefan Floerchinger

Abstract Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses and boundary conditions. We show that the relative entropy depends crucially on d , the dimension of Euclidean space. Furthermore, we demonstrate that the mutual information between two disjoint regions in $\mathbb {R}^d$ R d is finite if the two regions are separated by a finite distance and satisfies an area law. We then construct an example of “touching” regions between which the mutual information is infinite. We argue that the properties of mutual information in scalar field theories can be explained by the Markov property of these theories.Abstract Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses and boundary conditions. We show that the relative entropy depends crucially on d , the dimension of Euclidean space. Furthermore, we demonstrate that the mutual information between two disjoint regions in $$\mathbb {R}^d$$ R d is finite if the two regions are separated by a finite distance and satisfies an area law. We then construct an example of “touching” regions between which the mutual information is infinite. We argue that the properties of mutual information in scalar field theories can be explained by the Markov property of these theories.

arXiv:2307.15548Annales Henri PoincaréDec 17, 20242 citations

Classical information theory

Lajos Diósi

Springer eBooksFeb 18, 20070 citations

Classical Information Theory

Lajos Diósi

Lecture notes in physicsJan 1, 20111 citation

\mathbb {R}^d

R d is finite if the two regions are separated by a finite distance and satisfies an area law. We then construct an example of “touching” regions between which the mutual information is infinite. We argue that the properties of mutual information in scalar field theories can be explained by the Markov property of these theories.