In this survey, we begin with a concise introduction to information theory within Shannon's framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.
In this survey, we begin with a concise introduction to information theory within Shannon's framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.
These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.
Scientists often refer to matter, energy, and information as the three basic categories, though they are related by various scientific laws. Compared to the concepts of matter and energy, the scientific definition of information is relatively new: it was only in the 1940s that Claude Shannon introduced the definition of information as a reduction of uncertainty, laying the foundation for modern information theory. In molecular electronic structure theory, we routinely encounter nonnegative quantities that sum to unity, forming valid probability distributions. These include properly normalized electron densities, eigenvalues of reduced density matrices, squared wave function coefficients in orthonormal bases, and related quantities. The incorporation of information-theoretic concepts into quantum chemistry has yielded profound insights into electronic system behavior. The two dominant frameworks in electronic structure theory are quantum many-body theory and density functional theory. This thesis systematically integrates these approaches with: classical information theory, yielding the information-theoretic approach, and quantum information theory, generating distinct tools like orbital entanglement measures and quantum fidelity metrics. In this thesis, we subsequently demonstrate applications of these information-theoretic methods in quantum chemistry for analyzing electronic structure and chemical phenomena.
The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.
Quantum information, a field in which great advances have been made in the past decades, now presents opportunities for advanced chemistry. One roadblock to progress, especially for experimental chemical science, is that new concepts and technical definitions need to be learned. In this paper, we review some basic, but sometimes misunderstood, concepts of quantum information based on the mathematical formulation of quantum mechanics that will be useful for chemists interested in discovering ways that chemistry can contribute to the quantum information field. We cover topics including qubits and their density matrix formalism, quantum measurement as a quantum operation, information theory, and entanglement. We focus on the difference between the concepts in the quantum context and the classic context. We also discuss the relation and distinction among entanglement, correlation, and coherence. We aim to clarify the rigorous definition of these concepts and then indicate some examples in physical chemistry.