Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Quantum Developments of Classical Information Theory

Mostafa Mansour

Published Feb 7, 2026• 0 citations
Publisher

Abstract

These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.

DOI
10.5281/zenodo.19945728

Related papers

Principles of Classical and Quantum Information Theory

Bouchra El Alaoui, Abdallah Slaoui, R. Ahl Laamara

Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.

CRC Press eBooks•May 15, 2026•0 citations

Information Theory Meets Quantum Chemistry: A Review and Perspective

Yilin Zhao, Dongbo Zhao, Chunying Rong +2 more

In this survey, we begin with a concise introduction to information theory within Shannon's framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.

Entropy•Jun 16, 2025•15 citations

A Quantum Leap in Information Theory

Stefano Mancini, Andreas Winter

This book is intended as an introductory textbook at graduate or advanced undergraduate level. Students with diverse backgrounds, in mathematics, physics and computer science, can use it to become acquainted with the quantum theory of information. Its scope is deliberately chosen to be not too broad, while being self-consistent and usable for a semester-long course for beginners. The subject is presented using a balanced mix of intuitive arguments and rigorous formalism. The emphasis is not on the physical interpretation, but rather on the formal and mathematical features of quantum information theory, in order to be accessible from different backgrounds, the only requirement being a basic knowledge of linear algebra. In particular, a background in quantum mechanics is not needed, since quantum theory is used as a mere mathematical framework without entering into physical interpretations. The book introduces all necessary concepts from quantum theory and classical information theory, to cover the elements of quantum Shannon theory: quantum states and channels, data compression and entropy, error correcting codes and channel capacities, as well as the basics of entanglement theory and quantum cryptography. Examples are provided to help the reader in understanding the different concepts, together with exercises at the end of each chapter.

WORLD SCIENTIFIC eBooks•Jan 8, 2019•15 citations

Quantum Information and Entropy

Mark M. Wilde

In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.

Cambridge University Press eBooks•Nov 2, 2016•0 citations

Schumacher Compression

Mark M. Wilde

One of the fundamental tasks in classical information theory is the compression of information. Given access to many uses of a noiseless classical channel, what is the best that a sender and receiver can make of this resource for compressed data transmission? Shannon's compression theorem demonstrates that the Shannon entropy is the fundamental limit for the compression rate in the IID setting (recall the development in Section 13.4). That is, if one compresses at a rate above the Shannon entropy, then it is possible to recover the compressed data perfectly in the asymptotic limit, and otherwise, it is not possible to do so. This theorem establishes the prominent role of the entropy in Shannon's theory of information. In the quantum world, it very well could be that one day a sender and a receiver would have many uses of a noiseless quantum channel available, and the sender could use this resource to transmit compressed quantum information. But what exactly does this mean in the quantum setting? A simple model of a quantum information source is an ensemble of quantum states { pX(x) , ∣ψ x ⟩}, i.e., the source outputs the state ∣ψ x ⟩ with probability pX(x) , and the states {∣ψ x ⟩} do not necessarily have to form an orthonormal basis. Let us suppose for the moment that the classical data x is available as well, even though this might not necessarily be the case in practice. A naive strategy for compressing this quantum information source would be to ignore the quantum states coming out, handle the classical data instead, and exploit Shannon's compression protocol from Section 13.4.

Cambridge University Press eBooks•Apr 18, 2013•1 citation

Classical Shannon Theory

Mark M. Wilde

We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”

Cambridge University Press eBooks•Nov 2, 2016•0 citations
These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.
This book is intended as an introductory textbook at graduate or advanced undergraduate level. Students with diverse backgrounds, in mathematics, physics and computer science, can use it to become acquainted with the quantum theory of information. Its scope is deliberately chosen to be not too broad, while being self-consistent and usable for a semester-long course for beginners. The subject is presented using a balanced mix of intuitive arguments and rigorous formalism. The emphasis is not on the physical interpretation, but rather on the formal and mathematical features of quantum information theory, in order to be accessible from different backgrounds, the only requirement being a basic knowledge of linear algebra. In particular, a background in quantum mechanics is not needed, since quantum theory is used as a mere mathematical framework without entering into physical interpretations. The book introduces all necessary concepts from quantum theory and classical information theory, to cover the elements of quantum Shannon theory: quantum states and channels, data compression and entropy, error correcting codes and channel capacities, as well as the basics of entanglement theory and quantum cryptography. Examples are provided to help the reader in understanding the different concepts, together with exercises at the end of each chapter.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
One of the fundamental tasks in classical information theory is the compression of information. Given access to many uses of a noiseless classical channel, what is the best that a sender and receiver can make of this resource for compressed data transmission? Shannon's compression theorem demonstrates that the Shannon entropy is the fundamental limit for the compression rate in the IID setting (recall the development in Section 13.4). That is, if one compresses at a rate above the Shannon entropy, then it is possible to recover the compressed data perfectly in the asymptotic limit, and otherwise, it is not possible to do so. This theorem establishes the prominent role of the entropy in Shannon's theory of information. In the quantum world, it very well could be that one day a sender and a receiver would have many uses of a noiseless quantum channel available, and the sender could use this resource to transmit compressed quantum information. But what exactly does this mean in the quantum setting? A simple model of a quantum information source is an ensemble of quantum states { pX(x) , ∣ψ x ⟩}, i.e., the source outputs the state ∣ψ x ⟩ with probability pX(x) , and the states {∣ψ x ⟩} do not necessarily have to form an orthonormal basis. Let us suppose for the moment that the classical data x is available as well, even though this might not necessarily be the case in practice. A naive strategy for compressing this quantum information source would be to ignore the quantum states coming out, handle the classical data instead, and exploit Shannon's compression protocol from Section 13.4.
We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”