Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

Cambridge University Press eBooks

Classical Information and Entropy

Mark M. Wilde

Published Nov 2, 2016• 0 citations
Publisher

Abstract

All physical systems register bits of information, whether it be an atom, an electrical current, the location of a billiard ball, or a switch. Information can be classical, quantum, or a hybrid of both, depending on the system. For example, an atom or an electron or a superconducting system can register quantum information because the quantum theory applies to each of these systems, but we can safely argue that the location of a billiard ball registers classical information only. These atoms or electrons or superconducting systems can also register classical bits because it is always possible for a quantum system to register classical bits. The term information , in the context of information theory, has a precise meaning that is somewhat different from our prior “everyday” experience with it. Recall that the notion of the physical bit refers to the physical representation of a bit, and the information bit is a measure of how much we learn from the outcome of a random experiment. Perhaps the word “surprise” better captures the notion of information as it applies in the context of information theory. This chapter begins our formal study of classical information. Recall that Chapter 2 reviewed some of the major operational tasks in classical information theory. Here, our approach is somewhat different because our aim is to provide an intuitive understanding of information measures, in terms of the parties who have access to the classical systems. We define precise mathematical formulas that measure the amount of information encoded in a single physical system or in multiple physical systems. The advantage of developing this theory is that we can study information in its own right without having to consider the details of the physical system that registers it. We first introduce the entropy in Section 10.1 as the expected surprise of a random variable. We extend this basic notion of entropy to develop other measures of information in Sections 10.2–10.6 that prove useful as intuitive informational measures, but also, and perhaps more importantly, these measures are the answers to operational tasks that one might wish to perform using noisy resources. While introducing these quantities, we discuss and prove several mathematical results concerning them that are important tools for the practicing information theorist.

DOI
10.1017/9781316809976.013

Related papers

Classical Information and Entropy

Mark M. Wilde

All physical systems register bits of information, whether it be an atom, an electrical current, the location of a billiard ball, or a switch. Information can be classical, quantum, or a hybrid of both, depending on the system. For example, an atom or an electron or a superconducting system can register quantum information because the quantum theory applies to each of these systems, but we can safely argue that the location of a billiard ball registers classical information only. These atoms or electrons or superconducting systems can also register classical bits because it is always possible for a quantum system to register classical bits. The term information , in the context of information theory, has a precise meaning that is somewhat different from our prior “everyday” experience with it. Recall that the notion of the physical bit refers to the physical representation of a bit, and the information bit is a measure of how much we learn from the outcome of a random experiment. Perhaps the word “surprise” better captures the notion of information as it applies in the context of information theory. This chapter begins our formal study of classical information. Recall that Chapter 2 overviewed some of the major operational tasks in classical information theory. Here, our approach is somewhat different because our aim is to provide an intuitive understanding of information measures, in terms of the parties who have access to the classical systems.

Cambridge University Press eBooks•Apr 18, 2013•0 citations

Quantum Information and Entropy

Mark M. Wilde

In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.

Cambridge University Press eBooks•Nov 2, 2016•0 citations

Quantum Information and Entropy

Mark M. Wilde

In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumman entropy. It is the quantum analog of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state. The von Neumann entropy gives meaning to a notion of the information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state in an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is in a quantum system. The beginning definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical spatial correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations.

Cambridge University Press eBooks•Apr 18, 2013•0 citations

Quantum Information with Meaning Inside and Outside the Quantum

Alastair A. Abbott, Cristian S. Calude, Karl Svozil

The ability to transfer information securely is essential in the modern world we live in. While the field of cryptography has long researched ways to do this, events over the last couple of years have brought the issues of secure information transfer squarely into the public eye. As a consequence, security and cryptography have quickly become issues of social and political importance, sparking debates about the values of privacy and the role of government. An essential ingredient of modern cryptography is the generation of randomness: cryptographic techniques are built on the premise that one has access to random bits. It is well known, however, that computers cannot produce algorithmically random sequences – that is, sequences with maximal algorithmic information content – but are doomed to produce ‘pseudorandomness’. This lack of randomness can be, and has been, exploited to compromise security, see [4]. The active field of quantum information theory has proposed approaches to provide supposedly ‘unbreakable’ security by exploiting various quantum phenomena. This security unfortunately relies on assumptions about the nature of quantum measurements and their ability to generate random bits. Anton Zeilinger summarises this by postulating that the simplest quantum systems, qubit, can hold only one bit of classical information [6]. This foundational principle is in line with a wider paradigm shift to view quantum information as an extension of classical information, but it is nonetheless unsatisfying to simply postulate this principle, given its importance in determining the practical advantages of quantum information theory. In order to understand better how quantum mechanics can help generate meaningful information, we instead look to relate the outcomes of quantum measurements to formal properties of the system based on more fundamental assumptions. Indeed, we have shown that a) some of the postulated properties of quantum information follow from the formal structure of the theory and b) a purely formal notion of information within a quantum world can generate, via measurement, meaningful and useful information in the macroscopic world (for example, in cryptography). The indeterminism of quantum measurements can be formalised via the notion of value indefiniteness. To explain this concept, let us consider an arbitrary quantum system and ask whether the outcome of a measurement of any observable quantity A (such as the energy of the system, its angular momentum (spin), etc.) is determined prior to the measurement. If this is the case, then we say the observable is value definite with value v(A); otherwise, the observable is value indefinite and v(A) is undefined. Mathematically, one can reduce all observable quantities to so-called projection observables, which can only take the values 0 or 1. Thus, the question of whether the outcome of several measurements can be simultaneously determined in advance can be rephrased in terms of the information ‘carried’ by a particular system in a definite quantum state. Classically, one expects that all quantities are determined in advance, and hence all observables are value definite. Quantum mechanically, however, the belief is that this is not the case, and the information content of quantum systems is limited. Formulating carefully the notion of value indefiniteness allows us to formalise the notion of (quantum) indeterminism; however, this doesn’t help clarify whether quantum systems are indeed value indefinite or not. Staying in this formal framework, the Kochen-Specker theorem [5] provides a first positive result, showing that at least some observables must be value indefinite if one makes the assumption known as non-contextuality, which states that any definite values that exist must be independent of other compatible measurements that may or may not be performed on the system. Under the same assumption, this theorem can be strengthened to show that only one single observable can have the definite value 1 (see [2]). Furthermore, only observables that can be measured simultaneously with this single one can have the definite value 0; the rest must all be value indefinite. Since the preparation of a system involves precisely ensuring that, usually via measurement, the system is in a definite state with respect to some desired observable, this result shows that no other incompatible observable can be value definite. That is, preparing a system in a definite state by making the ‘preparation’ observable value definite specifies completely the information content of the quantum system. This is an example of syntactical quantum information acquiring meaning at the level of the quantum itself. Can the syntactical information at the level of the quantum generate meaning outside the quantum, that is, at the macroscopic level? The results cited above hold in the Hilbert-space framework of quantum mechanics and are formulated only in terms of a syntactical notion of information. Their real importance becomes evident when one interprets them in the context of quantum measurements. Specifically, if we prepare a quantum system in a known state, they allow us to ‘locate’ observables which we can measure, but which are value indefinite; that is, observables whose measurement results are not specified by any pre-existing property of the quantum system. Furthermore, with respect to a mathematical model of unpredictability which we developed in [3], the results of these measurements can be shown to be absolutely unpredictable. In this way we get a mathematical explanation and justification of the largely accepted intuition that quantum mechanics is inherently unpredictable, and that this unpredictability arises from the phenomenon of indefiniteness within the quantum world. Furthermore, if one considers a hypothetical infinite sequence generated by the repeated measurement of a quantum value indefinite observable, one can prove that these sequences must be strongly incomputable, technically ‘bi-immune’ [1]. Such sequences cannot be generated by any Turing machine or classical computer, showing that value indefiniteness leads to a clear classical/quantum split in a purely algorithmic context. Importantly, bi-immunity is a property of observed, macroscopic quantities, quite separate from the quantum framework in which the value indefiniteness is formalised. This form of macroscopic meaning created from the lack of syntactic information is precisely the scenario that quantum random number generators try to create, and which is essential for the development and certification of quantum cryptographic systems. The fact that the macroscopic information created goes beyond anything classically obtainable serves as a valuable practical resource, outside of and removed from the quantum formalism. To conclude, quantum information creates meaning within the quantum and via measure- ment, the lack of information within the quantum, creates meaning and valuable information at the macroscopic level. Acknowledgments The second author thanks Prof. S. Marcus for useful conversations on quantum information theory. This work was supported in part by Marie Curie FP7-PEOPLE-2010-IRSES Grant RANPHYS. References and Notes Abbott, A.A.; Calude, C.S.; Svozil, K. Strong Kochen-Specker theorem and incomputability of quantum randomness. Physical Review A 2012, 86, 062109. Abbott, A.A.; Calude, C.S.; Svozil, K. Value-indefinite observables are almost everywhere. Physical Review A 2013, 89, 032109. Abbott, A.A.; Calude, C.S.; Svozil, K. On the unpredictability of individual quantum measurement outcomes. CDMTCS Research Report 2014, 458. Calude, C.S. Quantum randomness & cryptology. CyberTalk 2013, 3, 42–43. Kochen, S.; Specker, E. The problem of hidden variables in quantum mechanics. Journal of Mathematics and Mechanics 1967, 17, 59–87. Zeilinger, A. A foundational principle for quantum mechanics. Foundations of Physics 1999, 29(4),631–643.

ISIS Summit: The Information Society at the Crossroads•Jun 23, 2015•0 citations

The Density Matrix

Jonathan Allday, Simon Hands

The quantum state contains all accessible information about a system. However, for various reasons, the information is less complete than we might expect given the classical states we are used to. This suggests the possibility that the state itself contains a measure of entropy (lack of information), which could potentially arise in two ways. Firstly, quantum states contain an inherent uncertainty with respect to the measurement outcomes of conjugate variables. Secondly, due to limitations in the preparation process, we may not be sure which state a system is occupying. The best we can do in this situation is to assign each potential state a classical probability and construct a specialist mathematical representation to deal with the situation. This is known as the density matrix. It turns out to be a rather handy tool in general and crucial for the exploration of quantum entropy. So, in this chapter, we are going to extend our technical understanding of quantum theory and the representation of states. This will arm us sufficiently to venture into the physics of the measurement problem and the entropy of quantum systems, which we will tackle in Chapter 12 .

CRC Press eBooks•Sep 3, 2024•0 citations

Classical Shannon Theory

Mark M. Wilde

We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”

Cambridge University Press eBooks•Nov 2, 2016•0 citations
All physical systems register bits of information, whether it be an atom, an electrical current, the location of a billiard ball, or a switch. Information can be classical, quantum, or a hybrid of both, depending on the system. For example, an atom or an electron or a superconducting system can register quantum information because the quantum theory applies to each of these systems, but we can safely argue that the location of a billiard ball registers classical information only. These atoms or electrons or superconducting systems can also register classical bits because it is always possible for a quantum system to register classical bits. The term information , in the context of information theory, has a precise meaning that is somewhat different from our prior “everyday” experience with it. Recall that the notion of the physical bit refers to the physical representation of a bit, and the information bit is a measure of how much we learn from the outcome of a random experiment. Perhaps the word “surprise” better captures the notion of information as it applies in the context of information theory. This chapter begins our formal study of classical information. Recall that Chapter 2 reviewed some of the major operational tasks in classical information theory. Here, our approach is somewhat different because our aim is to provide an intuitive understanding of information measures, in terms of the parties who have access to the classical systems. We define precise mathematical formulas that measure the amount of information encoded in a single physical system or in multiple physical systems. The advantage of developing this theory is that we can study information in its own right without having to consider the details of the physical system that registers it. We first introduce the entropy in Section 10.1 as the expected surprise of a random variable. We extend this basic notion of entropy to develop other measures of information in Sections 10.2–10.6 that prove useful as intuitive informational measures, but also, and perhaps more importantly, these measures are the answers to operational tasks that one might wish to perform using noisy resources. While introducing these quantities, we discuss and prove several mathematical results concerning them that are important tools for the practicing information theorist.
All physical systems register bits of information, whether it be an atom, an electrical current, the location of a billiard ball, or a switch. Information can be classical, quantum, or a hybrid of both, depending on the system. For example, an atom or an electron or a superconducting system can register quantum information because the quantum theory applies to each of these systems, but we can safely argue that the location of a billiard ball registers classical information only. These atoms or electrons or superconducting systems can also register classical bits because it is always possible for a quantum system to register classical bits. The term information , in the context of information theory, has a precise meaning that is somewhat different from our prior “everyday” experience with it. Recall that the notion of the physical bit refers to the physical representation of a bit, and the information bit is a measure of how much we learn from the outcome of a random experiment. Perhaps the word “surprise” better captures the notion of information as it applies in the context of information theory. This chapter begins our formal study of classical information. Recall that Chapter 2 overviewed some of the major operational tasks in classical information theory. Here, our approach is somewhat different because our aim is to provide an intuitive understanding of information measures, in terms of the parties who have access to the classical systems.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumman entropy. It is the quantum analog of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state. The von Neumann entropy gives meaning to a notion of the information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state in an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is in a quantum system. The beginning definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical spatial correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations.
The ability to transfer information securely is essential in the modern world we live in. While the field of cryptography has long researched ways to do this, events over the last couple of years have brought the issues of secure information transfer squarely into the public eye. As a consequence, security and cryptography have quickly become issues of social and political importance, sparking debates about the values of privacy and the role of government. An essential ingredient of modern cryptography is the generation of randomness: cryptographic techniques are built on the premise that one has access to random bits. It is well known, however, that computers cannot produce algorithmically random sequences – that is, sequences with maximal algorithmic information content – but are doomed to produce ‘pseudorandomness’. This lack of randomness can be, and has been, exploited to compromise security, see [4]. The active field of quantum information theory has proposed approaches to provide supposedly ‘unbreakable’ security by exploiting various quantum phenomena. This security unfortunately relies on assumptions about the nature of quantum measurements and their ability to generate random bits. Anton Zeilinger summarises this by postulating that the simplest quantum systems, qubit, can hold only one bit of classical information [6]. This foundational principle is in line with a wider paradigm shift to view quantum information as an extension of classical information, but it is nonetheless unsatisfying to simply postulate this principle, given its importance in determining the practical advantages of quantum information theory. In order to understand better how quantum mechanics can help generate meaningful information, we instead look to relate the outcomes of quantum measurements to formal properties of the system based on more fundamental assumptions. Indeed, we have shown that a) some of the postulated properties of quantum information follow from the formal structure of the theory and b) a purely formal notion of information within a quantum world can generate, via measurement, meaningful and useful information in the macroscopic world (for example, in cryptography). The indeterminism of quantum measurements can be formalised via the notion of value indefiniteness. To explain this concept, let us consider an arbitrary quantum system and ask whether the outcome of a measurement of any observable quantity A (such as the energy of the system, its angular momentum (spin), etc.) is determined prior to the measurement. If this is the case, then we say the observable is value definite with value v(A); otherwise, the observable is value indefinite and v(A) is undefined. Mathematically, one can reduce all observable quantities to so-called projection observables, which can only take the values 0 or 1. Thus, the question of whether the outcome of several measurements can be simultaneously determined in advance can be rephrased in terms of the information ‘carried’ by a particular system in a definite quantum state. Classically, one expects that all quantities are determined in advance, and hence all observables are value definite. Quantum mechanically, however, the belief is that this is not the case, and the information content of quantum systems is limited. Formulating carefully the notion of value indefiniteness allows us to formalise the notion of (quantum) indeterminism; however, this doesn’t help clarify whether quantum systems are indeed value indefinite or not. Staying in this formal framework, the Kochen-Specker theorem [5] provides a first positive result, showing that at least some observables must be value indefinite if one makes the assumption known as non-contextuality, which states that any definite values that exist must be independent of other compatible measurements that may or may not be performed on the system. Under the same assumption, this theorem can be strengthened to show that only one single observable can have the definite value 1 (see [2]). Furthermore, only observables that can be measured simultaneously with this single one can have the definite value 0; the rest must all be value indefinite. Since the preparation of a system involves precisely ensuring that, usually via measurement, the system is in a definite state with respect to some desired observable, this result shows that no other incompatible observable can be value definite. That is, preparing a system in a definite state by making the ‘preparation’ observable value definite specifies completely the information content of the quantum system. This is an example of syntactical quantum information acquiring meaning at the level of the quantum itself. Can the syntactical information at the level of the quantum generate meaning outside the quantum, that is, at the macroscopic level? The results cited above hold in the Hilbert-space framework of quantum mechanics and are formulated only in terms of a syntactical notion of information. Their real importance becomes evident when one interprets them in the context of quantum measurements. Specifically, if we prepare a quantum system in a known state, they allow us to ‘locate’ observables which we can measure, but which are value indefinite; that is, observables whose measurement results are not specified by any pre-existing property of the quantum system. Furthermore, with respect to a mathematical model of unpredictability which we developed in [3], the results of these measurements can be shown to be absolutely unpredictable. In this way we get a mathematical explanation and justification of the largely accepted intuition that quantum mechanics is inherently unpredictable, and that this unpredictability arises from the phenomenon of indefiniteness within the quantum world. Furthermore, if one considers a hypothetical infinite sequence generated by the repeated measurement of a quantum value indefinite observable, one can prove that these sequences must be strongly incomputable, technically ‘bi-immune’ [1]. Such sequences cannot be generated by any Turing machine or classical computer, showing that value indefiniteness leads to a clear classical/quantum split in a purely algorithmic context. Importantly, bi-immunity is a property of observed, macroscopic quantities, quite separate from the quantum framework in which the value indefiniteness is formalised. This form of macroscopic meaning created from the lack of syntactic information is precisely the scenario that quantum random number generators try to create, and which is essential for the development and certification of quantum cryptographic systems. The fact that the macroscopic information created goes beyond anything classically obtainable serves as a valuable practical resource, outside of and removed from the quantum formalism. To conclude, quantum information creates meaning within the quantum and via measure- ment, the lack of information within the quantum, creates meaning and valuable information at the macroscopic level. Acknowledgments The second author thanks Prof. S. Marcus for useful conversations on quantum information theory. This work was supported in part by Marie Curie FP7-PEOPLE-2010-IRSES Grant RANPHYS. References and Notes Abbott, A.A.; Calude, C.S.; Svozil, K. Strong Kochen-Specker theorem and incomputability of quantum randomness. Physical Review A 2012, 86, 062109. Abbott, A.A.; Calude, C.S.; Svozil, K. Value-indefinite observables are almost everywhere. Physical Review A 2013, 89, 032109. Abbott, A.A.; Calude, C.S.; Svozil, K. On the unpredictability of individual quantum measurement outcomes. CDMTCS Research Report 2014, 458. Calude, C.S. Quantum randomness & cryptology. CyberTalk 2013, 3, 42–43. Kochen, S.; Specker, E. The problem of hidden variables in quantum mechanics. Journal of Mathematics and Mechanics 1967, 17, 59–87. Zeilinger, A. A foundational principle for quantum mechanics. Foundations of Physics 1999, 29(4),631–643.
The quantum state contains all accessible information about a system. However, for various reasons, the information is less complete than we might expect given the classical states we are used to. This suggests the possibility that the state itself contains a measure of entropy (lack of information), which could potentially arise in two ways. Firstly, quantum states contain an inherent uncertainty with respect to the measurement outcomes of conjugate variables. Secondly, due to limitations in the preparation process, we may not be sure which state a system is occupying. The best we can do in this situation is to assign each potential state a classical probability and construct a specialist mathematical representation to deal with the situation. This is known as the density matrix. It turns out to be a rather handy tool in general and crucial for the exploration of quantum entropy. So, in this chapter, we are going to extend our technical understanding of quantum theory and the representation of states. This will arm us sufficiently to venture into the physics of the measurement problem and the entropy of quantum systems, which we will tackle in Chapter 12 .
We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”