We address the problem of quantifying the information content of a source for an arbitrary information theory, where the information content is defined in terms of the asymptotic achievable compression rate. The functions that solve this problem in classical and quantum theory are Shannon's and von Neumann's entropy, respectively. However, in a general information theory there are three different functions that extend the notion of entropy, and this opens the question as to whether any of them can universally play the role of the quantifier for the information content. Here we answer the question in the negative, by evaluating the information content as well as the various entropic functions in a toy theory called Bilocal Classical Theory.
We address the problem of quantifying the information content of a source for an arbitrary information theory, where the information content is defined in terms of the asymptotic achievable compression rate. The functions that solve this problem in classical and quantum theory are Shannon's and von Neumann's entropy, respectively. However, in a general information theory there are three different functions that extend the notion of entropy, and this opens the question as to whether any of them can universally play the role of the quantifier for the information content. Here we answer the question in the negative, by evaluating the information content as well as the various entropic functions in a toy theory called Bilocal Classical Theory.
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits replacing bits. In the mentioned cases the information content can be quantified through Shannon's and von Neumann's entropy, respectively. Here we extend the definition of information content to operational probabilistic theories, and prove relevant properties such as the subadditivity and the relation between purity and information content of a state. We prove the consistency of the present notion of information content when applied to the classical and the quantum case. Finally, the relation with one of the notions of entropy that can be introduced in general probabilistic theories, the maximum accessible information, is given in terms of a lower bound.
Information plays an important role in our understanding of the physical world. Hence we propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical worlds, our measure reduces to the von Neumann and Shannon entropies, respectively. It can even be used in a quantum or classical setting where we are only allowed to perform a limited set of operations. In a world that admits superstrong correlations in the form of non-local boxes, our measure can be used to analyze protocols such as superstrong random access encodings and the violation of 'information causality'. However, we also show that in such a world no entropic measure can exhibit all the properties we commonly accept in a quantum setting. For example, there exists no 'reasonable' measure of conditional entropy that is subadditive. Finally, we prove a coding theorem for some theories that is analogous to the quantum and classical settings, providing us with an appealing operational interpretation.
The idea that information can be localized is pervasive in physics. In thermodynamics, entropy flows from one reservoir to another, and in quantum gravity, the entropy content of some spatial regions is bounded. In quantum information theory, information is transmitted from one place to another, and entanglement can provide an advantage when there are constraints on such transmission.
Shannon entropy quantifies localizable information in the sense that marginalization defines an outer measure on the set of parts of a classical multipartite system. In contrast, von Neumann entropy does not quantify localizable information in this sense. However, a variant of von Neumann entropy, which originates in noncommutative geometry, does.
We prove that this adjusted von Neumann entropy is the minimum quantum entropy that quantifies localizable information. We also prove that this is the unique quantum entropy that characterizes Bell states in terms of redundant information, directly generalizing the classical case. We work with finite-dimensional C∗-algebras throughout, modeling finite quantum systems that may have superselection sectors.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
Classical information theory, built on Shannon entropy, and quantum information theory, built on von Neumann entropy, quantify uncertainty and compression properties of probability distributions and density operators. Both are powerful, but both treat information as a property of states of individual systems or ensembles. In this manuscript we argue that entanglement—the pattern of non-classical correlations between systems—is more fundamental than either Shannon or von Neumann information. We motivate this claim with a condensate thought experiment in which many bosons occupy the same single-particle state: Shannon and von Neumann entropies assign them identical information content, yet they can be distinguished by their distinct entanglement histories with external systems. Formally, states with identical marginal density operators but different purifications represent different global information, accessible only through their entanglement structure. Entropic quantities are then seen as coarse-grained summaries of this underlying relational information. We connect this perspective to an entanglement-geometric view of physics, including Entanglement Manifold Cosmology (EMC), in which spacetime and physical “truth” emerge from the geometry of correlations. The goal is an information-theoretic narrative that is rigorous enough for physics and information theory while readable to a non-specialist who is comfortable with basic linear algebra and quantum mechanics.
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality (IC) recently proposed by Pawlowski et al (2009 arXiv:0905.2292). We consider two entropic quantities, which we term measurement and mixing entropy. In the context of classical and quantum theory, these coincide, being given by the Shannon and von Neumann entropies, respectively; in general, however, they are very different. In particular, while measurement entropy is easily seen to be concave, mixing entropy need not be. In fact, as we show, mixing entropy is not concave whenever the state space is a non-simplicial polytope. Thus, the condition that measurement and mixing entropies coincide is a strong constraint on possible theories. We call theories with this property monoentropic . Measurement entropy is subadditive, but not in general strongly subadditive. Equivalently, if we define the mutual information between two systems A and B by the usual formula I ( A : B )= H ( A )+ H ( B )- H ( AB ), where H denotes the measurement entropy and AB is a non-signaling composite of A and B , then it can happen that I ( A : BC )< I ( A : B ). This is relevant to IC in the sense of Pawlowski et al : we show that any monoentropic non-signaling theory in which measurement entropy is strongly subadditive, and also satisfies a version of the Holevo bound, is informationally causal, and on the other hand we observe that Popescu–Rohrlich boxes, which violate IC, also violate strong subadditivity. We also explore the interplay between measurement and mixing entropy and various natural conditions on theories that arise in quantum axiomatics.