The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits replacing bits. In the mentioned cases the information content can be quantified through Shannon's and von Neumann's entropy, respectively. Here we extend the definition of information content to operational probabilistic theories, and prove relevant properties such as the subadditivity and the relation between purity and information content of a state. We prove the consistency of the present notion of information content when applied to the classical and the quantum case. Finally, the relation with one of the notions of entropy that can be introduced in general probabilistic theories, the maximum accessible information, is given in terms of a lower bound.
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits replacing bits. In the mentioned cases the information content can be quantified through Shannon's and von Neumann's entropy, respectively. Here we extend the definition of information content to operational probabilistic theories, and prove relevant properties such as the subadditivity and the relation between purity and information content of a state. We prove the consistency of the present notion of information content when applied to the classical and the quantum case. Finally, the relation with one of the notions of entropy that can be introduced in general probabilistic theories, the maximum accessible information, is given in terms of a lower bound.
We address the problem of quantifying the information content of a source for an arbitrary information theory, where the information content is defined in terms of the asymptotic achievable compression rate. The functions that solve this problem in classical and quantum theory are Shannon's and von Neumann's entropy, respectively. However, in a general information theory there are three different functions that extend the notion of entropy, and this opens the question as to whether any of them can universally play the role of the quantifier for the information content. Here we answer the question in the negative, by evaluating the information content as well as the various entropic functions in a toy theory called Bilocal Classical Theory.
In this chapter, we discuss several information measures that are important for quantifying the amount of information and correlations that are present in quantum systems. The first fundamental measure that we introduce is the von Neumann entropy (or simply quantum entropy ). It is the quantum generalization of the Shannon entropy, but it captures both classical and quantum uncertainty in a quantum state.1 The quantum entropy gives meaning to the notion of an information qubit . This notion is different from that of the physical qubit, which is the description of a quantum state of an electron or a photon. The information qubit is the fundamental quantum informational unit of measure, determining how much quantum information is present in a quantum system. The initial definitions here are analogous to the classical definitions of entropy, but we soon discover a radical departure from the intuitive classical notions from the previous chapter: the conditional quantum entropy can be negative for certain quantum states. In the classical world, this negativity simply does not occur, but it takes on a special meaning in quantum information theory. Pure quantum states that are entangled have stronger-than-classical correlations and are examples of states that have negative conditional entropy. The negative of the conditional quantum entropy is so important in quantum information theory that we even have a special name for it: the coherent information. We discover that the coherent information obeys a quantum data-processing inequality, placing it on a firm footing as a particular informational measure of quantum correlations. We then define several other quantum information measures, such as quantum mutual information, that bear similar definitions as in the classical world, but with Shannon entropies replaced with quantum entropies. This replacement may seem to make quantum entropy somewhat trivial on the surface, but a simple calculation reveals that a maximally entangled state on two qubits registers two bits of quantum mutual information (recall that the largest the mutual information can be in the classical world is one bit for the case of two maximally correlated bits). We then discuss several entropy inequalities that play an important role in quantum information processing: the monotonicity of quantum relative entropy, strong subadditivity, the quantum data-processing inequalities, and continuity of quantum entropy.
The idea that information can be localized is pervasive in physics. In thermodynamics, entropy flows from one reservoir to another, and in quantum gravity, the entropy content of some spatial regions is bounded. In quantum information theory, information is transmitted from one place to another, and entanglement can provide an advantage when there are constraints on such transmission.
Shannon entropy quantifies localizable information in the sense that marginalization defines an outer measure on the set of parts of a classical multipartite system. In contrast, von Neumann entropy does not quantify localizable information in this sense. However, a variant of von Neumann entropy, which originates in noncommutative geometry, does.
We prove that this adjusted von Neumann entropy is the minimum quantum entropy that quantifies localizable information. We also prove that this is the unique quantum entropy that characterizes Bell states in terms of redundant information, directly generalizing the classical case. We work with finite-dimensional C∗-algebras throughout, modeling finite quantum systems that may have superselection sectors.
General probabilistic theories are designed to provide operationally the most general probabilistic models including both classical and quantum theories. In this paper, we introduce a systematic method to construct a series of entropies, all of which generalize Shannon entropy in classical systems and von Neumann entropy in quantum systems. Using these entropies, the Holevo bound, an upper bound of the accessible information from a quantum system, is generalized to hold in any general probabilistic theory.
This paper introduces a novel definition of non-classical information entropy, rooted in the principles of statistical physics, particularly quantum mechanics. The core premise is that traditional information entropy, derived from classical probability theory, is insufficient to capture the essence of information within non-classical systems exhibiting phenomena such as superposition and entanglement. We propose a definition based on the concept of accessible states and their associated probabilities, extending the notion of entropy beyond the realm of purely deterministic systems. The paper details the mathematical formulation of this non-classical information entropy, denoted as *Hnc*, and presents a method for its calculation. The method relies on identifying the set of accessible states for a given system and then applying a generalized form of Shannon's entropy formula, modified to account for the inherent uncertainties associated with quantum mechanics. The resulting *Hnc* provides a quantitative measure of the information content within non-classical systems, offering a potential framework for studying information processing and dynamics in quantum environments. The key contribution lies in bridging the gap between information theory and non-classical statistical physics, offering a new lens through which to understand information in quantum systems.
The von Neumann entropy of a quantum state is a central concept in physics and information theory, having a number of compelling physical interpretations. There is a certain perspective that the most fundamental notion in quantum mechanics is that of a quantum channel, as quantum states, unitary evolutions, measurements, and discarding of quantum systems can each be regarded as certain kinds of quantum channels. Thus, an important goal is to define a consistent and meaningful notion of the entropy of a quantum channel. Motivated by the fact that the entropy of a state ρ can be formulated as the difference of the number of physical qubits and the "relative entropy distance"between ρ and the maximally mixed state, here we define the entropy of a channel N as the difference of the number of physical qubits of the channel output with the "relative entropy distance"between N and the completely depolarizing channel. We prove that this definition satisfies all of the axioms, recently put forward by Gour [IEEE Trans. Inf. Theory 65, 5880 (2019)IETTAW0018-944810.1109/TIT.2019.2907989], required for a channel entropy function. The task of quantum channel merging, in which the goal is for the receiver to merge his share of the channel with the environment's share, gives a compelling operational interpretation of the entropy of a channel. The entropy of a channel can be negative for certain channels, but this negativity has an operational interpretation in terms of the channel merging protocol. We define Rényi and min-entropies of a channel and prove that they satisfy the axioms required for a channel entropy function. Among other results, we also prove that a smoothed version of the min-entropy of a channel satisfies the asymptotic equipartition property.