Cambridge University Press eBooks

Schumacher Compression

Mark M. Wilde

Published Apr 18, 2013• 1 citation

Abstract

One of the fundamental tasks in classical information theory is the compression of information. Given access to many uses of a noiseless classical channel, what is the best that a sender and receiver can make of this resource for compressed data transmission? Shannon's compression theorem demonstrates that the Shannon entropy is the fundamental limit for the compression rate in the IID setting (recall the development in Section 13.4). That is, if one compresses at a rate above the Shannon entropy, then it is possible to recover the compressed data perfectly in the asymptotic limit, and otherwise, it is not possible to do so. This theorem establishes the prominent role of the entropy in Shannon's theory of information. In the quantum world, it very well could be that one day a sender and a receiver would have many uses of a noiseless quantum channel available, and the sender could use this resource to transmit compressed quantum information. But what exactly does this mean in the quantum setting? A simple model of a quantum information source is an ensemble of quantum states { pX(x) , ∣ψ x ⟩}, i.e., the source outputs the state ∣ψ x ⟩ with probability pX(x) , and the states {∣ψ x ⟩} do not necessarily have to form an orthonormal basis. Let us suppose for the moment that the classical data x is available as well, even though this might not necessarily be the case in practice. A naive strategy for compressing this quantum information source would be to ignore the quantum states coming out, handle the classical data instead, and exploit Shannon's compression protocol from Section 13.4.

DOI: 10.1017/cbo9781139525343.018

Related papers

Classical Shannon Theory

Mark M. Wilde

We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”

Cambridge University Press eBooksNov 2, 20160 citations

Entanglement Concentration

Mark M. Wilde

Entanglement is one of the most useful resources in quantum information processing. If a sender and receiver share noiseless entanglement in the form of maximally entangled states, then Chapter 6 showed how they can teleport quantum bits between each other with the help of classical communication, or they can double the capacity of a noiseless qubit channel for transmitting classical information. We will see further applications in Chapter 20 where they can exploit noiseless entanglement to assist in the transmission of classical or quantum data over a noisy quantum channel. Given the utility of maximal entanglement, a reasonable question is to ask what a sender and receiver can accomplish if they share pure entangled states that are not maximally entangled. In the quantum Shannon-theoretic setting, we make the further assumption that the sender and receiver can share many copies of these pure entangled states. We find out in this chapter that they can “concentrate” these non-maximally entangled states to maximally entangled ebits, and the optimal rate at which they can do so in the asymptotic limit is equal to the “entropy of entanglement” (the von Neumann entropy of half of one copy of the original state). Entanglement concentration is thus another fundamental task in noiseless quantum Shannon theory, and it gives a different operational interpretation to the von Neumann entropy. Entanglement concentration is perhaps complementary to Schumacher compression in the sense that it gives a firm quantum information-theoretic interpretation of the term “ebit” (just as Schumacher compression did for the term “qubit”), and it plays a part in demonstrating how the entropy of entanglement is the unique measure of entanglement for pure bipartite states.

Cambridge University Press eBooksApr 18, 20130 citations

Aspects of Quantum Information Compression for Pure States

Richard Jozsa

Consider a source of pure quantum states with density matrix ρ and von Neumann entropy S . It is known that the quantum information of the source may be faithfully compressed to 5 qubits/signal (asymptotically for long sequences) and that this limit is optimal if we demand high fidelity for long strings of signals. The usual method for achieving this compression, based on the typical subspace of the source, requires knowledge of the density matrix. We describe a new “universal” compression scheme which does not even require knowledge of ρ. More precisely, suppose that the source is known to have von Neumann entropy less than some given bound S 0 but is otherwise completely unspecified. Our method will faithfully compress the quantum information of any such source to S 0 qubits/signal. It is known that S is always less than or equal to the Shannon entropy H of the prior distribution of the source. According to Shannon’s theorem, H provides the limit for compression of classical signals, so one may ask the question: what is the physical origin of the extra compression available in the quantum case compared to the classical case (where the prior probabilities are the same)? One may suggest that this effect is a consequence of possible non-orthogonality of signal states. However we demonstrate that it is possible to make each pair of signal states of a source more parallel (keeping the prior probabilities the same) while increasing the von Neumann entropy. This casts doubt on the conventional wisdom that overlap of quantum states is a measure of their (non-)distinguishability. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Kluwer Academic Publishers eBooksDec 19, 20051 citation

Entanglement-Assisted Quantum Data Compression

Zahra Baghali Khanian, Andreas Winter

Ask how the quantum compression of ensembles of pure states is affected by the availability of entanglement, and in settings where the encoder has access to side information. We find the optimal asymptotic quantum rate and the optimal tradeoff (rate region) of quantum and entanglement rates. It turns out that the amount by which the quantum rate beats the Schumacher limit, the entropy of the source, is precisely half the entropy of classical information that can be extracted from the source and side information states without disturbing them at all ("reversible extraction of classical information").In the special case that the encoder has no side information, or that she has access to the identity of the states, this problem reduces to the known settings of blind and visible Schumacher compression, respectively, albeit here additionally with entanglement assistance. We comment on connections to previously studied and further rate tradeoffs when also classical information is considered.

arXiv:1901.06346IEEE International Symposium on Information Theory (ISIT)Jul 1, 201917 citations

Lower Bound on Expected Communication Cost of Quantum Huffman Coding

Anurag Anshu, Ankit Garg, Aram W. Harrow +1 more

Data compression is a fundamental problem in quantum and classical information theory. A typical version of the problem is that the sender Alice receives a (classical or quantum) state from some known ensemble and needs to transmit them to the receiver Bob with average error below some specified bound. We consider the case in which the message can have a variable length and the goal is to minimize its expected length. For classical messages this problem has a well-known solution given by Huffman coding. In this scheme, the expected length of the message is equal to the Shannon entropy of the source (with a constant additive factor) and the scheme succeeds with zero error. This is a single-shot result which implies the asymptotic result, viz. Shannon's source coding theorem, by encoding each state sequentially. For the quantum case, the asymptotic compression rate is given by the von-Neumann entropy. However, we show that there is no one-shot scheme which is able to match this rate, even if interactive communication is allowed. This is a relatively rare case in quantum information theory when the cost of a quantum task is significantly different than the classical analogue. Our result has implications for direct sum theorems in quantum communication complexity and one-shot formulations of Quantum Reverse Shannon theorem.

Leibniz international proceedings in informaticsJan 1, 20160 citations

Classical Communication

Mark M. Wilde

This chapter begins our exploration of “dynamic” information-processing tasks in quantum Shannon theory, where the term “dynamic” indicates that a quantum channel connects a sender to a receiver and their goal is to exploit this resource for communication. We specifically consider the scenario where a sender Alice would like to communicate classical information to a receiver Bob, and the capacity theorem that we prove here is one particular generalization of Shannon's noisy channel coding theorem from classical information theory (overviewed in Section 2.2). In later chapters, we will see other generalizations of Shannon's theorem, depending on what resources are available to assist their communication or depending on whether they are trying to communicate classical or quantum information. For this reason and others, quantum Shannon theory is quite a bit richer than classical information theory. The naive approach to communicate classical information over a quantum channel is for Alice and Bob simply to mimic the approach used in Shannon's noisy channel coding theorem. That is, they select a random classical code according to some distribution pX(x) , and Bob performs individual measurements of the outputs of a noisy quantum channel according to some POVM. The POVM at the output induces some conditional probability distribution pY∣X(y∣x) , which we can in turn think of as an induced noisy classical channel. The classical mutual information I(X; Y) of this channel is an achievable rate for communication, and the best strategy for Alice and Bob is to optimize the mutual information over all of Alice's inputs to the channel and over all measurements that Bob could perform at the output.

Cambridge University Press eBooksApr 18, 20130 citations