This chapter looks at the basics of classical information theory. While information theory provides an answer to the question how fast one can send information over a noisy channel, it usually does not give a constructive solution to this task. This is a problem error correction deals with. For the correction of error, the chapter discusses linear block codes. It illustrates the general layout of a general communication system. Given a channel and an information source, the basic problem is to transmit the messages produced by the information source through the channel as efficient and as reliable as possible. In his “A Mathematical Theory of Communication”, Claude Shannon has shown that one can treat these two problems separately. First, a noiseless channel is considered, which transmits every input perfectly, and then noisy channels are considered.
This chapter looks at the basics of classical information theory. While information theory provides an answer to the question how fast one can send information over a noisy channel, it usually does not give a constructive solution to this task. This is a problem error correction deals with. For the correction of error, the chapter discusses linear block codes. It illustrates the general layout of a general communication system. Given a channel and an information source, the basic problem is to transmit the messages produced by the information source through the channel as efficient and as reliable as possible. In his “A Mathematical Theory of Communication”, Claude Shannon has shown that one can treat these two problems separately. First, a noiseless channel is considered, which transmits every input perfectly, and then noisy channels are considered.
We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”
This chapter begins our exploration of “dynamic” information-processing tasks in quantum Shannon theory, where the term “dynamic” indicates that a quantum channel connects a sender to a receiver and their goal is to exploit this resource for communication. We specifically consider the scenario where a sender Alice would like to communicate classical information to a receiver Bob, and the capacity theorem that we prove here is one particular generalization of Shannon's noisy channel coding theorem from classical information theory (overviewed in Section 2.2). In later chapters, we will see other generalizations of Shannon's theorem, depending on what resources are available to assist their communication or depending on whether they are trying to communicate classical or quantum information. For this reason and others, quantum Shannon theory is quite a bit richer than classical information theory. The naive approach to communicate classical information over a quantum channel is for Alice and Bob simply to mimic the approach used in Shannon's noisy channel coding theorem. That is, they select a random classical code according to some distribution pX(x) , and Bob performs individual measurements of the outputs of a noisy quantum channel according to some POVM. The POVM at the output induces some conditional probability distribution pY∣X(y∣x) , which we can in turn think of as an induced noisy classical channel. The classical mutual information I(X; Y) of this channel is an achievable rate for communication, and the best strategy for Alice and Bob is to optimize the mutual information over all of Alice's inputs to the channel and over all measurements that Bob could perform at the output.
An upper limit is given to the amount of quantum information that can be transmitted reliably down a noisy, decoherent quantum channel using the high-probability states of quantum sources. A class of quantum error-correcting codes is presented that allows the information transmitted to attain this limit. The result is a quantum analog of Shannon's bound and code for the noisy classical channel [C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illinois Press, Chicago, 1948)].
This chapter begins our first exciting application of the postulates of the quantum theory to quantum communication. We study the fundamental, unit quantum communication protocols. These protocols involve a single sender, whom we name Alice, and a single receiver, whom we name Bob. The protocols are ideal and noiseless because we assume that Alice and Bob can exploit perfect classical communication, perfect quantum communication, and perfect entanglement. At the end of this chapter, we suggest how to incorporate imperfections into these protocols for later study. Alice and Bob may wish to perform one of several quantum information-processing tasks, such as the transmission of classical information, quantum information, or entanglement. Several fundamental protocols make use of these resources: 1. We will see that noiseless entanglement is an important resource in quantum Shannon theory because it enables Alice and Bob to perform other protocols that are not possible with classical resources only. We will present a simple, idealized protocol for generating entanglement, named entanglement distribution . 2. Alice may wish to communicate classical information to Bob. A trivial method, named elementary coding , is a simple way for doing so and we discuss it briefly. 3. A more interesting technique for transmitting classical information is superdense coding . It exploits a noiseless qubit channel and shared entanglement to transmit more classical information than would be possible with a noiseless qubit channel alone. […]
This chapter begins our first exciting application of the postulates of the quantum theory to quantum communication. We study the fundamental, unit quantum communication protocols. These protocols involve a single sender Alice and a single receiver Bob. The protocols are ideal and noiseless because we assume that Alice and Bob can exploit perfect classical communication, perfect quantum communication, and perfect entanglement. At the end of this chapter, we suggest how to incorporate imperfections into these protocols for later study. Alice and Bob may wish to perform one of several quantum informationprocessing tasks, such as the transmission of classical information, quantum information, or entanglement. Several fundamental protocols make use of these resources: 1. We will see that noiseless entanglement is an important resource in quantum Shannon theory because it enables Alice and Bob to perform other protocols that are not possible with classical resources only. We will present a simple, idealized protocol for generating entanglement, named entanglement distribution . 2. Alice may wish to communicate classical information to Bob. A trivial method, named elementary coding , is a simple way of doing so and we discuss it briefly. 3. A more interesting technique for transmitting classical information is superdense coding . It exploits a noiseless qubit channel and shared entanglement to transmit more classical information than would be possible with a noiseless qubit channel alone. 4. Finally, Alice may wish to transmit quantum information to Bob. A trivial method for her to do so is to exploit a noiseless qubit channel. However, it is useful to have other ways for transmitting quantum information because such a resource is difficult to engineer in practice. An alternative, surprising method for transmitting quantum information is quantum teleportation . The teleportation protocol exploits classical communication and shared entanglement to transmit quantum information. Each of these protocols is a fundamental unit protocol and provides a foundation for asking further questions in quantum Shannon theory. In fact, the discovery of these latter two protocols was the stimulus for much of the original research in quantum Shannon theory. One could take each of these protocols and ask about its performance if one or more of the resources involved is noisy rather than noiseless. Later chapters of this book explore many of these possibilities.
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.