We have in previous chapters on several occasions called attention to the quantum phenomenon of information. However, a precise discussion of the quantum effect on a communication channel has not been treated. Although it was in 1929 that Szilard described a basic concept of information from the statistical thermodynamics point of view, the concept was not actually pursued until 1948, by Shannon. As noted, the work by Shannon was originally based on a pure mathematical standpoint and seemed to have nothing to do with the physical sciences. However, the relationship between information theory and physical science was quickly recognized by Gabor in 1950, only a few years later. He advocated that information theory be treated as a branch of physics. But it was the work of Brillouin in the earlier 1950s that successfully established a profound relationship between physical entropy and information. Since then, work on the entropy principle of information has been presented in a broad range of applications. However, it was Stern’s work in 1960 that actually explained the quantum effect on a communication channel. Stern’s work essentially used Gabor’s time–frequency cells (logons) in a technique for approaching the maximum entropy of a photon source.
We have in previous chapters on several occasions called attention to the quantum phenomenon of information. However, a precise discussion of the quantum effect on a communication channel has not been treated. Although it was in 1929 that Szilard described a basic concept of information from the statistical thermodynamics point of view, the concept was not actually pursued until 1948, by Shannon. As noted, the work by Shannon was originally based on a pure mathematical standpoint and seemed to have nothing to do with the physical sciences. However, the relationship between information theory and physical science was quickly recognized by Gabor in 1950, only a few years later. He advocated that information theory be treated as a branch of physics. But it was the work of Brillouin in the earlier 1950s that successfully established a profound relationship between physical entropy and information. Since then, work on the entropy principle of information has been presented in a broad range of applications. However, it was Stern’s work in 1960 that actually explained the quantum effect on a communication channel. Stern’s work essentially used Gabor’s time–frequency cells (logons) in a technique for approaching the maximum entropy of a photon source.
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.
In these first few chapters, our aim is to establish a firm grounding so that we can address some fundamental questions regarding information transmission over quantum channels. This area of study has become known as “quantum Shannon theory” in the broader quantum information community, in order to distinguish this topic from other areas of study in quantum information science. In this text, we will use the terms “quantum Shannon theory” and “quantum information theory” somewhat interchangeably. We will begin by briefly overviewing several fundamental aspects of the quantum theory. Our study of the quantum theory, in this chapter and future ones, will be at an abstract level, without giving preference to any particular physical system such as a spin- 1/2 particle or a photon. This approach will be more beneficial for the purposes of our study, but, here and there, we will make some reference to actual physical systems to ground us in reality. You may be wondering, what is quantum Shannon theory and why do we name this area of study as such? In short, quantum Shannon theory is the study of the ultimate capability of noisy physical systems, governed by the laws of quantum mechanics, to preserve information and correlations. Quantum information theorists have chosen the name quantum Shannon theory to honor Claude Shannon, who single-handedly founded the field of classical information theory with a groundbreaking paper (Shannon, 1948). In particular, the name refers to the asymptotic theory of quantum information, which is the main topic of study in this book. Information theorists since Shannon have dubbed him the “Einstein of the information age.”1 The name quantum Shannon theory is fit to capture this area of study because we often use quantum versions of Shannon's ideas to prove some of the main theorems in quantum Shannon theory. We prefer the name “quantum Shannon theory” over such names as “quantum information science” or just “quantum information.” These other names are too broad, encompassing subjects as diverse as quantum computation, quantum algorithms, quantum complexity theory, quantum communication complexity, entanglement theory, quantum key distribution, quantum error correction, and even the experimental implementation of quantum protocols.
We cannot overstate the importance of Shannon's contribution to modern science. His introduction of the field of information theory and his solutions to its two main theorems demonstrate that his ideas on communication were far beyond the other prevailing ideas in this domain around 1948. In this chapter, our aim is to discuss Shannon's two main contributions in a descriptive fashion. The goal of this high-level discussion is to build up the intuition for the problem domain of information theory and to understand the main concepts before we delve into the analogous quantum information-theoretic ideas. We avoid going into deep technical detail in this chapter, leaving such details for later chapters where we formally prove both classical and quantum Shannontheoretic coding theorems. We do use some mathematics from probability theory (namely, the law of large numbers). We will be delving into the technical details of this chapter's material in later chapters (specifically, Chapters 10, 13, and 14). Once you have reached later chapters that develop some more technical details, it might be helpful to turn back to this chapter to get an overall flavor for the motivation of the development. Data Compression We first discuss the problem of data compression. Those who are familiar with the Internet have used several popular data formats such as JPEG, MPEG, ZIP, GIF, etc. All of these file formats have corresponding algorithms for compressing the output of an information source. A first glance at the compression problem might lead one to believe that it is not possible to compress the output of the information source to an arbitrarily small size, and Shannon proved that this is the case. This result is the content of Shannon's first noiseless coding theorem. An Example of Data Compression We begin with a simple example that illustrates the concept of an information source. We then develop a scheme for coding this source so that it requires fewer bits to represent its output faithfully. Suppose that Alice is a sender and Bob is a receiver. Suppose further that a noiseless bit channel connects Alice to Bob—a noiseless bit channel is one that transmits information perfectly from sender to receiver, e.g., Bob receives “0” if Alice transmits “0” and Bob receives “1” if Alice transmits “1.”
These notes (in French) provide a concise introduction to information theory, spanning its classical foundations: Shannon entropy, joint and conditional entropies, mutual information, statistical correlations, and Markov chains; as well as Shannon's source coding and channel capacity theorems, which govern the optimal compression and transmission of information through noisy channels. They also cover the quantum extensions of these concepts, including the density operator formalism, distances between quantum states, von Neumann entropy, coherent information, conditional entropy, entanglement theory, quantum discord, and quantum resource theory.
I nformation is an idea created in human brain. But information has physical aspects too. Emergence of new theory and technology in physical science always contribute to shape various theories, models and services in LIS. Shannon’s mathematical formula derives strength from the physical aspect of data or information and gives a new information theoretic approach to communication model by introducing concepts of bit and entropy. With the advancement of Quantum mechanics a new quantum mechanical model is under preparation. Phenomenon like quantum entanglement is successfully being used in computer technology which in turn helps reshape many LIS services. Quantum information science is a newly flourishing field which is expected to bring great advancements in information organization, processing, retrieval, automatic classification, NLP and other LIS related problems.
The most successful practical applications of quantum mechanics in information theory have been in the field of communication. This is a very old field that has taken inputs from engineering, pure mathematics and computer science apart from physics. It is only natural that application of quantum techniques here should be among the first to be considered and implemented. Modern technology in optical communication using laser light and optical fiber cables is sufficiently advanced that it can quickly be adapted to using photons as the quantum carriers of information.