The field of Information Theory is founded on Claude Shannon's seminal ideas relating to entropy. Nevertheless, his well-known avoidance of meaning (Shannon, 1948) still persists to this day, so that Information Theory remains poorly connected to many fields with clear informational content and a dependence on semantics. Herein we propose an extension to Quantum Information Theory which, subject to constraints, applies quantum entanglement and information entropy as linguistic tools that model semantics through measures of both difference and equivalence. This extension integrates Denotational Semantics with Information Theory via a model based on distributional representation and partial data triples known as Corolla.
The field of Information Theory is founded on Claude Shannon's seminal ideas relating to entropy. Nevertheless, his well-known avoidance of meaning (Shannon, 1948) still persists to this day, so that Information Theory remains poorly connected to many fields with clear informational content and a dependence on semantics. Herein we propose an extension to Quantum Information Theory which, subject to constraints, applies quantum entanglement and information entropy as linguistic tools that model semantics through measures of both difference and equivalence. This extension integrates Denotational Semantics with Information Theory via a model based on distributional representation and partial data triples known as Corolla.
Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures and point out some of their commonalities and interpretations. In particular we find the application of information theory to distributional semantics useful. By modeling the distributional meaning of words as density operators rather than vectors, more of their semantic structure may be exploited. Furthermore, properties of and interactions between words such as ambiguity, similarity and entailment can be simulated more richly and intuitively when using methods from quantum information theory.
This paper introduces a novel approach to information theory, leveraging quantum entanglement entropy as a fundamental measure of information. We propose that entanglement entropy can effectively quantify the efficiency of information transmission and provides a theoretical foundation that surpasses the limitations of classical information theory. The core of this work lies in the integration of entanglement entropy with Shannon entropy, establishing a new framework for analyzing information transfer. We explore scenarios where classical information theory struggles to adequately describe the limits of communication due to quantum mechanical effects, specifically entanglement. The paper aims to provide a theoretical underpinning for quantum communication protocols and potentially inform the development of quantum computation. We define key terms and formulate core concepts, focusing on the relationship between entanglement entropy, information content, and the constraints imposed by quantum mechanics. The central argument is that by considering the inherent interconnectedness of entangled systems, we can gain a deeper understanding of information processing at the most fundamental level. ---
Information theory, introduced by Claude Shannon in 1948, provides a mathematical model, predominantly probabilistic, for addressing the challenges associated with measuring the amount of information in a message and efficiently and reliably storing the message (compression and protection). This theory has always been intricately connected to communication theory. Information theory not only relies on communication theory but also draws from fields such as statistics, statistical physics, and inference theory. In this chapter, a concise overview is presented, covering the mathematical foundations of classical and quantum information, their manipulation and associated qualifications. The fundamental concepts of classical information theory, including Shannon entropy, joint entropy, conditional entropy, and mutual information, are introduced. Additionally, basic concepts related to quantum information and quantum correlation measurements are presented.
The idea that information can be localized is pervasive in physics. In thermodynamics, entropy flows from one reservoir to another, and in quantum gravity, the entropy content of some spatial regions is bounded. In quantum information theory, information is transmitted from one place to another, and entanglement can provide an advantage when there are constraints on such transmission.
Shannon entropy quantifies localizable information in the sense that marginalization defines an outer measure on the set of parts of a classical multipartite system. In contrast, von Neumann entropy does not quantify localizable information in this sense. However, a variant of von Neumann entropy, which originates in noncommutative geometry, does.
We prove that this adjusted von Neumann entropy is the minimum quantum entropy that quantifies localizable information. We also prove that this is the unique quantum entropy that characterizes Bell states in terms of redundant information, directly generalizing the classical case. We work with finite-dimensional C∗-algebras throughout, modeling finite quantum systems that may have superselection sectors.
In these first few chapters, our aim is to establish a firm grounding so that we can address some fundamental questions regarding information transmission over quantum channels. This area of study has become known as “quantum Shannon theory” in the broader quantum information community, in order to distinguish this topic from other areas of study in quantum information science. In this text, we will use the terms “quantum Shannon theory” and “quantum information theory” somewhat interchangeably. We will begin by briefly overviewing several fundamental aspects of the quantum theory. Our study of the quantum theory, in this chapter and future ones, will be at an abstract level, without giving preference to any particular physical system such as a spin- 1/2 particle or a photon. This approach will be more beneficial for the purposes of our study, but, here and there, we will make some reference to actual physical systems to ground us in reality. You may be wondering, what is quantum Shannon theory and why do we name this area of study as such? In short, quantum Shannon theory is the study of the ultimate capability of noisy physical systems, governed by the laws of quantum mechanics, to preserve information and correlations. Quantum information theorists have chosen the name quantum Shannon theory to honor Claude Shannon, who single-handedly founded the field of classical information theory with a groundbreaking paper (Shannon, 1948). In particular, the name refers to the asymptotic theory of quantum information, which is the main topic of study in this book. Information theorists since Shannon have dubbed him the “Einstein of the information age.”1 The name quantum Shannon theory is fit to capture this area of study because we often use quantum versions of Shannon's ideas to prove some of the main theorems in quantum Shannon theory. We prefer the name “quantum Shannon theory” over such names as “quantum information science” or just “quantum information.” These other names are too broad, encompassing subjects as diverse as quantum computation, quantum algorithms, quantum complexity theory, quantum communication complexity, entanglement theory, quantum key distribution, quantum error correction, and even the experimental implementation of quantum protocols.
Classical information theory, built on Shannon entropy, and quantum information theory, built on von Neumann entropy, quantify uncertainty and compression properties of probability distributions and density operators. Both are powerful, but both treat information as a property of states of individual systems or ensembles. In this manuscript we argue that entanglement—the pattern of non-classical correlations between systems—is more fundamental than either Shannon or von Neumann information. We motivate this claim with a condensate thought experiment in which many bosons occupy the same single-particle state: Shannon and von Neumann entropies assign them identical information content, yet they can be distinguished by their distinct entanglement histories with external systems. Formally, states with identical marginal density operators but different purifications represent different global information, accessible only through their entanglement structure. Entropic quantities are then seen as coarse-grained summaries of this underlying relational information. We connect this perspective to an entanglement-geometric view of physics, including Entanglement Manifold Cosmology (EMC), in which spacetime and physical “truth” emerge from the geometry of correlations. The goal is an information-theoretic narrative that is rigorous enough for physics and information theory while readable to a non-specialist who is comfortable with basic linear algebra and quantum mechanics.