Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in various fields. This paper explores the information-theoretic aspects of complex probability, focusing on its definition, properties, and applications. We extend Shannon entropy to complex probability and examine key properties, including maximum entropy, joint entropy, conditional entropy, equilibration, and cross entropy. These results offer a framework for understanding entropy in complex probability spaces and have potential applications in fields such as statistical mechanics and information theory.
Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in various fields. This paper explores the information-theoretic aspects of complex probability, focusing on its definition, properties, and applications. We extend Shannon entropy to complex probability and examine key properties, including maximum entropy, joint entropy, conditional entropy, equilibration, and cross entropy. These results offer a framework for understanding entropy in complex probability spaces and have potential applications in fields such as statistical mechanics and information theory.
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated extension of a classical probability measure. Building upon this foundation, we propose three fundamental information-theoretic quantities: complex entropy, which quantifies distribution uniformity through phase coherence; complex divergence, an asymmetric measure of dissimilarity between distributions; and the complex metric, a symmetric distance function satisfying the triangle inequality. We establish these concepts rigorously for both continuous and discrete probability distributions, proving key properties such as boundedness, continuity under total variation convergence, and clear extremal behaviors. A detailed comparative analysis with classical measures (Shannon entropy and Kullback-Leibler divergence) highlights the unique geometric and interpretive advantages of the proposed framework, particularly its sensitivity to distributional shape via a tunable phase parameter. We elucidate a profound formal analogy between the complex entropy integral and Feynman's path integral formulation of quantum mechanics, suggesting a deeper conceptual bridge. Finally, we demonstrate the practical utility of the complex metric through a detailed application in nonparametric two-sample hypothesis testing, outlining the testing procedure, advantages, limitations, and providing a conceptual simulation. This work opens new avenues for analyzing probability distributions through the lens of complex analysis and interference phenomena, with potential impacts across information theory, statistical inference, and machine learning.
Jaynes' maximum entropy (MaxEnt) principle selects, from all probability distributions consistent with a set of constraints, the one that maximizes the Shannon entropy. The principle is well-defined for classical, real-valued probabilities, but extending it to complex-valued probability theories requires care. Youssef's complex probability framework, in which quantum mechanics is reformulated as a Bayesian theory with complex amplitudes and the observable probability is the squared modulus (Born rule), does not admit a natural MaxEnt extension: the L2 normalization of that framework is incompatible with the L1 normalization assumed by Shannon entropy, and the resulting complex-valued entropy functional cannot be maximized without additional structure absent from Youssef's axioms. The present paper proposes a different extension of Kolmogorov's axioms to the complex domain, replacing non-negativity with the condition that the real part of every probability be non-negative. This framework, which retains L1 normalization and recovers classical probability when the imaginary part vanishes, admits a natural MaxEnt principle: the real part of the complex Shannon entropy is a well-posed real functional, its Euler-Lagrange equations produce an explicit complex Gibbs distribution with complex Lagrange multipliers, and the classical Gibbs distribution is recovered as the special case of real multipliers. The die problem (Jaynes' Brandeis dice example) is worked out in detail for both the fair and the loaded cases, illustrating how imaginary constraints redistribute the observable probability mass in a way that has no counterpart in classical MaxEnt.
This paper introduces a novel definition of non-classical information entropy, rooted in the principles of statistical physics, particularly quantum mechanics. The core premise is that traditional information entropy, derived from classical probability theory, is insufficient to capture the essence of information within non-classical systems exhibiting phenomena such as superposition and entanglement. We propose a definition based on the concept of accessible states and their associated probabilities, extending the notion of entropy beyond the realm of purely deterministic systems. The paper details the mathematical formulation of this non-classical information entropy, denoted as *Hnc*, and presents a method for its calculation. The method relies on identifying the set of accessible states for a given system and then applying a generalized form of Shannon's entropy formula, modified to account for the inherent uncertainties associated with quantum mechanics. The resulting *Hnc* provides a quantitative measure of the information content within non-classical systems, offering a potential framework for studying information processing and dynamics in quantum environments. The key contribution lies in bridging the gap between information theory and non-classical statistical physics, offering a new lens through which to understand information in quantum systems.
Abstract It is common knowledge that the Wigner function of a quantum state may admit negative values, so that it cannot be viewed as a genuine probability density. Here, we examine the difficulty in finding an entropy-like functional in phase space that extends to negative Wigner functions and then advocate the merits of defining a complex-valued entropy associated with any Wigner function. This quantity, which we call thecomplex Wigner entropy, is defined via the analytic continuation of Shannon’s differential entropy of the Wigner function in the complex plane. We show that the complex Wigner entropy enjoys interesting properties, especially its real and imaginary parts are both invariant under Gaussian unitaries (displacements, rotations, and squeezing in phase space). Its real part is physically relevant when considering the evolution of the Wigner function under a Gaussian convolution, while its imaginary part is simply proportional to the negative volume of the Wigner function. Finally, we define the complex-valued Fisher information of any Wigner function, which is linked (via an extended de Bruijn’s identity) to the time derivative of the complex Wigner entropy when the state undergoes Gaussian additive noise. Overall, it is anticipated that the complex plane yields a proper framework for analyzing the entropic properties of quasiprobability distributions in phase space.
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of Rényi entropies, parameterized by a real number α. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under bistochastic maps conditioned on the side information. We prove that the most general form of conditional entropy is captured by a family of conditional entropies that are averages of Rényi entropies of the conditioned distribution and parameterized by a probability measure on the positive reals.
Chapter 7 discusses quantum conditional entropy, extending the concept of conditional majorization and introducing the notion of negative quantum conditional entropy. The chapter starts with the basic definition of conditional entropy, exploring its key properties like monotonicity and additivity. It further delves into the concepts of conditional min- and max-entropies, emphasizing their roles in quantifying uncertainty in quantum states and their operational significance in quantum information theory. The text presents conditional entropy as a measure sensitive to the effects of entanglement, showing that negative conditional entropy is a distinctive feature of quantum systems, contrasting with the classical domain where entropy values are nonnegative. This negativity is particularly pronounced in the context of maximally entangled states and is connected to the fundamental differences between classical and quantum information processing. Moreover, the chapter includes theorems and exercises to solidify understanding, like the invariance of conditional entropy under local isometric channels and its reduction to entropy for product states. It concludes by underscoring the inevitability of negative conditional entropy in quantum systems, a topic of both theoretical and practical importance in the quantum domain.