In our previous papers, we proposed an analysis of the relations between information theory and observation theory. Referring to the parameters of the observation and using Shannon's formulation, we defined and calculated the information associated with a result of observation and the entropy of a random variable for a process of observation. This paper presents an extension of this theory for other definitions of entropy (Onicescu, hyperbolic). The method allows in particular to define Onicescu and hyperbolic entropies of continuous random variables. We examine the main properties of these new functions and propose a comparative analysis.
In our previous papers, we proposed an analysis of the relations between information theory and observation theory. Referring to the parameters of the observation and using Shannon's formulation, we defined and calculated the information associated with a result of observation and the entropy of a random variable for a process of observation. This paper presents an extension of this theory for other definitions of entropy (Onicescu, hyperbolic). The method allows in particular to define Onicescu and hyperbolic entropies of continuous random variables. We examine the main properties of these new functions and propose a comparative analysis.
An information measure inspired by Onicescu's information energy and Uffink's information measure (recently discussed by Brukner and Zeilinger) are calculated as functions of the number of particles N for fermionic systems (nuclei and atomic clusters) and correlated bosonic systems (atoms in a trap). Our results are compared with previous ones obtained for Shannon's information entropy, where a universal property was derived for atoms, nuclei, atomic clusters and correlated bosons. It is indicated that Onicescu's and Uffink's definitions are finer measures of information entropy than Shannon's.
The concept of entropy rst arose in the study of thermodynamics and statistical me- chanics. In information theory the so-called Shannon entropy plays a signi cant role, which measures how much information we gain, on average, when we learn the value of a random variable. An alternative interpretation is that the entropy of the random variable quanti es the amount of uncertainty about the random variable before we learn its value. The concept was introduced by Claude E. Shannon in his 1948 paper "A Mathematical Theory of Communication". In the rst part of the present work we consider this fundamental measure of information and show some of its properties, such as non-negativity. The relative entropy or the Kullback-Leibler divergence is a very useful measure of the distance between two prob- ability distributions. After introducing its de nition and other notions (joint entropy, conditional entropy or mutual information) we can prove additional assertions about the Shannon entropy and present statements about the relations between the above men- tioned quantities. In quantum information theory, there is a quantity, namely von Neumann entropy, which corresponds to Shannon entropy and originates from it. In the section Von Neu- mann entropy we will introduce this quantity which is a continuous functional on the states, or on their mathematical representatives, the density operators, and it ful ls in- equalities such as subadditivity and strong subaddititvity. Just as with the Shannon entropy, it is extremely useful to de ne a quantum version of the relative entropy, which is one of the most important numerical quantities appearing in quantum information theory. It is used as a measure of distinguishability between quantum states. Among others its properties are jointly convexity, monotonicity which are speci cally convenient. Transformations on structures which preserve operations, quantities or relations among elements appear in many areas of mathematics. These are the so-called preservers which show up also in physics and even in chemistry where they are usually called symmetries. Therefore, the study of preserver transformations is an important area of research. The third section of this work is based on a paper, in which Lajos Molnár described the structure every bijective map on the space of all density operators on a given Hilbert space, which preserves the relative entropy. This result is closely related to Wigner's famous theorem on the form of quantum mechanical symmetry transformations. The above theorem of Molnár can be extended into two directions. On the one hand, the result holds also in the case when the transformation under consideration acts on the space of all positive semide nite operators not only on density operators (i.e., we omit the condition of unit trace). On the other hand the statement remains valid even if we omit the strongly restrictive condition that the transformation is bijective. The second result is a joint work with Prof. Lajos Molnár which appeared in the journal Linear Algebra and Its Applications.
This paper introduces a novel definition of non-classical information entropy, rooted in the principles of statistical physics, particularly quantum mechanics. The core premise is that traditional information entropy, derived from classical probability theory, is insufficient to capture the essence of information within non-classical systems exhibiting phenomena such as superposition and entanglement. We propose a definition based on the concept of accessible states and their associated probabilities, extending the notion of entropy beyond the realm of purely deterministic systems. The paper details the mathematical formulation of this non-classical information entropy, denoted as *Hnc*, and presents a method for its calculation. The method relies on identifying the set of accessible states for a given system and then applying a generalized form of Shannon's entropy formula, modified to account for the inherent uncertainties associated with quantum mechanics. The resulting *Hnc* provides a quantitative measure of the information content within non-classical systems, offering a potential framework for studying information processing and dynamics in quantum environments. The key contribution lies in bridging the gap between information theory and non-classical statistical physics, offering a new lens through which to understand information in quantum systems.
Our way to deal with the given topic is a connection of both the mathematical definitions of information entropies and their mutual relations within a system of stochastic quantities especially with thermodynamic entropies defined on an isolated system in which a realization of our (repeatable) observation is performed [it is a (cyclic) transformation of heat energy of an observed, measured system].We use the information description to analyze Gibbs paradox reasoning it as a property of such observation, measuring of an (equilibrium) thermodynamic system.We state a logical proof of the II. P.T. as a derivation of relations among the entropies of a system of stochastic variables, realized physically, and, the Equivalence Principle of the I., II. and III. Principle of Thermodynamics is formulated.
We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: (1) classical probabilities and random variables; (2) quantum probabilities and observable operators; (3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes, that accounts for the main information functions: entropy, mutual-informations at all orders, and Kullback–Leibler divergence and generalizes them in several ways. The article is divided into two parts, that can be read independently. In the first part, the introduction, we provide an overview of the results, some open questions, future results and lines of research, and discuss briefly the application to complex data. In the second part we give the complete definitions and proofs of the theorems A, C and E in the introduction, which show why entropy is the first homological invariant of a structure of information in four contexts: static classical or quantum probability, dynamics of classical or quantum strategies of observation of a finite system.
Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in various fields. This paper explores the information-theoretic aspects of complex probability, focusing on its definition, properties, and applications. We extend Shannon entropy to complex probability and examine key properties, including maximum entropy, joint entropy, conditional entropy, equilibration, and cross entropy. These results offer a framework for understanding entropy in complex probability spaces and have potential applications in fields such as statistical mechanics and information theory.