In information theory, the entropy of a random variable quantifies the average level of uncertainty or information associated with the variable's potential...

k_{B}} is the Boltzmann constant. The defining expression for entropy in the theory of information established by Claude E. Shannon in 1948 is of the form:...

In information theory, the conditional entropy quantifies the amount of information needed to describe the outcome of a random variable Y {\displaystyle...

In information theory, the Rényi entropy is a quantity that generalizes various notions of entropy, including Hartley entropy, Shannon entropy, collision...

in information theory are mutual information, channel capacity, error exponents, and relative entropy. Important sub-fields of information theory include...

using quantum information processing techniques. Quantum information refers to both the technical definition in terms of Von Neumann entropy and the general...

In information theory, joint entropy is a measure of the uncertainty associated with a set of variables. The joint Shannon entropy (in bits) of two discrete...

In the mathematical theory of probability, the entropy rate or source information rate is a function assigning an entropy to a stochastic process. For...

In information theory and statistics, negentropy is used as a measure of distance to normality. The concept and phrase "negative entropy" was introduced...

In statistics and information theory, a maximum entropy probability distribution has entropy that is at least as great as that of all other members of...

Differential entropy (also referred to as continuous entropy) is a concept in information theory that began as an attempt by Claude Shannon to extend...

arguing that the entropy of statistical mechanics and the information entropy of information theory are the same concept. Consequently, statistical mechanics...

In information theory, the entropy power inequality (EPI) is a result that relates to so-called "entropy power" of random variables. It shows that the...

inference techniques rooted in Shannon information theory, Bayesian probability, and the principle of maximum entropy. These techniques are relevant to any...

In information theory, the binary entropy function, denoted H ⁡ ( p ) {\displaystyle \operatorname {H} (p)} or H b ⁡ ( p ) {\displaystyle \operatorname...

message source Differential entropy, a generalization of Entropy (information theory) to continuous random variables Entropy of entanglement, related to...

In information theory, an entropy coding (or entropy encoding) is any lossless data compression method that attempts to approach the lower bound declared...

Entropic gravity, also known as emergent gravity, is a theory in modern physics that describes gravity as an entropic force—a force with macro-scale homogeneity...

concept of mutual information is intimately linked to that of entropy of a random variable, a fundamental notion in information theory that quantifies the...

In statistical mechanics, entropy is formulated as a statistical property using probability theory. The statistical entropy perspective was introduced...

In quantum information theory, quantum relative entropy is a measure of distinguishability between two quantum states. It is the quantum mechanical analog...

In information theory, information dimension is an information measure for random vectors in Euclidean space, based on the normalized entropy of finely...

distance Information gain in decision trees Information gain ratio Information theory and measure theory Jensen–Shannon divergence Quantum relative entropy Solomon...

entropy is an entropy measure used in quantum information theory. It is a generalization of the conditional entropy of classical information theory....

coined the term entropy. Since the mid-20th century the concept of entropy has found application in the field of information theory, describing an analogous...

December 2018). "Measuring Integrated Information: Comparison of Candidate Measures in Theory and Simulation". Entropy. 21 (1): 17. arXiv:1806.09373. Bibcode:2018Entrp...

(1998). "On characterization of entropy function via information inequalities". IEEE Transactions on Information Theory. 44 (4): 1440–1452. doi:10.1109/18...

Rate–distortion theory is a major branch of information theory which provides the theoretical foundations for lossy data compression; it addresses the...

In information theory, redundancy measures the fractional difference between the entropy H(X) of an ensemble X, and its maximum possible value log ⁡ (...

In quantum information theory, strong subadditivity of quantum entropy (SSA) is the relation among the von Neumann entropies of various quantum subsystems...