Shannon Entropy Calculator
Shannon entropy measures the average information (uncertainty) in a probability distribution. Enter comma-separated probabilities or raw counts and choose your unit — bits for information theory, nats for physics and machine learning, or hartleys for decimal digits.
Entropy unit
Average information per outcome (0 = certain, Hmax = fully random)
How does this calculator work?
Shannon entropy H = −Σ pᵢ log₂(pᵢ) measures average uncertainty in bits (or nats with ln, hartleys with log₁₀). Enter probabilities or counts; counts are auto-normalised. Maximum entropy log₂(n) occurs for a uniform distribution; efficiency H/Hmax shows how close you are to that maximum.
Formula
How this is calculated
Shannon entropy H = −Σ pᵢ log₂(pᵢ) quantifies the average surprise (information content) of drawing an outcome from a distribution. Each term −log₂(pᵢ) is the self-information of outcome i — how many bits it takes to encode that event. Rare events have high self-information; certain events have zero. H is the probability-weighted mean of these values.
For a fair coin (p = 0.5 each) H = 1 bit — the maximum for a two-outcome process. For a loaded coin (p = 0.9, 0.1) H ≈ 0.47 bits — less uncertainty. For n equally likely outcomes H = log₂(n), the maximum possible entropy, so efficiency H/Hmax tells you how close the distribution is to uniform.
The choice of logarithm base changes only the unit: log₂ gives bits (the standard in coding and information theory), natural log gives nats (used in physics and KL divergence), and log₁₀ gives hartleys (decimal digits of information). The ratios between them are fixed: 1 bit ≈ 0.693 nats ≈ 0.301 hartleys.
Frequently asked questions
Yes. Enter event counts (e.g. 50, 30, 20 for 100 observations) and the calculator normalises them to probabilities automatically. The resulting entropy is identical to using the proportions directly.
High entropy means the distribution is more uniform — outcomes are more equally likely, making it harder to predict the result. Maximum entropy (H = Hmax) occurs when all outcomes are equally probable. Zero entropy means one outcome has probability 1 (a certain event).
Cross-entropy is the standard loss function for classification, and KL divergence (relative entropy) measures how one distribution differs from another. Decision trees use information gain — the reduction in entropy after a split — to choose the best feature to split on.
Also known as
TG we-Calculate Editorial Team. (2026). Shannon Entropy Calculator [Online calculator]. TG we-Calculate. https://we-calculate.com/calculator/shannon-entropy-calculator
TG we-Calculate Editorial Team. "Shannon Entropy Calculator." TG we-Calculate. 2026. https://we-calculate.com/calculator/shannon-entropy-calculator.
TG we-Calculate Editorial Team, "Shannon Entropy Calculator," TG we-Calculate, 2026. [Online]. Available: https://we-calculate.com/calculator/shannon-entropy-calculator
@misc{wecalculate_shannon_entropy_calculator, title = {Shannon Entropy Calculator}, author = {{TG we-Calculate Editorial Team}}, howpublished = {\url{https://we-calculate.com/calculator/shannon-entropy-calculator}}, year = {2026}, note = {TG we-Calculate} }
Did this calculator help you?