by convention to be equal to zero whenever . This is justified because for any logarithmic base.

Entropy of an information source

Based on the probability mass function of a source, the Shannon entropy H, in units of bits per symbol, is defined as the expected value of the information content of the symbols.

The amount of information conveyed by an individual source symbol with probability is known as its self-information

1.Previous
3.Next