Entropy

• A mathematical term

• Measures “the actual amount of information”

• English sentences have about 1.5 bits per character

  • therefore, a passphrase for a 128 bit key would be about 80 characters long!

• Relates to “predictability” and so is relevant to security

  • you have no security if your secret can be guessed

Previous slide

Next slide

Back to first slide

View graphic version

I’m avoiding too much mathematics, but the actual formula is pretty simple. If there is a set of N possibilities, and one of them must occur, let p(i) be the probability of the ith one. Then the entropy is:

_-p(i)log2(p(i))

It turns out that this (rounded up) is the minimum number of bits which can represent all of the possibilities.

There are other definitions of entropy, but this is the one that is meant in this tutorial.

A great reference for this stuff is “Basic Concepts in Information Theory and Coding” (subtitled “the Adventures of Secret Agent 00111”) by Golomb, Peile, and Scholtz. It is surprisingly readable.