Information Theory

How to measure just how wrong your predictions are — from intuition to the math. Surprise, Shannon entropy, cross-entropy, KL divergence, maximum-likelihood estimation, and the sampling tricks that pull real text out of a probability vector.