Applied Information Theory

Where the math from Class 02 actually lives in modern systems. Distillation as KL between teacher and student, Chinchilla and the compute-optimal scaling laws, speculative decoding as a draft–verify trick, and L²M as the information-theoretic limit on long-context memory.