Basic Notions
- Entropy
- Differential entropy
- Graph entropy
- Conditional entropy
- Mutual information
- Kulback-Leibler divergence
- Typicality and typical sequences
- Information inequalities
- Data-processing inequality
- Fano's inequality
- Jensen's inequality
- Asymptotic equipartition property
- Log-sum inequality
- Non-Shannon information inequalities