- Case 1:
If X is a DMS then, each xi are statistically independent. Therefore
- Case 2:
If X is not a DMS, i.e., X has memory then, its Entropy Rate may be defined as
- Case 3:
If X is an Ergodic source, then
An Ergodic source
Ergodic source are source whose
- Probabilities don't change with time, i.e., stationary.
- For any and all of its statistics,
Time average = Ensemble average
Asymptotic Equipartition Property (AEP) makes lossless data compression possible. Lossless data compression is called data compaction.❹
Cover, T. M., & Thomas, J. A. (2006). Lemma 16.8.1. In 2nd Edition, Elements of Information Theory (pp. 646-647). Hoboken, New Jersey: John Wiley & Sons, Inc.
More on AEP (p:2) ➽