Breaking down the information channel
Fig 1 Breaking down the information channel

Imagine an information source (a DMS X) with X = {0, 1}, p(x0) = 0.4, p(x1) = 0.6, H(X) = 0.97095 emitting symbol xi into an information channel. Now assume that the information channel is broken down to two parts resulting in a 3-level output (Y3) and a 2-level output (Y2).

A directed graph can then be shown for the part of the information channel prior to the break down as

Fig 2 Directed graph of information channel before break-up

❶

Because the information channel gets broken down, this directed graph can be sliced between the confounding zones. Therefore, slicing of the outputs creates the Erasure Channel (e, erasure) and the Binary Symmetric Channel (BSC).

Fig 3 Slicing the broken down information channel parts

The directed graphs for the two parts of the information channel are therefore

Fig 4 Directed graphs of the two information channels

❷

The mutual information for the respective channels are then

I(X; Y) for Y sup 3 = 0.77676 and I(X; Y) for Y sup 2 = 0.6901
Therefore mutual information
I(X;Y) of
Y3 Erasure Channel > Y2 BSC
because whenever we get "e" as the symbol, the system knows it is wrong and re-runs it for an output (0 or 1) in Y3 Erasure Channel.

❸

Next:

Why is I(X;Y3) > I(X;Y2) (p:4) ➽

✪