Professional Documents
Culture Documents
Comsysf311 L39
Comsysf311 L39
Comsysf311 L39
Channel Capacity:
Discrete and AWGN
• Entropy: H(X) = 1
P
i P (xi ) log2 P (xi )
bits per message
• Entropy: H(X) = 1
P
i P (xi ) log2
P (xi )
bits per message
• Entropy: H(Y ) = 1
P
j P (yj ) log2 P (yj ) bits per message
• Entropy: H(X) = 1
P
i P (xi ) log2
P (xi )
bits per message
• Entropy: H(Y ) = 1
P
j P (yj ) log2 P (yj ) bits per message
• Condition entropy:
1
P P
H(X|Y ) = i j P (xi , yj ) log2 P (xi |yj )
bits per
message
• Entropy: H(X) = 1
P
i P (xi ) log2
P (xi )
bits per message
• Entropy: H(Y ) = 1
P
j P (yj ) log2 P (yj ) bits per message
• Condition entropy:
1
P P
H(X|Y ) = i j P (xi , yj ) log2 P (xi |yj )
bits per
message
• Condition entropy:
1
P P
H(Y |X) = j i P (yj , xi ) log2 P (yj |xi )
bits per
message
R∞ 1
• Entropy: H(X) = −∞
fX (x) log2 fX (x)
dx bits per
message
R∞
• Entropy: H(X) = −∞ fX (x) log2 fX1(x) dx bits per
message
• Condition entropy:
R∞ R∞
H(X|Y ) = −∞ −∞ fXY (x, y) log2 fX|Y1(x|y) dxdy bits
per message
• Mutual information:
I(X : Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X)
• Mutual information:
I(X : Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X)
• I(X : Y ) = i j P (xi , yj ) log2 PP(x(x)P
i ,yj )
P P
(y )i j
• Mutual information:
I(X : Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X)
• I(X : Y ) = i j P (xi , yj ) log2 PP(x(x)P
i ,yj )
P P
i (yj )
j |xi )
• I(X : Y ) = i j P (xi )P (yj /xi ) log2 P PP(x(y)P
P P
i (yj |xi )
i
• Mutual information:
I(X : Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X)
• Mutual information:
I(X : Y ) = H(X) − H(X|Y ) = H(Y ) − H(Y |X)
R∞ R∞ f (x,y)
• I(X : Y ) = −∞ f (x, y) log2 fXX,Y
−∞ XY (x)fY (y)
dxdy bits
per message
• C = B log2 (1 + S
N0 B
), where N0 : PSD, N : Power.
• C = B log2 (1 + S
N0 B
), where N0 : PSD, N : Power.
• limB → ∞C = limB → ∞B log2 (1 + S
N0 B
)
• C = B log2 (1 + S
N0 B
), where N0 : PSD, N : Power.
• limB → ∞C = limB → ∞B log2 (1 + S
N0 B
)
• limB → ∞B log2 (1 NSB ) =
0
limB → ∞ NS0 [ NS0 B log2 (1 + S
N0 B
)]
• C = B log2 (1 + S
N0 B
), where N0 : PSD, N : Power.
• limB → ∞C = limB → ∞B log2 (1 + S
N0 B
)
• limB → ∞B log2 (1 NSB ) =
0
limB → ∞ NS0 [ NS0 B log2 (1 + S
N0 B
)]
S
• limB → ∞B log2 (1 + N0 B
) = log2 e NS0 = 1.44 NS0