Matdid 325610
Matdid 325610
Matdid 325610
Introduction
The most important continuous alphabet channel is the Gaussian channel
depicted in Figure. This is a time-discrete channel with output Yi at time i, where
Yi is the sum of the input Xi and the noise Zi . The noise Zi is drawn i.i.d. from a
Gaussian distribution with variance N. Thus,
Yi = Xi + Zi, Zi ∼ N(0,N).
1
Gaussian Channel
This channel is a model for some common communication channels, such as
wired and wireless telephone channels and satellite links.
Without further conditions, the capacity of this channel may be infinite. If the
noise variance is zero, the receiver receives the transmitted symbol perfectly.
Since X can take on any real value, the channel can transmit an arbitrary real
number with no error.
If the noise variance is nonzero and there is no constraint on the input, we can
choose an infinite subset of inputs arbitrarily far apart, so that they are
distinguishable at the output with arbitrarily small probability of error. Such a
scheme has an infinite capacity as well. Thus if the noise variance is zero or the
input is unconstrained, the capacity of the channel is infinite.
Power Limitation
The most common limitation on the input is an energy or power constraint.We
assume an average power constraint. For any codeword (x1, x2, . . . , xn)
transmitted over the channel, we require that:
1 n 2
∑ xi ≤ P
n i =1
The additive noise in such channels may be due to a variety of causes. However,
by the central limit theorem, the cumulative effect of a large number of small
random effects will be approximately normal, so the Gaussian assumption is valid
in a large number of situations.
2
Usage of the Channel
We first analyze a simple suboptimal way to use this channel. Assume that we
want to send 1 bit over the channel.
Given the power constraint, the best that we can do is to send one of two levels,
+√P or −√P. The receiver looks at the corresponding Y received and tries to
decide which of the two levels was sent.
Assuming that both levels are equally likely (this would be the case if we wish to
send exactly 1 bit of information), the optimum decoding rule is to decide that
+√P was sent if Y > 0 and decide −√P was sent if Y < 0.
Probability of Error
The probability of error with such a decoding scheme can be computed as
follows:
1 1
Pe = Pr(Y < 0 | X = + P ) + Pr(Y > 0 | X = − P )
2 2
1 1
= Pr( Z < − P | X = + P ) + Pr( Z > P | X = − P )
2 2
= Pr( Z > P )
= 1 − Φ( P N )
3
Using such a scheme, we have converted the Gaussian channel into a discrete
binary symmetric channel with crossover probability Pe.
Similarly, by using a four-level input signal, we can convert the Gaussian channel
into a discrete four input channel.
In some practical modulation schemes, similar ideas are used to convert the
continuous channel into a discrete channel. The main advantage of a discrete
channel is ease of processing of the output signal for error correction, but some
information is lost in the quantization.
Definitions
We now define the (information) capacity of the channel as the maximum of the
mutual information between the input and output over all distributions on the
input that satisfy the power constraint.
Definition The information capacity of the Gaussian channel with power constraint
P is
C = max2 I ( X ; Y )
f ( x ):EX ≤ P
4
We can calculate the information capacity as follows: Expanding I (X; Y), we
have:
since Z is independent of X.
Now, h(Z) = 1/2 log 2πeN, and: EY2 = E(X + Z)2 = EX2 + 2EXEZ + EZ2 = P
+ N, since X and Z are independent and EZ = 0.
Information Capacity
Applying this result to bound the mutual information, we obtain:
I ( X ; Y ) = h(Y ) − h( Z )
1 1
≤ log 2πe( P + N ) − log 2πeN
2 2
1 ⎛ P⎞
= log⎜1 + ⎟
2 ⎝ N⎠
We will now show that this capacity is also the supremum of the rates achievable
for the channel.
5
(M,n) Code for the Gaussian Channel
Definition An (M, n) code for the Gaussian channel with power constraint P
consists of the following:
∑x ( w) ≤ nP
2
w = 1, 2, . . .,M. i =1
i
The rate and probability of error of the code are defined as for the discrete case.
6
Capacity of the Gaussian Channel
Theorem The capacity of a Gaussian channel with power constraint
P and noise variance N is
1 ⎛ P⎞
C= log⎜1 + ⎟
2 ⎝ N⎠
bits per transmission.
If we assign everything within this sphere to the given codeword, when this
codeword is sent there will be an error only if the received vector falls outside the
sphere, which has low probability.
7
Capacity of the Gaussian Channel
Similarly, we can choose other codewords and their corresponding decoding
spheres.
These spheres are scattered throughout the space of received vectors. The
received vectors have energy no greater than n(P + N), so they lie in a sphere of
radius √n(P + N). The maximum number of nonintersecting decoding spheres in
this volume is no more than
Cn (n( P + N )) n / 2 ⎛ P⎞
= 2 n / 2 log⎜1 + ⎟
⎝ N⎠
n/2
Cn (nN )
√nP
√nN
√n(P+N)
8
Converse to the Coding Theorem for
Gaussian Channels
The capacity of a Gaussian channel is C = 1/2 log(1 + P/N ). In fact, rates R >
C are not achievable.
The proof parallels the proof for the discrete channel. The main new ingredient
is the power constraint.
Bandlimited Channels
A common model for communication over a radio network or a telephone line is
a bandlimited channel with white noise. This is a continuous-time channel. The
output of such a channel can be described as the convolution:
where X(t) is the signal waveform, Z(t) is the waveform of the white Gaussian
noise, and h(t) is the impulse response of an ideal bandpass filter, which cuts out
all frequencies greater than W.
9
Bandlimited Channels
We begin with a representation theorem due to Nyquist and Shannon which
shows that sampling a bandlimited signal at a sampling rate 1/2W is sufficient to
reconstruct the signal from the samples.
Nyquist Theorem
Theorem Suppose that a function f (t) is bandlimited to W, namely, the
spectrum of the function is 0 for all frequencies greater than W. Then the
function is completely determined by samples of the function spaced 1/2W
seconds apart.
10
Capacity of Bandlimited Channels
A general function has an infinite number of degrees of freedom—the value of
the function at every point can be chosen independently.
The values of the function at the sample points can be chosen independently,
and this specifies the entire function.
Assuming that the channel has bandwidth W, we can represent both the input
and the output by samples taken 1/2W seconds apart.
11
Capacity of Bandlimited Channels
If the noise has power spectral density N0/2 watts/hertz and bandwidth W hertz,
the noise has power N0/2 (2W) = N0W and each of the 2WT noise samples in
time T has variance N0WT/2WT = N0/2.
Looking at the input as a vector in the 2TW-dimensional space, we see that the
received signal is spherically normally distributed about this point with covariance
N0/2 (I) .
Let the channel be used over the time interval [0, T ]. In this case, the energy per
sample is PT/2WT = P/2W, the noise variance per sample is N0/2 (2W) T/2WT
= N0/2, and hence the capacity per sample is
1 ⎛ P / 2W ⎞ 1 ⎛ P ⎞
C= log⎜1 + ⎟⎟ = log⎜⎜1 + ⎟⎟
2 ⎜⎝ N0 / 2 ⎠ 2 ⎝ N 0W ⎠
bits per sample.
12
Capacity of Bandlimited Channels
Since there are 2W samples each second, the capacity of the channel can be
rewritten as:
⎛ P ⎞
C = W log⎜⎜1 + ⎟⎟
⎝ N 0W ⎠
This equation is one of the most famous formulas of information theory. It gives
the capacity of a bandlimited Gaussian channel with noise spectral density N0/2
watts/Hz and power P watts.
If we let W →∞ we obtain:
P
C= log 2 e
N0
as the capacity of a channel with an infinite bandwidth, power P, and noise
spectral density N0/2. Thus, for infinite bandwidth channels, the capacity grows
linearly with the power.
Practical modems achieve transmission rates up to 33,600 bits per second in both
directions over a telephone channel. In real telephone channels, there are other
factors, such as crosstalk, interference, echoes, and nonflat channels which must
be compensated for to achieve this capacity.
13
Example: Telephone Line
The V.90 modems that achieve 56 kb/s over the telephone channel achieve this
rate in only one direction, taking advantage of a purely digital channel from the
server to final telephone switch in the network.
In this case, the only impairments are due to the digital-to-analog conversion at
this switch and the noise in the copper link from the switch to the home.
These impairments reduce the maximum bit rate from the 64 kb/s for the digital
signal in the network to the 56 kb/s in the best of telephone lines.
The frequency response is far from flat over this band. If the entire bandwidth is
used, it is possible to send a few megabits per second through this channel.
Schemes such at DSL (Digital Subscriber Line) achieve this using special
equipment at both ends of the telephone line (unlike modems, which do not
require modification at the telephone switch).
14