KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING

EC 8501 – DIGITAL COMMUNICATION

HANDOUTS
 KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY
DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING
EC 8501 – DIGITAL COMMUNICATION
UNIT – I
COURSE HANDOUTS

INFORMATION THEORY

1.1 INTRODUCTION:
Information is the source of a communication system, whether it is analog or
digital. Information theory is a mathematical approach to the study of coding of
information along with the quantification, storage, and communication of information.

CONDITIONS OF OCCURRENCE OF EVENTS

If we consider an event, there are three conditions of occurrence.

 If the event has not occurred, there is a condition of uncertainty.

 If the event has just occurred, there is a condition of surprise.

 If the event has occurred, a time back, there is a condition of having

some information.

These three events occur at different times. The difference in these conditions helps us gain
knowledge on the probabilities of the occurrence of events.

ENTROPY
When we observe the possibilities of the occurrence of an event, how surprising or uncertain
it would be, it means that we are trying to have an idea on the average content of the
information from the source of the event.

Entropy can be defined as a measure of the average information content per source
symbol. Claude Shannon, the “father of the Information Theory”, provided a formula for it
as

Where pi is the probability of the occurrence of character number i from a given stream of
characters and b is the base of the algorithm used. Hence, this is also called as Shannon’s
Entropy.

The amount of uncertainty remaining about the channel input after observing the channel
output, is called as Conditional Entropy. It is denoted by H(x∣y)H(x∣y)
Mutual Information
Let us consider a channel whose output is Y and input is X

Let the entropy for prior uncertainty be X = H(x)

(This is assumed before the input is applied)

To know about the uncertainty of the output, after the input is applied, let us consider
Conditional Entropy, given that Y = Yk

This is a random variable for H(X/Y=Y0)...............H(X/Y=YK)

H(X/Y=Y0)...............H(X/Y=YK)
With probabilities p(y0)............p(Yk−1)p(Y0)............p(Yk−1) respectively.

The mean value of H(X/Y=Yk)H(X/Y=Yk) for output alphabet y is

Now, considering both the uncertainty conditions (before and after applying the inputs), we
come to know that the difference, i.e. H(x)−H(x/y)H(x)−H(x/y) must represent the
uncertainty about the channel input that is resolved by observing the channel output.
This is called as the Mutual Information of the channel.
Denoting the Mutual Information as I(x;y) we can write the whole thing in an equation, as
follows
I(x;y)=H(x)−H(x | y)

Hence, this is the equational representation of Mutual Information.

Properties of Mutual information
These are the properties of Mutual information.
Mutual information of a channel is symmetric.
I(x;y)=I(y;x) Mutual information is non-negative.
I(x;y)≥0I(x;y)≥0
Mutual information can be expressed in terms of entropy of the channel output.
I(x;y)=H(y)−H(y | x) Where H(y | x) is a conditional entropy
Mutual information of a channel is related to the joint entropy of the channel input and the
channel output.
I(x;y)=H(x)+H(y)−H(x,y)
Where the joint entropy H(x,y) is defined by
CHANNEL CAPACITY:
We have so far discussed mutual information. The maximum average mutual information, in
an instant of a signaling interval, when transmitted by a discrete memoryless channel, the
probabilities of the rate of maximum reliable transmission of data, can be understood as
the channel capacity.
It is denoted by C and is measured in bits per channel use.

1.2 DISCRETE MEMORYLESS SOURCE

A source from which the data is being emitted at successive intervals, which is independent
of previous values, can be termed as discrete memory less source.
This source is discrete as it is not considered for a continuous time interval, but at discrete
time intervals. This source is memory less as it is fresh at each instant of time, without
considering the previous values.

The Code produced by a discrete memory less source, has to be efficiently represented,
which is an important problem in communications. For this to happen, there are code words,
which represent these source codes.
For example, in telegraphy, we use Morse code, in which the alphabets are denoted
by Marks and Spaces. If the letter E is considered, which is mostly used, it is denoted
by “.” Whereas the letter Q which is rarely used.

Let us take a look at the block diagram:

Where Sk is the output of the discrete memory less source and bk is the output of the source
encoder which is represented by 0s and 1s.

The encoded sequence is such that it is conveniently decoded at the receiver.

Let us assume that the source has an alphabet with k different symbols and that
the kth symbol Sk occurs with the probability Pk, where k = 0, 1…k-1.

Let the binary code word assigned to symbol Sk, by the encoder having length lk, measured
in bits.
Hence, we define the average code word length of the source encoder as

L represents the average number of bits per source symbol

 If L min= minimum possible value of

Then coding efficiency can be defined as

However, the source encoder is considered efficient when η=1

For this, the value Lmin has to be determined.

Let us refer to the definition, “Given a discrete memory less source of entropy H(δ) the
average code-word length for any source encoding is bounded as ≥H(δ)."

In simpler words, the code word (example: Morse code for the word QUEUE is -.- ..- . ..- . )
is always greater than or equal to the source code (QUEUE in example). Which means, the
symbols in the code word are greater than or equal to the alphabets in the source code

Hence with Lmin=H(δ), the efficiency of the source encoder in terms of Entropy H(δ) may
be written as

This source coding theorem is called as noiseless coding theorem as it establishes an error-
free encoding. It is also called as Shannon‟s first theorem.

The noise present in a channel creates unwanted errors between the input and the output
sequences of a digital communication system. The error probability should be very
low, nearly ≤ 10-6 for a reliable communication.

The channel coding in a communication system, introduces redundancy with a control, so as

to improve the reliability of the system. The source coding reduces redundancy to improve
the efficiency of the system.

Channel coding consists of two parts of action.

 Mapping incoming data sequence into a channel input sequence.

 Inverse Mapping the channel output sequence into an output data sequence.

The final target is that the overall effect of the channel noise should be minimized.

The mapping is done by the transmitter, with the help of an encoder, whereas the inverse
mapping is done by the decoder in the receiver.
1.3 Discrete Memoryless Channels
The focus in this section is on information transmission through a discrete memoryless
channel rather than on information generation through a DMS. A discrete channel is a
statistical model with an input X and an output Y. During each signaling interval (symbol
period), the channel accepts an input symbol from X, and in response, it generates an output
symbol from Y, generally a noisy version of X. The channel is discrete when the alphabets
of X and Y are both finite. X and Y in all practical channels are random variables. In a discrete
memoryless channel (DMC), the current output symbol depends only on the current input
symbol and not on any of the previous input symbols.

Channel Transition Probabilities

The input X consists of input symbols x1, x2, …, xJ and the a priori probabilities of these
source symbols are p(x1), p(x2), …, p(xJ), respectively, and are all assumed to be known. The
output Y consists of output symbols y1, y2, …, yM, and the transition probabilitiesp(ym/xj),
for m=1,2,…,M and j=1,2,…,J, are all assumed to be known as they describe a DMC. In
short, a DMC is completely defined by the J×M channel matrix consisting of all transition
probabilities. Since each input to the channel results in some output, we always have the
following:
∑m=1Mpym/xj=1,j=1,2,…,J (1)
The joint probability distribution of the random variables X and Y, i.e., the joint probability of
transmitting xj and receiving ym for all cases, are also given by:
pxj,ym=pym/xjpxj,j=1,2,…,J;m=1,2,…,M (2)
The marginal probability distribution of the output random variable is thus as follows:
pym=∑j=1Jpym/xjpxj,m=1,2,…,M (3)

BINARY SYMMETRIC CHANNEL

 A binary symmetric channel (or BSC) is a common communications channel

model used in coding theory and information theory.
 In this model, a transmitter wishes to send a bit (a zero or a one), and the
receiver receives a bit.
 It is assumed that the bit is usually transmitted correctly, but that it will be
"flipped" with a small probability (the "crossover probability").
 This channel is used frequently in information theory because it is one of the
simplest channels to analyze.

 A binary symmetric channel with crossover probability p denoted by, is a

channel with binary input and binary output and probability of error p; that is,
if X is the transmitted random variable and Y the received variable, then the
channel is characterized by the conditional probabilities

Pr( Y = 0 | X = 0 ) = 1 – p
 Pr( Y = 0 | X = 1) = p
Pr( Y = 1 | X = 0 ) = p
Pr( Y = 1 | X = 1 ) = 1 − p

 It is assumed that 0 ≤ p ≤ 1/2. If p > 1/2, then the receiver can swap the output
(interpret 1 when it sees 0, and vice versa) and obtain an equivalent channel
with crossover probability 1 − p ≤ 1/2.
 This channel is often used by theorists because it is one of the simplest noisy
channels to analyse. Many problems in communication theory can be reduced to
a BSC.
 Conversely, being able to transmit effectively over the BSC can give rise to
solutions for more complicated channels.

MEASURE OF INFORMATION
The basic goal of a communication system is to transmit and subsequently receive the desired
information as efficiently as possible. Therefore it is necessary to understand the concept of
information and the ways it can be measured. The definition of information based on the idea
of lack of information i.e., the less information one has the greater is the information to be
gained.thus information is a measure of uncertainty, i.e. the less is the possibility of
occurrence of a certain message, the higher is its information content.
For example, the weather forecast of a particular city stated as follows
i) It would be hot and sunny

ii) There would scattered rain

iii) There would be a cyclone with thunder storm

The amount of information received about the city is totally different for the three messages.
The first message contains very little information because the weather in a desert city in
summer is expected to be hot and sunny for maximum time. The second message forecasting
a scattered rain contains some more information because it is not an event that occurs often.
The forecast of a cylonic storm contains even more information compared to the second
message. This is because the third forecast is a rearest event in the city. Hence on an
conceptual basis the amount of information received from the knowledge of occurance of that
event may be related to the probability of occurance of that event.
If the uncertainty of choosing of a particular message is high, the message has high
information. Thus the information can also bedefined as the choice of one message out of a
finite set of messages.
A practical source in a communication system is a device which produces messages and it
can be either analog or discrete. In this chapter only discrete source have been discussed.
Since analog source can be transformed to discrete source. A information source is a source
which has only a finite set of symbols called as source alphabet and the elements of the set
are called as symbols or letters.
In any message there is some redundaancy, repetition, or periodicity occurs, the the efficiency
of the system is reduced. It can be improved only if all the redundancy is removed.
The mathematical measure of information should be function of the probability of the
outcome and it should satisfy the following
i) information should be proportional to the uncertainity of an outcome

ii) information caontained in independent outcomes should added.

Let us consider a source denoted by „x‟ having an alphabet {x1 x2 x3 . . . xm}

Since information is closely associated with the uncertainty of the occurrence of a particular
symbol. The information content of a symbol xi associated with its occurrence is defined as
 1
I(x i )=log =-logP(x i ) (1)
p(x i )
Where p(xi) = Probability of occurrence of even „E‟
x = Measure of uncertainty of event
I(xi) = Information carried.
If there are „n‟ possible events which are equally likely, the probability of any event is
1
p(E) = if there is only one event occurs then p(E) = 1;
n
Thus I(E) = log 1 = 0.
The computation of I(E) cannot be made until the base of the logarithm has been specified.
A base of two is most often employed in specifying information because of popularity of the
binart system. Then the unit of information defined is measured in bits per symbol.
Consider two equiprobable messages A and B transmitted using binary pulses, where pulse
(0) denotes the absence of pulse and occurrence of pulse (1) denotes the presence of the
message. Let us consider two pulses representing the message can have have four distinct
possibilities and therefore said to have 4 bits of information. In general, any of „N‟
equiprobable messages contain log N bits of information. i.e. a minimum of log2N binary
pulses are required to transmit such a message.
For example when an unbiased coin in tossed, the probability of getting a head or tail is ½.
Therefore the information carried by the symbol carrying the outcome of head toss is
I(E) = log22 = 1 bit/symbol

ENTROPY OR AVERAGE AND INFORMATION RATE

Messages produced by information sources consists of sequences of symbols that correspond

to the message. At the receiver the entire message is treated as a single unit, but the
communication system deals each message as symbols and alphabets. Thus it is necessary to
define the information content of symbol.
In order to obtain the information content of the symbol, we should note that the flow of
information in a system may flucuate widely due to randomness involved in the selection of
symbols in a long message.
The average amount of information per source symbol in a particular interval is called
entropy.
The entropy or average information transmitted are studied under the following assumption.
 The source is stationary i.e., the probability of occurrence of each message remains
constant with time.

 The successive symbols are stability independent i.e., each message emitted from the
source is independent of previous message.

Consider a source emitting „m‟ possible symbols x1, x2, x3, ………. xm with their
probabilities of occurrence is p(x1), p(x2) …., p(xm).
m
The total probability is P(x1) + P(x2) + ………+ P(xm) =  p(x )
i-1
1 (2)

For a long interval, „L‟ messages have been generated then the number of message xt = ptL.
1
The amount of information in message x1 = log thus the total amount of information in
p 1
all x1 message

1
= P(xt) L log (3)
p(x t )
The amount of information in all „L‟ messages will be
 1 1 1
I(t) = p(x1)Llog + p(x2)Llog +………….+ p(xm)Llog (4)
p(x1 ) p(x 2 ) p(x m )
The average information per message or entropy is given by

H(x) = I (x) = p(x1)log

t 1
+ p(x2)log
1
+………….+ p(xm)log
1
L p(x1 ) p(x 2 ) p(x m )

m
1
=  p(x
K=1
K )log
p(x K )
m m
1
in simple form H=  P log P =- P logP
K=1
K
K=1
K K (5)
k
If a source emitting a sequnce of „n‟ symbols, then the total information to be transmitted is
nH bits. Let us assume the source produces „r‟ symbols/sec, thus the time duration of the
equation can be written as

n
Tb =
r
The average rate at which the information must be transferred is called Information rate and
is given by

nH
R= = rH bits/sec (6)
nr
r
 Let us consider a two sources of equal entropy, generating r1 and r2 message per second,
respectively. The first source will transmit the information at a rate R 1 = r1H the
second source will transmit the information at a R2 = r2H.

 If r2 > r1 the R2 > R1 i.e., more information transmitted from the second source, in a given
period, placing greater demands on the communication channel. Thus the source is not
defined by its entropy alone but also its rate of information.

From, the equation (5) it can be concluded that, the entropy depends upon the symbol
probability pi and also on the alphabets size „m‟ However „H‟ satisfies the following
equation.

0  H  log2 M
H = 0 corresponds to n0 uncertainly or freedom of choice, this occurs when the cource emits
always only one symbol.
H = log2M corresponds to maximum freedom of choice or uncertainty, which occurs when P i
1
= for all i, i.e., all the symbols are equiprobable.
M
In order to study the variation of „H‟ between these two extremes consider the binary source,
it emits two symbols with their probability of one with „p‟ and other with q.
1.7. ENTROPY IN BINARY CASE

The symbol „0‟ is tranmitted with the probability „p‟ and another symbol „1‟ is transmitted
with the probability „q‟

therefore P + q = 1 or q = 1 – p

m
We know entropy H(x) = - P(x1 )logP(x1 )=-[plogp+qlogq].
i=1
 = - [(p log p + (1-p) log (1-p)]

To maximize H(x) differentiate the above equation with respect to „p‟ and equated to zero

dH(x)  1  1  
ie =0=-  P   logp+(1-p)   (-1)log(1-p) 
dP  P  1-p  
= - [log p – log (1-p)] = 0.

or log P = log (1-p)

1
or P = (1-p) therefore P=
2
1
The maximum value of H(x) occurs at P = i.e.,
2

At P = 0, and P = 1, H(x) = 0.

1
To find the maximum value of entropy substitute P = .
2

 1  1 1  1 
Then H(x) = -   log + log   
 2  2 2  2 

1
= - log    log 2 2 = 1 bit / symbol.
2
PROPERTIES OF ENTROPY

i) The entropy function is functionally symmetry in every PK

i.e., H(PK, PK-1) = H(PK-1, PK).
ii) H(K) is continuous in the inteval 0  PK  1
iii) 0  H(x)  log M
iv) H(x) = 0 if all probabilities are zero except for one, which must be unity.
v) H(x) = log M if all probabilities are equal.
vi) The entropy of the source Sn is „n‟ time that of primary source
i.e., H(Sn) = nH(s).
vii) Partitioning of entropy into different sub sets does not effect the value of
entropy H(x)
i.e H(x) = H(x1, x2, x3, ….. xm, xm+1, xm+2, ….xq).
= H(x1, x2, x3 …. xm) + H(xm+1, xm+2, ….xq)

JOINT AND CONDITIONAL ENTROPY

let us consider two spaces d1 and d2 and their product space will be S = S1S2.
A source S1 emits with „m‟ symbol of xt and source S2 emits with „n‟ symbol of yt.
X = xt = {x1, x2, x3 ….xm) with probability P(x1), P(x2), ……P(xm) and
Y = yt = {y1, y2, y3, ….ym) with probability P(y1), P(y2)….P(yn)
Then the complete set of event is S = S1 S2.
 x1 y1 x1 y2 ......x1 yn 
 x y x y ......x y 
 2 1 2 2 2 n 

 XY            

 
         
 x m y1 xm y2 ......xm yn 
 

Now we have set of complete probability schemes.

P(X) = P(xi)
P(Y) = P(yj)
P(XY) = P[xiyj]
When there are three probability schemes, there will be there associated entropies.
M
i.e., H(x) =  p(x i ) log p(x i )
i=1

m
where P(xi) =  p(x y )
j=1
i j

n
and H(Y) =  p(y j ) log p(y j )
j=1

m
where P(Yi) =  p(x i y j )
i=1

m n
similarly H(XY) =  p(x i y j ) log p(x i y j )
i=1 j=1

where H(X) and H(Y) are marginal entropies of „X‟ and „Y‟ and
H(XY) = joint entropy of „X‟ and „Y‟ respectively.
 X  P(XY)
The conditional probability p(X/Y) is given by P   =
 Y  P(Y)
We know that YK may occurs in conjunction with x1 x2 … xm

x x x 
Thus (X/yj) =  1 , 2 ,..... m 
 y j y j y j 

and its associated probability is

 x  x   x 
P(x/yj) =  P  1  , P  2  ,..... P  m  
  y j   y j   y 
 j 
  P(x1 y1 ) P(x 2 y j ) P(x m y j ) 
=  , ........ 
 p(y j ) p(y j ) p(y j ) 

If P(xyj) + P(x2yj) + …………+ P(xmyj) = P(yj) = 1

then the probability scheme is complete. Therefore an entropy may be associated with it is,
M P(x i y j ) P(x i yi ) m x  x 
H(X / yj) =  log = - P  i  logP  i 
P(y j ) P(y j ) y  y 
i=1 i=1  j  j
Take the average of this conditional entropy for al;l admissible values of yj, in order to obtain
a measure of an average conditional entropy of the system.
n
i.e H(X/Y) = H(X/y j ) =  P(y )H(X/y )
j=1
j j

n m
=  p(y j ) P(x i /y j )logp(x i /y j )
y=1 i=1

n m
=    p(y )P(x /y )logp(x /y )
i=1 j=1 j i j i j

m n
=  P (x i y j )logP(x i /y j )
i=1 j=1

m n
Similarly H(Y/X) =  P (x i y j )logP(yi /x j )
i=1 j=1

Where H(X / Y) and H(Y / X) are average conditional entropies.

There are five entropies associated with a 2D probability scheme they are (let X represent a
transmitter, „Y‟ represents receiver).
H(X) = Entropy of transmitter, it indicates the probabilistic nature of
the transmission.
H(Y) = Entropy of receiver, it indicate the probabilistic nature of
receiving the information.
H(XY) = Entropy of a communication system as a whole.
H(X / Y) = Represents the information lost in the channel ie, the
received character yj. When transmitting characterize xi or
measure of information about transmitter. Where it is known
that Y is received.
H(Y / X) = Measure of information about the receiver, where it is known
that „X‟ is transmitted it indicate a measure of error or noise.
RELATION BETWEEN DIFFERENT ENTROPIES
(i) if x, y are not independent then
m n
H (XY) =    P(x i y j )logP(x i y j )
i=1 j=1
using Baye‟s theorem P(xiyj) = P(xi / yj) p(yj) = P(yj / xi) P(xi). we get,
 m n
H(XY) =    P(x i y j )log P(x i /y j )P(y j ) 
i=1 j=1
m n
=    P(x i y j ) logP(x i /y j )+logP(y j ) 
i=1 j=1
m n
=   P(x i y j )log  x i /y j  +P(x i y j )logP(y j ) 
i=1 j=1

m n
= H(X/Y)  P(x i ,y j )logP(y j )
i=1 j=1

n
m 
= H(X/Y)   P(x i y j )  logP(y j )
j=1  i=1 
n
= H(X/Y)  P(y j )logP(y j )
j=1

H(XY) = H(X / Y) – H (Y) = H(Y / X) + H(X).

(ii) if „x‟ and „Y‟ are independent events then P(xiyj) = P(xi).P(yj)
m n
Thus H(XY) =  P(x i ,y j )logP(x i y j )
i=1 j=1

m n
=    P(x i y j )log  P(x i )P(y j ) 
i=1 j=1
m n
=    P(x i y j ) logP(xi )+logP(y j )
i=1 j=1
m n m n
=  P(x i y j )logP(x i )   P(x i ,y j)logP(y j)
i=1 j=1 i=1 j=1

m n
=  P(x i )logP(x i )   P(y j )logP(y j )
i=1 j=1

H(X, Y) = H(X) + H(Y)

ENTROPY IN CONTINUOUS CASE

Let us consider the continuous signals which are band limited to „fm‟ Hz. Then the signal can
1
be uniquely specified by samples, taken at every second. The information content of
2f m
these samples can be determined, so that the entropy of the continuous signal can be
calculated.
If each sample value may have an infinite number of values, each values having a probability
is zero, thus

H =  P(i) logP(i) bits/symbol
i=-

Will give the entropy of the given sample as infinite.

i.e., the signal with continuous density function has an infinite entropy associated with it.
Now, since the channel capacity is determined by the source entropy it follows an infinite
channel capacity is required to transmit the signal, this is the conditon for perfect and most
efficient transmission. But in practice an infinite channel capacity is impossible, thus the
signal will suffer from error.

The entropy of sample point is H(x) =   p (x) log p (x) dx
-

Where p (x) = probability density function associated with signal x(t).

1
The samples occurs at every second then
2f

H (x) = 2f  p (x) log p (x) dx
-

note that H (x) is not an absolute measure of information, but only relative subject to a
coordinate system which can be changed.
If the continuous signal x(t) is limited to a finite range of values then H(x) is maximum, when
p(x) is uniformly distributed over the range
1 1
i.e., if p(x) = a for  x
2α 2α
α/2

then H(x) =   α log α dx =  logα

-α/2

compare this with discrete system when the entropy is maximum when the probability of all
symbols were equal.
SOURCE CODING THEOREM
A conversion of the discrete message sequence into a sequence of binary symbols is known
as source coding. The main problem of coding, technique is the development of an efficient
source encoder. An objective of source coding is to minimize the average bit rate required for
representation of the source by reducing the redundancy of the source. The basic requirement
of the source encoder is
i) The code words produced by the source encoder should be in binary form.
ii) The source code is uniquely decodable so that the original source sequence binary
sequences.
Let us consider a discrete memory less source encoder as shown below. The source generates
a output of x1, x2, ….xn is converted by the source encoder in to binary values of „0‟s and
„1‟s.
Code length and code efficiency
Let „X‟ be a discrete memory source with finite entropy H(X) and an alphabet X = {x 1 x2 x3
….xm} with corresponding probability of occurance are P(x1), P(x2)….P(xm).
The binary code word assigned to symbol xi by the encoder have length ni measured in bits.
The length of a code word is the number of binary digits in the code word. The average code
word length „L‟ per source symbol is given by
m
L =  P(x )n
i=1
i i

Where L = average number of bits / symbol used in the source coding process and code
efficiency is defined as
L min
η=
L
Where Lmin is the minimum possible value of „L‟ when η approaches unity, then the code is
said to be effiecient.
The code reduncy is defined as γ = 1 – η
The source coding theorem states that for a source „x‟ with entropy H(x), the average code
length „L‟ per symbol is bounded as
L  H(X)
H(x)
If L = H(x) then code efficiency η =
L
Example for Source Encoding Theorem
Case (a)
Let us consider a discrete binary source has two outputs „P‟ and „Q‟ with the probability of
0.6 and 0.4 respectively. The source output is connected to a binary channel, it has the
transmission rate of 2 symbols per second. Assume source rate = 2.5 symbols / second.
for total probability p =1, the channel capacity is 1 bit/symbols and the information rate is 2
bit / second.
The source output H(x) = – p(x1) log2p(x1) – p(x2) log2p(x2)
H(x) = –0.6 log20.6 – 0.4 log20.4 = 0.442 +0.528 = 0.970 bits / symbols.
log10 x
We know log2x = = 3.3219 log10x
log10 2
The source information rate = rH(x) = 2.5(0.970)
= 3.3976 bits / second.
In this case the source information rate is greater than channel capacity or given source rate
thus the transmission is not possible.
Case (b)
It the source the probability p1 = 0.2; p2 = 0.8 and source information rate = 2.5 symbols /
second.
H(x) = – 0.2 log2 0.2 – 0.8 log2 0.8 = 0.464 + 0.257 = 0.721.
The source information rate = rH(x) = 2.5(0.721) = 1.8025.
In this case the source information rate is less than the channel capacity rate so the
transmission is possible.
MUTUAL INFORMATION AND CHANNEL CAPACITY
Mutual information of channel is the average information raised by the receiver when the
state of transmission is known. The mutual information can be represented by
I(X, Y) = H(X) – H(X/Y)
Where H(x) = entropy of the source
H(X/Y) = Entropy of the source when the state of receiver is
known.
m  m n 
i.e., I(X, Y) =  p(x i )logp(x i )    p(x i y j) log p(x i/y j) 
i=1  i=1 j=1 
n
we know P(xi) =  p(x y )
j=1
i j

m n  m n 
then I(X, Y) =  P(x i y j )log p (x i / y j )    P(x iy j)log p (x i) 
 i=1 j=1   i=1 j=1 
m n
 p(x i /y j ) 
I(X, Y) =  p(x y )log i j 2  
i=1 j=1  p(x i ) 
Similarly I(Y, X) = H(Y) – H(Y/X)

m n p(y j /x i )
I(Y, X) =  p(x y )log
i=1 j=1
i j 2
p(y j )

CHANNEL CAPACITY OF GAUSSIAN CHANNEL

If the signal is band limited, and the samples are taken at „2ω‟ per second.
then the rate of information transmission is

R(x) = 2ωH(x) = 2ωlog 2πeσ2

ωlog ( 2πeσ 2 ) = ωlog (2πeσ2 )

If p(x) is a band limited gaussian noise with an average noise power „N‟
Then
R(x) = ω log (2πeN) [similarly σ2 = N]
If a continuous source transmitting information over a noisy channel, then the received signal
is transmitted signal „x‟ and a noise „n‟, Thus the joint entropy of the source is given by

R(x n) = R (x) + R (n/x)

Assume „x‟ and „n‟ are independent, thus R (x n) = R (x) + R (n)
The output signal is the signal and noise, thus

or H(x, y) = H(x, n) or H(x, y) = H(x1, n2)

or H(y) + H(x/n) = H(x) + H(n)

or R(y) + R(x/y) = R(x) + R(n)

The information rate of receiver is

R = R(x) – R(x/y) = R(y) – R(n)

Thus the channel capacity is
C = maximum (R) bits/second
= maximum (R(y) – R(n)) bits/second.

For maximizing „R‟, R(Y) should be maximized the average power of received signal is
signal + noise i.e., „S + N‟, the R(y) is maximum, if y(t) is also a gaussian random process
because noise in gaussian.

Thus R(y) = log (2πe(S + n)) bits / second.

entropy of the noise is R(n) = ω log (2πeN) bits / second.
... The channel capacity is

C = maximum [R(y) - R(n)]

= ω log 2πe(S + N) - ω log (2πeN) 
S + N  S
C = ω log   = ω log 1+ 
 N   N
The above equation is known as shanmon‟s Hartley theorem. The theorem states that, the
channel capacity for white band limited gaussian channel is

 S
C = ω log 1+  bits
 N
Where ω = channel bandwidth we know the power spectual density of noise is
N = ηω

 S 
... C = ω log 1+  bits/second.
 ηω 
The trade-off between S/N and Bandwidth
We know the channel capacity interms of bandwidth is as follows

 S 
C = ω log 1+ 
 ηω 
S
For noiseless channel   , then the channel capacity is infinite. But, practically the
N
channel capacity does not become infinite, if the bandwidth approaches infinity, because the
increase in bandwidth. Therefore the channel capacity approaches an upper limit with
increasing bandwidth, it can be proved as follows

 S  S 
C = ω log 1+  = ω log 1+ 
 N  ηω 

ηω/S
S ηω  S  S  S 
= . log 1+  = log 1+ 
η S  ηω  η  ηω 

1
S
= log(1+x) x
η

1
S
We know Lt
x 0
(1+x) x = e thus x =
ηω
When ω   , then x  0 hence, the above equation becomes,

S ηω/δ
Lt (1+ ηω )
x 0
= e

S S
C =
log 2 e = 1.44 = R max
η η
This equation represents an upper limit on the channel capacity with increasing bandwidth,
where the channel capacity is finite. Hence, the channel capacity of a channel of infinite
bandwidth with gaussian moise is finite.

DISCRETE MEMORY LESS CHANNEL

A channel is a transmission medium of communication system. The channel may be

(i) Continuous channel or (ii) Discrete channels.
If a source of information generation is continuous then the continuous channel is used for
information, transmission, transformation otherwise discrete channel is used.
A discrete memory less channel is a statistical model with a input „X‟ and output „Y‟ ie, noisy
version of „X‟ and „X‟ and „Y‟ are random variable. The channel accepts input signal „X‟
selected from an alphabet „A‟ and in response it delivers an output symbol r from an apbhabet
„B‟. The channel is said to be discrete when both of the apbhabets „A‟ and „B‟ have finite
sizes. It is said to be memory less if output depends on the current input symbol only.

Figure 1: Communication channel representation

The channel is described by an input alphabet

i.e., A = x 0 x1........xi-1
the corresponding output alphabet B =  y0 y1........yi-1 the transitions probabilty is
 y 
p  j  =p(Y=y j \X=x i ) for all „i‟ and „j‟. In general we know
 xi 

0  p (y j /x i )  1 for all „i‟ and „j‟.

The size of input symmbol and output symbol need not be the same size. Due to the coding
process the input symbol may be greater than input symbol i.e., j  1 of output symbol be
less than the input symbol i.e., i  1.
A discrete memory less channel may be less arrange in a matrix form as follows

P(y0 /x 0 ) P(y1/x1 ) ...... P(yn-1/x 0 ) 

P(y /x ) P(y /x ) ...... P(y /x ) 
 0 1 1 1 n-1 1 
P = .  = Channel transition matrix
. 
 
P(y0 /x m-1 ) P(y1/x i-1 ) ...... P(y n-1/x m-1 ) 
The fundamental property of the channel matrix „P‟defined that the rising the elements along
any row of the matrix is always equal to one

n-1
i.e.,  p(y /x )=1 for all „i'
j=0
j i

The joint probability distribution of the random variable „X‟ and „Y‟ is given by

P(xi, yj) = p(xi, yj) = p(yj / xi)p(xi)

The marginal probability distribution of the output random variable „Y‟ is obtained by
averaging p(xi, yj) i.e.,
n-1
p(yj) =  p(y /x )p(x ) for j = 0, 1…..n -1
j=0
j i i

When p(xi) = prior probability

p(yj/xi) = Channel matrix or matix of transition probability

p(yk) = probability of output symbol

i) Noise free channel

The noise free channel means there is one to one

correspondence between input and output. i.e.,
each input symbol is received as one and only
one output symbol as shown in figure 2.
If the joint probability matrix p(X, Y) a
diagonal matrix such as

Figure 2: Noise free channel

  p(x1 y1 ) 0 0 ........ 0 
 
 0 p(x 2 y2 ) 0 ........ 0 
 
P(X Y) =  ....................  (1)
 
 .................... 
 0 ................ p(x y ) 
 n n 
then the channel probability matrix p(X Y) = p(y/x) = 1 then the matrix will be

1 0 0 ...... 0 
0 1 0 ...... 0 
p(X Y) = 
0 0 1 ...... 0 
 
0 0 0 ...... 1
(2)

we know
m n n
H(X Y) =  p(x i y j ) log p(x i y j )=   p(x iy j) log p(x iy j)
i=1 j=1 i=1

because p(xiyj) = 0 for i  k as shown in equation (1)

From equation (1) we get p(xi yj) = p(xi) = p(yj)

thus H(XY) = H(x) = H(y) and

H(Y/X) = H(X/Y) = -m(1log1) = 0

Thus for noise free channel

I (X Y) = H(X) – H(X/Y) = H(X) = H(Y) = H(XY)

The channel capacity c= maximum I(X, Y) = max H(X) = logM bits/message

ii) Noisy Channel

When the channel has noise it becomes difficult to reconstruct the transmitted signal
faithfully. Assume the channel noise is a stationary process, in the sense that sensitive
symbols are perturbed independently.

In otherwords it can be defined as the channel there is no correlation between the input and
output symbol. Thus the joint probability matrix is provides corretion input and output.

 p1 p1 ........ p1 
P(x y) =  p 2 p 2 ........ p 2 
 p n p n ........ p n 

n
1
Let we know p
i=1
i =
m

P(xi) = mpi for i = to m.

1
P(yj) = for j = 1 to n.
m
If p(xi yj) = pi then

p(xi yj) = p(xi) p(yj) (1)

The equation (1) shows that xi and yj are independent for all „i' and „j‟ i.e., the input and
output are independent for the channel.
From equation (1)
p(x i y j )
= p(x i ) or p(x i / y j ) = p(x i )
p(y j )

p(x i y j )
and = p(y j ) or p(y j / x i ) = p(y j )
p(y j )

m n
thus H(Y / X) =  p(x i y j ) log p(y j / x i )
i=1 j=1

Figure 3: Noisy channel

m n
=  p(x i ) p(y j ) log p(y j )
i=1 j=1

m n
=  p(x i ) p(y j ) log p(y j )
i=1 j=1
n
m 
= -  p(x i )  p(y j ) log p(y j )
j=1  i=1 

n m
= - p(y j ) log p(y j ) since  p(x i ) = 1
j=1 i=1

= H(Y).
Similarly H(X/Y) = H(X)

Thus for the channel shown in figure 3.

I(X, Y) = H(X) – H(X/Y) = H(X) – H(X) = 0

= H(Y) – H (Y/X) = H(Y) – H(Y) = 0

The equation (2) states that no information is transmitted through the channel.

If the channel is represented as shown in figure 4 then the joint probability matrix is

 p1 p 2 ........ p n 
P [X, Y] =  p1 p 2 ........ p n 
 p1 p 2 ........ p n 
n
1
Then p
j=1
j =
n

1
and p (x i ) = for i=1, 2, ……..m.
n
p (y j ) = mp j for j=1, 2, ……n. Figure 4: Noisy channel

p (xi y j ) = p (xi ) p (y j ) = p j (3)

The equation (3) shows that xi and yj are independent for all „i' and „j‟, i.e., no information is
tranmitted through it. i.e., I (X Y) = 0.
From the above theory it is concluded that, the joint probability matrix if each row consists
the same element or each column consists the same element then the channel is said to be
noisy channel.

Binary symmetric channel

A symmetric channel means

i) H(Y / xi) is independent of „i' i.e., entropy corresponding to each row of p(Y / X) is
the same.
n
ii)  p(y
i=1
j / x i ) is independent of „j‟ i.e., the giving all the coloums of p(Y/X) is the

same.
In other words, the symmetric channel can be defined as, the channel is symmetirc if the rows
coloum‟s of the channel metrix p(Y/x) are indentical except permutation shown as below.

 0.3 0.3 0.4 

p(Y / X) = 0.4 0.3 0.3 =
 0.3 0.4 0.3

 0.5 0.5
p(Y / X) = 0.4 0.6  = not a rows and coloums are identical except for
 
0.3 0.7  permutation (i.e., each contain two
symmetrical channel. 0.3 and 0.4)

The symmetric channel with two inputs

(0, 1) and two output (0, 1) are known as Binary symmetric channel. In this case m = n = 2
then the channel matrix is given by

 p 1-p  p q 
P(Y / X) =   =   where q = 1 – p
1-p p  q p 

For symmetric channel

n
I(X, Y) = H(Y) – H(Y / X) = H(Y) -  p(Y / x i ) p(x i )
i=1

n
= H(Y) – A p(x i ) = H(Y) – A
i=1

The channel capacity is C = maximum I (X; Y)

C = maximum [H(Y) – A] = maximum [H(Y)] – A = log n – A

Where maximum H(Y) = log n

Cascaded Channel
If the channel are cascaded as shown in figure 5. The message from X0 to Z0 reached in two
ways is.

X0  Y0  Z0 and X0  Y1  Z0 the corresponding probability are p2 and q2 where q = (1-

p).

P1 = p2 + q2 = (p + q)2 – 2pq = 1 – 2pq

Figure 5: Cascaded channel

Similarly the message form x1 to Z1 reacher in two ways.

X1  Y1  Z1 or X1  Y0  Z1 , the corresponding probabilities are p(1 – p) and (1 – p)

p or p q and qp.

Hence q1 = pq + qp = 2pq.

The channel matrix is given by

1 - 2pq 2pq   p1 q1 
P(Z/X) =   =  1 1
 2pq 1 - 2pq  q p 

Channel capacity C = 1 – H (q1) = 1– H (2pq).

The above equation represents the channel capacity of two cascaded BSC is less than single
BSC.

CODING

Coding is a procedure for mapping a given set of message (m1, m2, ….mn) into a new set of
encoded messages (C1, C2,…Cn) such a way that the transformation is one to one for each
message. This is known as source coding. It is possible to device codes for special purposes
such as secrecy or minimum probability of error without relevance to the efficiency of
transmission. This is known as channel coding.

Advantages of coding

i. Improve transmission efficiency

ii. Reduce the probability of error and correcting errors.

The messages are first encoded by the encoder then transmitted via channel. At the recceiving
end the received messages are first decoded in the decoder and then original messages are
recovered.

The following terminology is associated with coding.

a) Letter, symbol or character : Any individual number of the alphabet set.

b) Message or word : a finite sequence of letters or an alphabet.

c) Length of the word : the number off letters in a message.

d) Coding – encoding or enciphering : A procedure for associating words

constructed from a finite alphabet of a language with a given words of another
messages in a one to one manner.

e) Decoding or deciphering : the inverse operation of assigning words of the

second language corresponding to the given words of first language.

f) Uniquely decipherable or separable encoding and decoding : In this operation,

the correspondence of all possible sequences o words between the two
languages in one to one when there is no space between the words.

g) Irreducible : When no encoded words can be obtained from each other by the
addition of more letters, the code is said to be irrsducibel when a code is
irreducible it is also uniquely decipherable, but the reverse is not true.

Example : Let C1 = 0; C2 = 10 and C3 = 110. This code is irreducible as an addition of „0‟ or a

„1‟ to any of the code words does not produce other codewords. The same code is uniquely
decipherable as any received message string can be uniquely decoded.

For example : We receive 0 110 10 10 0 0 10 110 110 110…

C1 C3 C2 C2 C1 C1 C2 C3 C3 C2

It is uniquely decoded. Thus it can be said that when a code is irreducible, it is also uniquely
decipherable.

Coding Efficiency

Let „M‟ be thee number of symbols in an encoding alphabet. Let there be „N‟ messages {x 1,
x2, . . .xn) with the probabilities (P1, P2, …..Pn). Let ni be number of symbols in the ith
message. The average length of the message per code word is given by
N
L =  n i p(n i ) letters / message.
i=1

i should be minimum to have an efficient transmission coding efficiency.

L min
η=
L

Let H(x) = entropy of the source in bits / message.

Log M = Maximum average information associated with each letter in bits / letter.

H (x) L min H (x)

Lmin = thus η = =
log m L L log m
Redundancy = 1 – η = Repetition of occrrence of message.

SHANNON FANO-CODING

This method of coding is directed towards constructing reasonably efficient separable binary
codes.

let x = ensemble of the messages to be transmitted.

p = Corresponding Probability of [x].

The sequence Ck of binary numbers of length nk associated to each message xk should the
following conditions.

i) No sequence of employed binary numbers Ck can be obtained from each other by adding
more binary digits to the shorter sequence.

ii) The transmission of an encoded message is reasonably efficient.

Procedures for Shannon – fano coding

 The message is first written in the order of non-increasing probability.

 The message set, then is partitioned or divided into two most equiprobable susets (X1)
and (X2).

 Assign „O‟ to each message contained in X1 and assign „1‟to each message contained in
X2 and vice-versa.

 The same procedure is repeated for subsets of (X1) and (X2) i.e., X1 is partitioned into X11
and X12 is partitioned into X21 and X22. The codewords for

X11 = 00 Where MSB depends on the initial

X12 = 01 value assigned for X1 i.e., „0‟.

X21 = 10

X22 = 11

This procedure is continued until each subset contains only one message note that each digit
„0‟ or „1‟ in each partitioning of the probability space appears with a more or less equal
probability and is independent of the previous or subsequent partitioning. Hence p(0) and
p(1) are also equiprobable.

HUFFMAN CODING

Huffman coding method leads to the lowest possible value of I for a given m, resulting in a
maximum η. Hence it is also known as the minimum redundancy code or optimum code. The
procedure for obtaining Huffman coding is as follows:

 Arrange the message in order of descending probability.

 Combine the last two messages (for binary coding) into one message (last „µ‟ message
into one message for µ‟ ary coding) by adding their probabilities.

 Rearrange the message in descending order of probability.

 Combine the last two messages into one message.

 Repeat the procedure until the number of message is reduced to two (for binary coding)
µ‟ value for µ‟ ary coding.

 Assign „0‟ and „1‟ to these last two messages as their first digits in the code sequence.

 Go back and assign the numbers „0‟ and „1‟ to the second digit for the two messages that
were combined in the previous step.

 Kept regressing this way until the first column is reached.

 The first column coding gives the final optimum code.

N
The average length of optimum code is given by L = PL
i=1
i i

Where Pi = Probability of the ith message.

Li = Length of the code for the ith message.

It is also noted the Huffman code is uniquely decodable with the result that a sequence of
coded messages can be decoded without any ambiguity.
 KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY
DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING
EC 8501 – DIGITAL COMMUNICATION

UNIT – II Waveform Coding & Representation

Design a modulation system that transmit one bit per sample which use the uniform
step size for quantizer. .
DM Transmitter block diagram

Block Diagram explanation

The summer in the accumulator adds quantizer output±δ with previous
 sample approximation.
 Present sample u(kTs)=u(k-1)Ts±δ.
 The previous sample approximation u(k-1)Ts is restored by delaying one
 sample period Ts.

DM Receiver block diagram

Block Diagram explanation

Accumulator generates the staircase approximated signal output and is delayed by one
sampling period Ts.
 It is then added to input signal.
  If i/p is 1adds + δ step to the previous o/p.
 If i/p is 0adds - δ step to the previous o/p
1)Slope overload Distortion
(2+2+1)
 Large error between the staircase approximated signal and the original signal x(t).
 This is reduced by having Large step size .
2) Granular Noise
 It is the Error between the input and approximated signal.
 This is reduced by having Small step size .
 Bit rate=v fs=1xfs=fs

Describe the linear predictive coding technique in speech encoding.

Linear Prediction Filter


 LPC Analyzeranalyzes the speech signal by estimating the formants, removing
their effects from the speech signal, and estimating the intensity and frequency of
the remaining buzz.
 The analyzer determines LP coefficients for the synthesis filter.
 Voiced-un voiced decision used to analyze the given speech segment is voiced
or unvoiced..
 Pitch analysis Estimate the fundamental frequency of 40 Hz, at least 50ms of
the speech signal must be analyzed.. Autocorrelation methods need at least two
pitch periods to detect pitch.
 Speech Synthesis using LPC

 The received LPC signal is applied to the decoder or demux.

 It separates the filter parameters and error signal.
 The LP coefficients are given to the Synthesis filter.
 The o/p of synthesis filter is added to the error signal which gives speech signal.

Explain the principle of DPCM system along with the transmitter and receiver
DPCMTransmitter

Block Diagram explanation

•The sampled signal is denoted by x(nTs).
•Predicted signal is denoted by x (nTs).
•Value of present sample is predicted from the past samples.
•Comparator finds out the difference between the actual sample x(nTs) and
• predicted sample value x (nTs).
•This is called error & denoted by e(nTs)
•The predicted value is produced by using a prediction filter.
•The quantized output signal eq(nTs) and previous prediction is added and
• given as input to prediction filter.This signal is called xq(nTs).
•This makes the prediction more and more close to the actual sampled signal.
 •If the quantized output signal eq(nTs) is very small and can be encoded by using small no
of bits.

DPCM Decoder Block Diagram

Block Diagram explanation

Decoder reconstruct the quantized error signal from incoming binary signal.
• The prediction filter output and quantized error signals are summed up to give the
quantized version of original signal.
• Original signal=prediction filter output+Q-error signal
Calculation of SNR

Identify the type of modulation scheme which use the principle for continuously
variable slope technique
ADM Transmitter block diagram

Block Diagram explanation

• x(nTs)sampled signal of x(t)

• x (nTs)last sample approximation.
• Error e(nTs)=x(nTs)- x (nTs)
• Step size increased or decreased according to one bit quantizer output.
• If q-output is 1 δ doubled for next sample.
• If q-output is 0 δ reduced by 1 step.
• ADM modulators can take continuos changes in step size or discrete changes in step
size.
 DM Receiver block diagram

Block Diagram explanation

• Logic Step size control produces step size for each incoming bit.
• Accumulator build up the staircase waveform
• LPF used to smoothens out the staircase waveform and reconstruct the original signal.
Step Wave forms for ADM

5) Summarize the features of DPCM system along with the encoder and decoder
diagrams.
ADPCM Transmitter block diagram
Block Diagram explanation
 An 8-bit PCM value is input and converted to a 14-bit linear format.
 The predicted value is subtracted from this linear value to generate a difference signal.
 Adaptive quantization is performed on this difference, producing the 4-bit ADPCM
value to be transmitted.
 The adaptive predictor computes a weighted average of the last six
dequantized difference values and the last two predicted values.
 The coefficients of the filter are updated based on their previous values, the current
difference value, and other derived values.

DM Receiver block diagram

Block Diagram explanation

 In the receiving decoder  transmitted ADPCM value is used to update the inverse
adaptive quantizer, which produces a dequantized version of the difference signal.
 This dequantized value is added to the value generated by the adaptive predictor to
produce the reconstructed speech sample.
 This signal is converted in to PCM signal at the data rate of 64kbps..
 KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY
DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING
EC 8501 – DIGITAL COMMUNICATION
UNIT – III - Baseband Transmission & Reception

1. Obtain an expression for Nyquist criterion for distortion less baseband

transmission for zero ISI.
a)Time domain criterion

From the above eqn,the second term will be zero if ISI is zero,this is possible if the
received pulse p(t) is controlled such that
If p(t) satisfies the above condition,then the received signal is free from ISI.

The above condition in time domain for perfect reception in absence of noise

b)Frequency domain criterion

 Let p(nTb) represents the impulses at which p(t) is sampled for decision.

 Fourier spectrum of theses impulses is given by

 fbsampling , Pδ(f)spectrum of p(nTb),P(f)spetrum of p(t)

 p(nTb) as Infinite length of impulses are weighted with amplitudes of p(t)
The above frequency domain condition for zero ISI.The above equation is known
Nyquist pulse shaping criterion for baseband transmission
Find PSD for unipolar signaling NRZ scheme. Plot the spectrum
Binary sequence has equiprobable occurrence of 0 and 1.hence the probabilities of
occurrence are ½ each.

for n=0has two possible values 0 and 1E[ak ak-n] will be given as
For n=1 has four possible values 0 (a2k ),0 (a2k ),0 (a2k ), and 1 (a2k ),hence

Hence psd is given by

 Here k=0 and sinc function has value at 0,hence

Write short notes on how eye pattern is used to study the performance of a digital
transmission system

 Used to study the ISI effect in base-band digital tx.

 The name „eye‟ is given because it looks like an eye.
 Eye pattern is obtained by CRO.
 When the binary sequence is transmitted over a baseband transmission system, due to
nature of the transmission channel ,the signal becomes continuously increasing and
decreasing amplitudes at the receiver.
 For small number of patterns (ISI),small number of input signals.
 For large number of patterns (ISI),large number of input signals.

2. Construct a Duo-binary signaling scheme with and without precoder for

controlled ISI
1) Duo-binary Encoder
 Duo binary encoding reduces the max frequency of the base band signal.
 Duo means double the tx capacity of binary system.
• Consider the input sequence {bk} which contains binary symbols 1 and 0.
• By using level shifter this sequence is converted in to bipolar NRZ sequence {ak}
Input sequence if ak = 1 if bk = 1
if ak =-1 if bk = 0
 From the block diagram,the encoder accepts the sequence {ak} and converts in to
three level signal (-2,0,+2).
 The output of duo-binary encoder is given by ck = ak +ak-1
 Because the type of encoder the uncorrelated sequence {ak} of two level pulses is
converted to correlated sequence of three level pulses.
 This introduces Intersymbol Interference(ISI) in the signal to reduces the bandwidth.
Reconstruction
 Let a^k represents the estimate of ak. âk = ck – âk -1
 From the above resultsif ck is received,then âk will have error

Frequency response
 Tb is a delay element.
 e-j2πftb  frequency response of delay element
 1+e-j2πftb  frequency response of delay line filter
 Delay line filter is connected with the ideal Nyquist channel .hence overall frequency
response is given by

2)Differential Encoder
• Precoder is used in a duo binary encoder to avoid error propagation
  output of encoder dk = bk + dk-1 dk =1 if bk = 1
If dk = 0 if bk = 0

The sequence {dk} is applied to level shifter.All the output of level shifter sequence
{ak} is bipolar
output of level shifter
 dk=1 then ak= 1
 dk = 0 then ak = -1
 The sequence {ak} is then applied to duobinary encoder.then the output summer ck =
ak +ak-1
 ck =0 if bk = 1
 if ck =± 2 if bk = 0
 Decoder

• The sequence ck will be uncorrelated with three levels ±2 and 0.

• The Full wave rectifier allows a input signal in a both amplitude.
• The magnitude of ck is taken from the received sequence, and is compared
with 1 by the decision device
• Decision device produce the output 1 if |ck| < 1
Decision device produce the output 0 if |ck| > 1
Necessity of equalization in digital transmission? Explain how it is carried out by
using adaptive filters along with LMS algorithm
Adaptive equalizationAdapt itself to the effecy of channel.
 Weight valueIt is varried according to the nature of signal received.(LMS algorithm
used for weight updatation)
 Error ValueIt is calculated from the output and desired output.
 Feedback loopTo know the amount of distortion in received sequence.
 Operating modes1)Training mode 2)Decision directed mode
 Applications
 Telephone channel-Noise and Echo removing
 The sequence x(nT) is applied to the input of
the adaptive filter.
 The output y(nT)of filter is given as

 It is one of the algorithms to change the tap weights of the adaptive filter recursively.
 The tap weights are adapted by this algorithm as follows.
 KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY
DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING
EC 8501 – DIGITAL COMMUNICATION
UNIT – IV - Digital modulation schemes

Draw the block diagram of QPSK transmitter and receiver with signal space diagram,
bandwidth and also calculate bit error probability.

QPSK transmitter

 The input binary sequence is first converted

to a bipolar signal b(t).It represents binary 1
by +1V and 0 by -1V.
 The demultiplexer divides b(t) in to odd

sequence bo(t) and even sequence be(t)

 Balance modulator produces modulated
output by multiplying bit sequence with
carrier.
 The output of upper modulator se(t)= be(t)
√Ps sin(2πfot)
 The output of Lower modulator sot)= bo(t) √Ps cos(2πfot)
 Adder which adds these signals(se(t) and so(t))
 Adder output is given by s(t)= be(t) √Ps sin(2πfot) + bo(t) √Pscos(2πfot)

QPSK Receiver

The received signal is raised s(t) is raised its power

4th power.
Coherent carriers are applied to two synchronous detector.
Integrator integrates the product signal over two bit interval Ts=2Tb
the output of two integrators is sampled at one bit period.
Hence the mux output is given by
 b(t) = be(t) √Ps Tb + bo(t) √Ps Tb
Distance between two signal points given by d=2√Ps Tb
Bandwidth is given by BW= fb

Explain BPSK expressions, signal space, spectral characteristics, wave form generation
and reception
BPSKTransmitter
 The baseband signal b(t) is applied as a
modulating signal to Balance Modulator
 NRZ level encoder converts the binary sequence
into bipolar NRZ signal.
 Output of BPSK s(t)= b (t) √Pscos(2πfot)

BPSK Receiver

 Phase shiftbased on input, signal goes phase shift s(t)= b (t) √Pscos(2πfot+θ)
 Square law deviceused for carrier separation.1/2 +1/2 cos2(2πfot+θ)
 BPFallow signal frequency is centered at 2f0.the output of filter is given by
cos2(2πfot+θ)
 Freq-dividerfiltered signal is divided by factor 2cos(2πfot+θ)
 Syn-demodulatormultiply input signal and recovered carrier. b (t)
√P/2[1+cos2(2πfot+θ)]
 Integratorintegrates signal over 1bit period.The output of integrator is given by
 so(kTb)= b (kTb)

Signal Space Representation

Distance between two signal points given by d=2√Eb

With the help of block diagram explain BFSK transmitter and receiver
BFSK transmitter

 Input sequence b(t) is same as PH(t).

 An inverter is added after b(t) to get PL(t).
 PH(t)and PLt) are unipolar signals.
 The level shifter converts +1 level to √PsTb and 0 for
 Two carrier signals are orthogonal to each other.ɸ1(t) and ɸ2(t)

BFSK receiver

 Correlators are supplied with locally

generated carriers.ɸ1(t) and ɸ2(t).
 If the tx frequency is fH,then the output
s1(t) will be higher than s2(t),
 Hence y(t)>0.
Describe the importance of Costas loop for carrier synchronization with the block
diagram

 It is used in BPSK here two types of PLL used.

 Both are used common vco and separate phase comparator.
 VCO operates at the carrier frequency of fc and phase angle θ.
 Output of multiplier is given by

Discuss the 16-QAM scheme with block schematics to generate and receive and relevant
waveforms at various stages.
 In this method amplitude of the carrier signal is varied hence it indirectly change the
phase of the carrier
 For 4-bit symbol,then 16 possible symbols are generated at a distance of d=2a.
 The distance of 16-QAM is greater than 16-PSK and lesser than QPSK.
 QAM Transmitter
 The input bit stream is applied to a
serial to parallel converter.
 Bits bk, ,bk+1 are applied to upper
converter.Bits bk+2 , bk+3are applied
to upper converter are applied to
lower converter.
 Modulator modulates the carrier
based on msg bit.
 Adder combines odd and even
 sequence
 Adder output is given by s(t)= Ae(t) √Pscos(2πfot) + Ao(t) √Ps sin(2πfot)

QAM Receiver

 Carrier recovery circuit obtains quadrature carriers from received QASK signal.
 The integrators integrate the multiplied signals over one symbol period.
 The in-phase and quadrature carriers are multiplied with QASK signal.
 KONGUNADU COLLEGE OF ENGINEERING AND TECHNOLOGY
DEPARTMENT OF ELECTRONICS AND COMMUNIATION ENGINEERING
EC 8501 – DIGITAL COMMUNICATION

UNIT – V - Error control coding

Explain the viterbi algorithm to decode a convolution coded message with a suitable
example.
 Viterbi decoding:

Design the encoder for the (7,4) cyclic code generated by G(p)=P3+P+1 and verify its
operation for any message vector.
3. The parity check matrix of a particular (7,4) linear block code is given by
1110100
11 01010
1011001

(a) Construct generator matrix

(b) Construct code generated matrix
(c) What is the minimum distance between code vectors
(d) Determine error correcting capacity
(e) Draw the encoder diagram
4. A convolution code is described by g1={0 1 0}; g2={1 1 1}; g3={1 0 1}.Sketch the
encoder diagram.
(a) Draw the encoder tree,state diagram and trellis diagram
(b) If the input message sequence is 101010,determine the output sequence of the
encoder
(c) Using viterbi algorithm extract the input sequence for the received sequence
Viterbi decoding: