BTECH (SEM VII) THEORY EXAMINATION 2023-24 INFORMATION THEORY & CODING TIME: 3 HRS M.MARKS: 100
Note: 1. Attempt all Sections. If require any missing data; then choose suitably. SECTION A
1. Attempt all questions in brief.
Q no. Question Marks CO a. What is Fano’s Inequality? 2 1 b. Express the relationship between entropy and mutual information using 2 1 both additive and multiplicative forms. c. Explain the role of entropy in measuring the compressibility of data. 2 2 d. State the Kraft inequality and explain its significance in code design. 2 2 e. Write the relation between Channel Capacity and Bandwidth. 2 3 f. Define a symmetric channel and explain its significance in capacity 2 3 calculations. g. Define Hamming codes with example. 2 4 29 h. 2 4
98 Explain Minimum distance of block codes with example. _0
i. Draw the state diagram for a given convolutional code, labeling states, 2 5
9. and transitions. P1
24 j. Show how generator matrices and generator polynomials represent a 2 5
1. 4D
convolutional code.
.2 SECTION B P2
25 2. Attempt any three of the following: Q
|1
Q no. Question Marks CO
15
a. Given two random variables X and Y, Define joint entropy (H(X,Y)) 10 1
and conditional entropy (H(X|Y)). How do they relate to individual 8:
entropies? :2
b. Describe Shannon– Fano Coding in detail with expressions of average 10 2
13
length L, H(X), and code efficiency η & Redundancy 𝛾.
4
c. Write different Properties of Channel Capacity. A telephone channel has 10 3
02
BW of 3 kHz and SNR of 1023 is connected to a teletype machine
having 32 different symbols. Determine the maximum symbol rate -2
possible for error free transmission.
01
d. Define block codes and explain their primary purpose in digital 10 4
3-
communication. |2
e. Describe the process of encoding data using a convolutional encoder, 10 5
emphasizing the sliding window and memory elements. SECTION C 3. Attempt any one part of the following: a. What is Conditional Entropy? Consider a discrete memory less source 10 1 with alphabet S = {s0, s1, s2, s3, s4 …..} and respective probabilities of occurrence P= , , , , … . . Calculate the entropy of the source (in bits).
BTECH (SEM VII) THEORY EXAMINATION 2023-24 INFORMATION THEORY & CODING TIME: 3 HRS M.MARKS: 100
b. Define entropy (H) of a discrete random variable X with probability 10 1
mass function (PMF) p(x). Explain its significance as a measure of uncertainty. Also Derive the chain rule for entropy (H(X1,X2,...,Xn)). 4. Attempt any one part of the following: a. Explain Huffman Codes. Five source messages are probable to appear 10 2 as p(m1) = 0.4, p(m2) = 0.15,p(m3) = 0.15,p(m4) = 0.15,p(m5) = 0.15. Calculate the efficiency of Huffman code in %. b. State the AEP theorem and explain its significance for information 10 2 theory. 5. Attempt any one part of the following: a. Derive the channel coding theorem for a discrete memoryless channel 10 3 using jointly typical sequences. A communication channel with additive white Gaussian noise, has a bandwidth of 4 kHz and SNR of 15. Calculate the channel capacity in kbps. 29
98 b. Define the BSC and derive its channel capacity (C) as a function of the 10 3 _0
crossover probability (p). Explain the intuitive meaning of C in this
9. context. P1
24 6. Attempt any one part of the following:
1. 4D
a. Explain the structure and operation of single-parity check codes, 10 4
.2 P2
providing an example. 25 b. Describe Generator matrices and Parity-check matrices with suitable 10 4 Q
|1
relations. The generator matrix for a (6, 3) block code is given below. Find all the code vectors of this code. 15
1 0 0 0 1 1 𝐺 8:
0 1 0 1 0 1 0 0 1 1 1 0 :2
7. Attempt any one part of the following:
13
a. Write short note on (i) BCH codes (ii) RS codes, (iii) Binary Cycle 10 5 4
Codes, 02
b. Describe the steps involved in the Viterbi algorithm for decoding 10 5
