Scribd Logo
Upload

Welcome to Scribd!

  • 54 views

    EE6340: Information Theory Problem Set 7: I I I I I I I I 1 2 N N N

    Uploaded by

    Supriya Diwan
    This document provides 4 problems related to information theory and channel capacity. Problem 1 asks to show that the capacity of a binary symmetric channel with dependent noise variables is greater than or equal to the capacity of the same channel with independent noise. Problem 2 asks to find the capacity of a time-varying discrete memoryless channel. Problem 3 asks to show that adding a row to a channel transition matrix does not decrease capacity. Problem 4 asks about a channel that provides two independent observations of the output and asks to relate its capacity to the capacity of a single-output channel.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    EE6340: Information Theory Problem Set 7: I I I I I I I I 1 2 N N N

    Uploaded by

    Supriya Diwan
    54 views1 page
    This document provides 4 problems related to information theory and channel capacity. Problem 1 asks to show that the capacity of a binary symmetric channel with dependent noise variables is greater than or equal to the capacity of the same channel with independent noise. Problem 2 asks to find the capacity of a time-varying discrete memoryless channel. Problem 3 asks to show that adding a row to a channel transition matrix does not decrease capacity. Problem 4 asks about a channel that provides two independent observations of the output and asks to relate its capacity to the capacity of a single-output channel.

    Original Title

    problem_set7

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides 4 problems related to information theory and channel capacity. Problem 1 asks to show that the capacity of a binary symmetric channel with dependent noise variables is greater than or equal to the capacity of the same channel with independent noise. Problem 2 asks to find the capacity of a time-varying discrete memoryless channel. Problem 3 asks to show that adding a row to a channel transition matrix does not decrease capacity. Problem 4 asks about a channel that provides two independent observations of the output and asks to relate its capacity to the capacity of a single-output channel.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    54 views1 page

    EE6340: Information Theory Problem Set 7: I I I I I I I I 1 2 N N N

    Uploaded by

    Supriya Diwan
    This document provides 4 problems related to information theory and channel capacity. Problem 1 asks to show that the capacity of a binary symmetric channel with dependent noise variables is greater than or equal to the capacity of the same channel with independent noise. Problem 2 asks to find the capacity of a time-varying discrete memoryless channel. Problem 3 asks to show that adding a row to a channel transition matrix does not decrease capacity. Problem 4 asks about a channel that provides two independent observations of the output and asks to relate its capacity to the capacity of a single-output channel.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 1

    EE6340: Information Theory

    Problem Set 7

    1. Channels with memory have higher capacity. Consider a binary symmetric channel with
    Yi = Xi ⊕ Zi , where ⊕ is mod 2 addition, and Xi , Yi ∈ {0, 1}.
    Suppose that {Zi } has constant marginal probabilities P r{Zi = 1} = p = 1 − P r{Zi = 0},
    but that Z1 , Z2 , .....Zn are not necessarily independent. Assume that Z n is independent
    of the input X n . Let C = 1 − H(p, 1 − p). Show that

    max I(X1 , X2 , ..., Xn ; Y1 , Y2 , ..., Yn ) ≥ nC.


    p(x1 ,x2 ,...xn )

    2. Time-varying channels. Consider a time-varying discrete memoryless channel.


    Let Y1 , Y2 , ...Yn be conditionally
    Qn independent given X1 , X2 , ...Xn , with conditional distri-
    bution given by p(y|x) = i=1 pi (yi |xi ). Let X = (X1 , X2 , ...Xn ), Y = (Y1 , Y2 , ..., Yn ).
    Find maxp(x) I(X; Y).

    3. Can signal alternatives lower capacity? Show that adding a row to a channel transition
    matrix does not decrease capacity.

    4. A channel with two independent looks at Y. Let Y1 and Y2 be conditionally independent


    and identically distributed given X.

    (a) Show that I(X; Y1 , Y2 ) = 2I(X; Y1 ) − I(Y1 ; Y2 ).


    (b) Conclude that the capacity of the channel (1) is less than twice the capacity of the
    channel (2).

    X (Y1 , Y2 )

    (1)

    X Y1

    (2)

    You might also like