GATE EC Information Theory questions

Let the relevant bandwidth $(B)$ of a digital communication system be $1 ~\text{MHz}$ and $k T=-174 \mathrm{~dBm} / \mathrm{Hz}$, where $k$ is Boltzmann's constant and '$T$' is equivalent noise temperature of the receiver. The power $(S)$ of signal received through an additive Gaussian channel is $-80 ~\text{dBm}.$ Which of the following …

Consider an additive white Gaussian noise (AWGN) channel with bandwidth $W$ and noise power spectral density $\frac{N_{0}}{2}$. Let $P_{a v}$ denote the average transmit power constraint. Which one of the following plots illustrates the dependence of the channel capacity $C$ on the bandwidth $W$ (keeping $P_{a v}$ and $N_{0}$ fixed)?

$X$ and $Y$ are Bernoulli random variables taking values in $\{0,1\}$. The joint probability mass function of the random variables is given by: $$\begin{array}{l} P(X=0, Y=0)=0.06 \\ P(X=0, Y=1)=0.14 \\ P(X=1, Y=0)=0.24 \\ P(X=1, Y=1)=0.56 \end{array}$$ The mutual information $I(X ; Y)$ is $\_\_\_\_\_\_\_\_$ (rounded off to two decimal places).

A source transmits symbols from an alphabet of size $16$. The value of maximum achievable entropy (in bits) is $\_\_\_\_\_\_$.

The frequency of occurrence of $8$ symbols $(a-h)$ is shown in the table below. A symbol is chosen and it is determined by asking a series of $\text{"yes/no"}$ questions which are assumed to be truthfully answered. The average number of questions when asked in the most efficient sequence, to determine the chosen symbol, is _______________ (rounded off …

The transition diagram of a discrete memoryless channel with three input symbols and three output symbols is shown in the figure. The transition probabilities are as marked. The parameter $\alpha$ lies in the interval $[0.25, 1].$ The value of $\alpha$ for which the capacity of this channel is maximized, is ___________ (rounded off to two decimal …

A speech signal, band limited to $4\:\text{kHz}$, is sampled at $1.25$ times the Nyquist rate. The speech samples, assumed to be statistically independent and uniformly distributed in the range $-5\:V$ to $+5\:V$, are subsequently quantized in an $8$-bit uniform quantizer and then transmitted over a voice-grade $\text{AWGN}$ telephone channel. If …

In a high school having equal number of boy students and girl students, $75\%$ of the students study Science and the remaining $25\%$ students study Commerce. Commerce students are two times more likely to be a boy than are Science students. The amount of information gained in knowing that a randomly selected girl student studies Commerce (rounded off to …

A binary random variable $X$ takes the value $+2$ or $-2$. The probability $P(X=+2)=\alpha$. The value of $\alpha$ (rounded off to one decimal place), for which the entropy of $X$ is maximum, is __________.

Let $\left ( X_{1},X_{2} \right )$ be independent random variables. $\left ( X_{1} \right )$ has mean $0$ and variance $1$, while $\left ( X_{2} \right )$ has mean $1$ and variance $4$. The mutual information $I\left (X_{1},X_{2} \right )$ between $X1$ and $X2$ in bits is_______.

Which one of the following graphs shows the Shannon capacity (channel capacity) in bits of a memoryless binary symmetric channel with crossover probability $p$ ?

Consider a binary memoryless channel characterized by the transition probability diagram shown in the figure. The channel is

A binary communication system makes use of the symbols “zero” and “one”. There are channel errors. Consider the following events: …

The capacity of a Binary Symmetric Channel $\text{(BSC)}$ with cross-over probability $0.5$ is ________.

A fair coin is tossed repeatedly until a ‘Head’ appears for the first time. Let $L$ be the number of tosses to get this first ‘Head’. The entropy $H(L)$ in bits is _____.

Let $U$ and $V$ be two independent and identically distributed random variables such that $P(U=+1)=P(U=-1) = \dfrac{1}{2}.$ The entropy $H(U + V)$ in bits is

A source generates three symbols with probabilities $0.25,0.25,0.50$ at a rate of $3000$ symbols per second. Assuming independent generation of symbols, the most efficient source encoder would have average bit rate as

A video transmission system transmits $625$ picture frames per second. Each frame consists of a $400 \times 400$ pixel grid with $64$ intensity levels per pixel. The data rate of the system is

Source encoding in a data communication system is done is order to