1 5

COURSE CODE
20AIPC502

DEEP LEARNING – QUESTION BANK

UNIT 1 - 5
 UNIT 1 – INTRODUCTION TO DEEP LEARNING & NEURAL NETWORK

1. How Neural networks is efficient as compared to conventional programming models?

Because of their style of learning, artificial neural networks can, in essence, “program themselves.” While the
conventional computers must learn only by doing different sequences or steps in an algorithm, neural
networks are continuously adaptable by truly altering their own programming.
2. Enumerate the salient features of Neural Network

Characteristic Artificial Neural Network Biological(9Real) Neural

Network
Speed Faster in processing Slower in processing
information. Response time information. The response
is in nanoseconds. time is in milliseconds.
Processing Serial processing Massively parallel processing
Size & Complexity Less size and complexity Highly complex
Fault tolerance Fault intolerant. New information is added
Corrupt information cannot
retrieve in case of failure of by adjusting the inter
the system. connection strengths without
DEEP LEARNING
destroying old information.
Control Mechanism Control unit for controlling No specific control
computing activities mechanism
3. Paraphrase the applications of neural network
⮚ Facial recognition
⮚ Stock market prediction
⮚ Social media
⮚ Healthcare
⮚ Weather forecasting
4. What do you mean by pattern recognition?
Pattern recognition is the process of recognizing patterns by using a machine learning algorithm. Pattern
recognition can be defined as the classification of data based on knowledge already gained or on statistical
information extracted from patterns and/or their representation.
Examples
⮚ Speech recognition
⮚ Speaker identification
⮚ Automatic medical diagnosis

5. How Neural Network is useful for Pattern recognition?

Similar to the way that human beings learn from mistakes, neural networks also could learn from their
mistakes by giving feedback to the input patterns. This kind of feedback would be used to reconstruct the
input patterns and make them free from error, thus increasing the performance of the neural networks.

6. What do you understand by Perceptron? Also, explain its type.

A perceptron is a neural network unit (an artificial neuron) that does certain computations
to detect features. It is an algorithm for supervised learning of binary classifiers. This
algorithm is used to enable neurons to learn and processes elements in the training set one
at a time.
 There are two types of perceptrons:

Single-Layer Perceptron
Single layer perceptrons can learn only linearly separable patterns.

Multilayer Perceptrons DEEP LEARNING

Multilayer perceptrons or feedforward neural networks with two or more layers has ve the
higher processing power.

7. What is the role of weights and bias?

For a perceptron, there can be one more input called bias. While the weights determine the
slope of the classifier line, bias allows us to shift the line towards left or right. Normally bias
is treated as another weighted input with the input value x0.

8. What is a Neural Network?

Neural Networks replicate the way humans learn, inspired by how the neurons in our brains
fire, only much simpler.
The most common Neural Networks consist of three network layers:
1. An input layer
2. A hidden layer (this is the most important layer where feature extraction
takes place, and adjustments are made to train faster and function better)
3. An output layer
Each sheet contains neurons called “nodes,” performing various operations. Neural
Networks are used in deep learning algorithms like CNN, RNN, GAN, etc.
9. What are the main differences between AI, Machine Learning, and Deep Learning?
⮚ AI stands for Artificial Intelligence. It is a technique which enables machines to
mimic human behavior. DEEP LEARNING
⮚ Machine Learning is a subset of AI which uses statistical methods to enable
machines to improve with experiences.
⮚ Deep learning is a part of Machine learning, which makes the computation of multi-
layer neural networks feasible. It takes advantage of neural networks to simulate
human-like decision making.

10. What are the applications of deep learning?

There are various applications of deep learning:
o Computer vision
o Natural language processing and pattern recognition
o Image recognition and processing
o Machine translation
o Sentiment analysis
o Question Answering system
o Object Classification and Detection
o Automatic Handwriting Generation

11. What do you understand by Autoencoder?

Autoencoder is an artificial neural network. It can learn representation for a set of data
without any supervision. The network automatically learns by copying its input to the output;
typically,internet representation consists of smaller dimensions than the input vector. As a
 result, they can learn efficient ways of representing the data. Autoencoder consists of two
parts; an encoder tries to fit the inputs to the internal representation, and a decoder
converts the internal state to the outputs.

12. What are the disadvantages of deep learning?

There are some disadvantages of deep learning, which are:
⮚ Deep learning model takes longer time to execute the model. In some cases, it
even takes several days to execute a single model depends on complexity.
⮚ The deep learning model is not good for small data sets, and it fails here.

13. What Is a Multi-layer Perceptron(MLP)?

As in Neural Networks, MLPs have an input layer, a hidden layer, and an output layer. It
has the same structure as a single layer perceptron with one or more hidden layers. A
single layer perceptron can classify only linear separable classes with binary output (0,1),
but MLP can classify nonlinear classes.
Except for the input layer, each node in the other layers uses a nonlinear activation
function. This means the input layers, the data coming in, and the activation function is
based upon all nodes and weights being added together, producing the output. MLP uses a
DEEP LEARNING
supervised learning method called “backpropagation.” In backpropagation, the neural
network calculates the error with the help of cost function. It propagates this error backward
from where it came (adjusts the weights to train the model more accurately).

14. What Do You Understand by Backpropagation?

This is one of the most frequently asked deep learning interview questions.
Backpropagation is a technique to improve the performance of the network. It
backpropagates the error and updates the weights to reduce the error.

15. What Is the Difference Between a Feedforward Neural Network and Recurrent Neural
Network?
In this deep learning interview question, the interviewee expects you to give a detailed
answer.
A Feedforward Neural Network signals travel in one direction from input to output. There
are no feedback loops; the network considers only the current input. It cannot memorize
previous inputs (e.g., CNN).
A Recurrent Neural Network’s signals travel in both directions, creating a looped network. It
considers the current input with the previously received inputs for generating the output of a
layer and can memorize past data due to its internal memory.
 DEEP LEARNING

16. What Are Hyperparameters?

This is another frequently asked deep learning interview question. With neural networks,
you’re usually working with hyperparameters once the data is formatted correctly. A
hyperparameter is a parameter whose value is set before the learning process begins. It
determines how a network is trained and the structure of the network (such as the number
of hidden units, the learning rate, epochs, etc.).

17. What are Neural Networks? What are the types of Neural networks?
In simple words, a neural network is a connection of many very tiny processing elements
called as neurons. There are two types of neural network-
Biological Neural Networks– These are made of real neurons.Those tiny CPU’s which
you have got inside your brain..if u have..Not only brain,,but neurons actually make the
whole nervous system.
Artificial Neural Networks– Artificial Neural Networks is an imitation of Biological Neural
Networks,,by artificial designing small processing elements, in lieu of using digital
computing systems that have only the binary digits. The Artificial Neural Networks are
 basically designed to make robots give the human quality efficiency to the work.

18. How human brain works?

It is weird at the same time amazing to know that we really do not know how we think.
Biologically, neurons in human brain receive signals from host of fine structures called as
dendrites. The neuron sends out spikes of electrical activity through a long, thin stand
known as an axon, which splits into thousands of branches. At the end of each branch, a
structure called a synapse converts the activity from the axon into electrical effects that
inhibit or excite activity from the axon into electrical effects that inhibit or excite activity in
the connected neurons. When a neuron receives excitation input that is sufficiently large
compared with its inhibitory input, it sends a spike of electrical activity down its axon.
Learning occurs by changing the effectiveness of the synapses so that the influence of one
neuron on another changes.

19. What is simple Artificial Neuron?

It is simply a processor with many inputs and one output….It works in either the Training
Mode or Using Mode. In the training mode, the neuron can be trained to fire (or not), for
particular input patterns. In the using mode, when a taught
DEEP input pattern is detected at the
LEARNING
input, its associated output becomes the current output. If the input pattern does not belong
in the taught list of input patterns, the firing rule is used to determine whether to fire or not.
An Artificial Neuron

20. What is Regularization?

Simple speaking: Regularization refers to a set of different techniques that lower the
complexity of a neural network model during training, and thus prevent the overfitting.

21. What are the applications of neural networks?

⮚ Used in medical field
⮚ Used in telephone communication
⮚ Business applications

16- Marks

1. What is deep learning, explain its uses and application and history.
2. Draw and explain the concept of neurons.
3. Define neural networks and explain the various layers of neural networks.
4. Explain the concepts of perceptron and how it works.
5. Briefly discuss single layer and multi-layer perceptron.
6. Discuss the feed forward and back propagation networks.
7. What is regularization, how does Regularization help reduce Over fitting?
 DEEP LEARNING

UNIT 2 - DEEP LEARNING ARCHITECTURES

1. What are the different types of deep neural networks?
Following are the different types of deep neural networks:-
• FeedForward Neural Network:- This is the most basic type of neural network, in
which flow control starts at the input layer and moves to the output layer. These networks
only have a single layer or a single hidden layer. There is no backpropagation mechanism
in this network because data only flows in one way. The input layer of this network
receives the sum of the weights present in the input. These networks are utilised in the
computer vision-based facial recognition method.
 ●
● Radial Basis Function Neural Network:- This type of neural network usually has more
than one layer, preferably two. The relative distance from any location to the center is
determined in this type of network and passed on to the next layer. In order to avoid
blackouts, radial basis networks are commonly employed in power restoration systems to
restore power in the shortest period possible.
● Multi-Layer Perceptrons (MLP):- A multilayer perceptron (MLP) is a type of feedforward
artificial neural network (ANN). MLPs are the simplest deep neural networks, consisting of
DEEP LEARNING
a succession of completely linked layers. Each successive layer is made up of a collection
of nonlinear functions that are the weighted sum of all the previous layer's outputs
(completely linked). Speech recognition and other machine learning systems rely heavily
on these networks.

● Convolutional Neural Network (CNN):- Convolutional Neural Networks are mostly used
in computer vision. In contrast to fully linked layers in MLPs, one or more convolution
layers extract simple characteristics from input by performing convolution operations in
CNN models. Each layer is made up of nonlinear functions of weighted sums at various
coordinates of spatially close subsets of the previous layer's outputs, allowing the weights
to be reused.
The AI system learns to automatically extract the properties of these inputs to fulfill a
specific task, such as picture classification, face identification, and image semantic
segmentation, given a sequence of images or videos from the actual world.
● Recurrent Neural Network (RNN):- Recurrent Neural Networks were created to solve the
sequential input data time-series problem. RNN's input is made up of the current input and
prior samples. As a result, the node connections create a directed graph. Furthermore,
each neuron in an RNN has an internal memory that stores the information from previous
samples' computations. Because of their superiority in processing data with a variable
DEEP LEARNING
input length, RNN models are commonly employed in natural language processing (NLP).
The goal of AI in this case is to create a system that can understand human-spoken
natural languages, such as natural language modeling, word embedding, and machine
translation.
Each successive layer in an RNN is made up of nonlinear functions of weighted sums of
outputs and the preceding state. As a result, the basic unit of RNN is termed "cell," and
each cell is made up of layers and a succession of cells that allow recurrent neural
network models to be processed sequentially.

● Modular Neural Network:- This network is made up of numerous tiny neural networks,
rather than being a single network. The sub-networks combine to form a larger neural
network, which operates independently to achieve a common goal. These networks are
extremely useful for breaking down a large-small problem into smaller chunks and then
solving it.
 ● Sequence to Sequence Model:- In most cases, this network is made up of two RNN
networks. The network is based on encoding and decoding, which means it has an
encoder that processes the input and a decoder that processes the output. This type of
network is commonly employed for text processing when the length of the inputting text
differs from the length of the outputted text.
2. Features of Neural Network DEEP LEARNING
1. NN systems are modelled on the human brain and nervous system
2. It can extract data without any input from the user
3. Every node is essentially a machine learning algorithm
4. Useful when solving problems for which the data set is very large

3. What Are Unsupervised Machine Learning Techniques?

There are two techniques used in unsupervised learning: clustering and association.
Clustering
Clustering problems involve data to be divided into subsets. These subsets, also called
clusters, contain data that are similar to each other. Different clusters reveal different
details about the objects, unlike classification or regression.

Association
In an association problem, we identify patterns of associations between different variables
or items.
For example, an e-commerce website can suggest other items for you to buy, based on
the prior purchases that you have made, spending habits, items in your wish list, other
customers’ purchase habits, and so on.
4. What is the Difference Between Supervised and Unsupervised Machine Learning?
• Supervised learning - This model learns from the labeled data and makes a future
prediction as output
• Unsupervised learning - This model uses unlabeled input data and allows the
algorithm to act on that information without guidance.

5, How neural network works? DEEP LEARNING

Observe the image given below:

This image shows the representation of how neural networks work. This image shows that
neural network is divided into different layers and each layer is divided into a block that
accomplishes its own task and then passes to the next layer. The first layer of neural
network is known as input layer that acquires the data and feed it to the neural network.
Some hidden layers are there which are not visible but all processing occurs in these
layers. These hidden layers have its own machine learning algorithm which is executes on
the data received from the input layer. Then the processed output is fed to subsequent
hidden layer. Similarly, at the last hidden layer passes the final processed data to the
output layer.

6. What is the difference between Machine Learning and Deep Learning?

Machine Learning forms a subset of Artificial Intelligence, where we use statistics and
algorithms to train machines with data, thereby, helping them improve with experience.
Deep Learning is a part of Machine Learning, which involves mimicking the human brain in
 terms of structures called neurons, thereby, forming neural networks.

7. How is Deep Learning better than Machine Learning?

Machine Learning is powerful in a way that it is sufficient to solve most of the problems.
However, Deep Learning gets an upper hand when it comes to working with data that has
a large number of dimensions. With data that is large in size, a Deep Learning model can
easily work with it as it is built to handle this.

8. What are some of the most used applications of Deep Learning?

Deep Learning is used in a variety of fields today. The most used ones are as follows:
• Sentiment Analysis
• Computer Vision
• Automatic Text Generation
• Object Detection
• Natural Language Processing
• Image Recognition

9. What is the meaning of overfitting?

Overfitting is a very common issue when working with Deep Learning. It is a scenario
where the Deep Learning algorithm vigorously hunts
DEEPthrough the data to obtain some valid
LEARNING
information.
This makes the Deep Learning model pick up noise rather than useful data, causing very
high variance and low bias. This makes the model less accurate, and this is an
undesirable effect that can be prevented.

10. What are some of the Deep Learning frameworks or tools that you have used?
This question is quite common in a Deep Learning interview. Make sure to answer based
on the experience you have with the tools.
However, some of the top Deep Learning frameworks out there today are:
• TensorFlow
• Keras
• PyTorch
• Caffe2
• CNTK
• MXNet
• Theano

11. What is the use of the Activation function?

The activation function is used to introduce nonlinearity into the neural network so that it
can learn more complex function. Without the Activation function, the neural network
would be only able to learn function, which is a linear combination of its input data.
Activation function translates the inputs into outputs. The activation function is responsible
for deciding whether a neuron should be activated or not. It makes the decision by
calculating the weighted sum and further adding bias with it. The basic purpose of the
activation function is to introduce non-linearity into the output of a neuron.

12. What is ReLU function?

A node or unit which implements the activation function is referred to as a rectified linear
activation unit or ReLU for short. Generally, networks that use the rectifier function for the
hidden layers are referred to as rectified networks.
Adoption of ReLU may easily be considered one of the few milestones in the deep
learning revolution.

13. How many types of activation function are available?

⮚ Binary Step
⮚ Sigmoid
 ⮚ Tanh
⮚ ReLU
⮚ Leaky ReLU
⮚ Softmax
⮚ Swish

14. what are the different layers of Autoencoders?

An Autoencoder consist of three layers:
• Encoder
• Code
• Decoder

DEEP LEARNING

15. What do you understand by Boltzmann Machine?

A Boltzmann machine (also known as stochastic Hopfield network with hidden units) is a
type of recurrent neural network. In a Boltzmann machine, nodes make binary decisions
with some bias. Boltzmann machines can be strung together to create more sophisticated
systems such as deep belief networks. Boltzmann Machines can be used to optimize the
solution to a problem.
Some important points about Boltzmann Machine-
o It uses a recurrent structure.
o It consists of stochastic neurons, which include one of the two possible states,
either 1 or 0.
o The neurons present in this are either in an adaptive state (free state) or clamped
state (frozen state).
o If we apply simulated annealing or discrete Hopfield network, then it would
become a Boltzmann Machine.

16. What is Deep Learning?

Deep Learning involves taking large volumes of structured or unstructured data and using
complex algorithms to train neural networks. It performs complex operations to extract
hidden patterns and features (for instance, distinguishing the image of a cat from that of a
dog)
17. What is a Boltzmann machine?
A Boltzmann machine is a type of recurrent neural network that uses binary decisions,
alongside biases, to function. These neural networks can be hooked up together to create
DEEPand
deep belief networks, which are very sophisticated LEARNING
used to solve the most complex
problems out there.

18. What is a CNN?

CNNs are convolutional neural networks that are used to perform analysis on images and
visuals. These classes of neural networks can input a multi-channel image and work on it
easily.
These Deep Learning questions must be answered in a concise way. So make sure to
understand them and revisit them if necessary.

19. What are the various layers present in a CNN?

There are four main layers that form a convolutional neural network:
• Convolution: These are layers consisting of entities called filters that are used as
parameters to train the network.
• ReLu: It is used as the activation function and is always used with the convolution
layer.
• Pooling: Pooling is the concept of shrinking the complex data entities that form
after convolution and is primarily used to maintain the size of an image after shrinkage.
• Connectedness: This is used to ensure that all of the layers in the neural network
are fully connected and activation can be computed using the bias easily.

20. What is a Restricted Boltzmann Machine?

A Restricted Boltzmann Machine, or RBM for short, is an undirected graphical model that
is popularly used in Deep Learning today. It is an algorithm that is used to perform:
• Dimensionality reduction
• Regression
• Classification
• Collaborative filtering
• Topic modeling
Next up on this top Deep Learning interview questions and answers blog, let us take a
look at the advanced questions.

21. What are some of the limitations of Deep Learning?

There are a few disadvantages of Deep Learning as mentioned below:
• Networks in Deep Learning require a huge amount of data to train well.
• Deep Learning concepts can be complex to implement sometimes.
• Achieving a high amount of model efficiency is difficult in many cases.
 These are some of the vital advanced deep learning interview questions that you have to
know about!

22. What are some of the examples of supervised learning algorithms in Deep
Learning?
There are three main supervised learning algorithms in Deep Learning:
• Artificial neural networks
• Convolutional neural networks
• Recurrent neural networks

23. What are some of the examples of unsupervised learning algorithms in Deep
Learning?
There are three main unsupervised learning algorithms in Deep Learning:
• Autoencoders
• Boltzmann machines
• Self-organizing maps

24. What Is the Role of Activation Functions in a Neural Network?

At the most basic level, an activation function decides whether a neuron should be fired or
not. It accepts the weighted sum of the inputs and biasLEARNING
DEEP as input to any activation function.
Step function, Sigmoid, ReLU, Tanh, and Softmax are examples of activation functions.

25. What Are the Softmax and ReLU Functions?

Softmax is an activation function that generates the output between zero and one. It
divides each output, such that the total sum of the outputs is equal to one. Softmax is often
used for output layers.

ReLU (or Rectified Linear Unit) is the most widely used activation function. It gives an
output of X if X is positive and zeros otherwise. ReLU is often used for hidden layers.
 DEEP LEARNING
26. What are some of the uses of Autoencoders in Deep Learning?
⮚ Autoencoders are used to convert black and white images into colored images.
⮚ Autoencoder helps to extract features and hidden patterns in the data.
⮚ It is also used to reduce the dimensionality of data.
⮚ It can also be used to remove noises from images.

27. What do you understand by Leaky ReLU activation function?

Leaky ReLU is an advanced version of the ReLU activation function. In general, the ReLU
function defines the gradient to be 0 when all the values of inputs are less than zero. This
deactivates the neurons. To overcome this problem, Leaky ReLU activation functions are
used. It has a very small slope for negative values instead of a flat slope.

28. What are the different layers of Autoencoders? Explain briefly.

An autoencoder contains three layers:
Encoder
The encoder is used to compress the input into a latent space representation. It encodes
the input images as a compressed representation in a reduced dimension. The
compressed images are the distorted version of the original image.
Code
The code layer is used to represent the compressed input which is fed to the decoder.
Decoder
The decoder layer decodes the encoded image back to its original dimension. The
decoded image is a reduced reconstruction of the original image. It is automatically
reconstructed from the latent space representation.

16 marks

1. What is deep learning? How it differs from machine learning and various applications of deep learning.
2. Explain the Representation Learning and methods
3. What is an activation function? Explain the rectified linear activation function.
4. Discuss the activation function LRELU and ERELU
5. Write a detailed note on Unsupervised Training of Neural Networks.
6. What are advantages and disadvantages of deep learning discuss with real time example
7. What are Autoencoders? Explain in detail?

DEEP LEARNING

UNIT 3 - CONVOLUTIONAL NEURAL NETWORK

1. Why do we prefer Convolutional Neural networks (CNN) over Artificial Neural

networks (ANN) for image data as input?
 1.Feedforward neural networks can learn a single feature representation of the image but
in the case of complex images, ANN will fail to give better predictions, this is because it
cannot learn pixel dependencies present in the images.

2. CNN can learn multiple layers of feature representations of an image by applying filters,
or transformations.

3. In CNN, the number of parameters for the network to learn is significantly lower than the
multilayer neural networks since the number of units in the network decreases, therefore
reducing the chance of overfitting.

2. What is ResNet?
ResNet, short for Residual Network is a specific type of neural network that was
introduced in 2015 by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun in their
paper “Deep Residual Learning for Image Recognition”.The ResNet models were
extremely successful which you can guess from the following:

⮚ Won 1st place in the ILSVRC 2015 classification competition with a top-5 error
rate of 3.57% (An ensemble model)
⮚ Won the 1st place in ILSVRC and DEEP COCOLEARNING
2015 competition in ImageNet
Detection, ImageNet localization, Coco detection and Coco segmentation.
⮚ Replacing VGG-16 layers in Faster R-CNN with ResNet-101. They observed
relative improvements of 28%
⮚ Efficiently trained networks with 100 layers and 1000 layers also.

3. What AlexNet Brought To The World Of Deep Learning

Take a minute to understand the trivial techniques and neural network architecture that
revolutionized deep learning approaches
The AlexNet convolutional neural network(CNN) was introduced in the year 2012. Since
then, the utilization of deep convolutional neural network has skyrocketed to the point
where several machine learning solutions leverage deep CNNs.
This article will present the essential findings, and talking points of the research paper, in
which the AlexNet architecture was introduced.

4. CNN real world application?

CNN has been used in a variety of real-world applications, including cancer detection and
biometric authentication. CNN can also be applied to visual question answering or image
captioning where CNN networks take an input image and generate natural language
answers about that image.

5. Need for ResNet

Mostly in order to solve a complex problem, we stack some additional layers in the Deep
Neural Networks which results in improved accuracy and performance. The intuition
behind adding more layers is that these layers progressively learn more complex features.
For example, in case of recognising images, the first layer may learn to detect edges, the
second layer may learn to identify textures and similarly the third layer can learn to detect
objects and so on. But it has been found that there is a maximum threshold for depth with
the traditional Convolutional neural network model. Here is a plot that describes error% on
training and testing data for a 20 layer Network and 56 layers Network.

6. What Are the Different Layers on CNN?

There are four layers in CNN:
1. Convolutional Layer - the layer that performs a convolutional operation, creating
several smaller picture windows to go over the data.
2. ReLU Layer - it brings non-linearity to the network and converts all the negative
 pixels to zero. The output is a rectified feature map.
3. Pooling Layer - pooling is a down-sampling operation that reduces the
dimensionality of the feature map.
4. Fully Connected Layer - this layer recognizes and classifies the objects in the
image.

7. What is Pooling on CNN, and How Does It Work?

Pooling is used to reduce the spatial dimensions of a CNN. It performs down-sampling
operations to reduce the dimensionality and creates a pooled feature map by sliding a filter
matrix over the input matrix.

DEEP LEARNING

16 Marks

1. Draw and explain the architecture of convolutional network

2. Explain the following.
(i) striding
(ii) pooling
3. Give a detailed note on Resnet?
4. Briefly discuss the concept of AlexNet?
5. Explain the various applications of CNN.

UNIT 4 SEQUENCE MODELING: RECURRENT NETS

1. What is Unfolding Computational Graphs

• A Computational Graph is a way to formalize the structure of a
set of computations
• Such as mapping inputs and parameters to outputs and loss
• We can unfold a recursive or recurrent computation into a
 computational graph that has a repetitive structure
• Corresponding to a chain of events
• Unfolding this graph results in sharing of parameters across a
deep network structure
2. What are some applications of Bidirectional RNN?
⮚ Translation.
⮚ Handwritten Recognition.
⮚ Protein Structure Prediction.
⮚ Part-of-speech tagging.
⮚ Dependency Parsing.
⮚ Entity Extraction.
3. What are some limitations of Bidirectional RNN?
One limitation with BRNN is that the entire sequence must be available before we can
make predictions. For some applications such as real-time speech recognition, the entire
utterance may not be available and BRNN may not be adequate.

In the case of language models, the task is to predict the next word given preceding
words. BRNN is clearly not suitable since it expects future words as well. Applying BRNN
DEEP LEARNING
in this application will give poor accuracy. Moreover, BRNN is slower than RNN since
results of the forward pass must be available for the backward pass to proceed. Gradients
will therefore have a long dependency chain.

LSTMs capture long-term dependencies better than RNN and also solve the
exploding/vanishing gradient problem. However, stacking many layers of BiLSTM creates
the vanishing gradient problem. Deep neural networks so successful with CNNs are not so
successful with BiLSTMs.
4. Build an Encoder-Decoder Model With Recurrent Neural Networks
For better understanding, we can divide the model into three basic components:

From “Understanding Encoder-Decoder Sequence to Sequence Model” by Simeon

Kostadinov [3]
The encoder
Layers of recurrent units where, in each time step, an input token is received, collecting
relevant information and producing a hidden state. This depends on the type of RNN; in
our example, a LSTM, the unit mixes the current hidden state and the input and returns an
output, discarded, and a new hidden state.
The encoder vector
The encoder vector is the last hidden state of the encoder, and it tries to contain as much
of the useful input information as possible to help the decoder get the best results. It’s the
only information from the input that the decoder will get.
The decoder
Layers of recurrent units — e.g., LSTMs — where each unit produces an output at a time
step t. The hidden state of the first unit is the encoder vector, and the rest of the units
 accept the hidden state from the previous unit. The output is calculated using a softmax
function to obtain a probability for every token in the output vocabulary.
5. What Are the Applications of a Recurrent Neural Network (RNN)?
The RNN can be used for sentiment analysis, text mining, and image captioning.
Recurrent Neural Networks can also address time series problems such as predicting the
prices of stocks in a month or quarter.
6. Why Recurrent Neural Networks?
RNN were created because there were a few issues in the feed-forward neural network:

Cannot handle sequential data

Considers only the current input
Cannot memorize previous inputs
The solution to these issues is the RNN. An RNN can handle sequential data, accepting
the current input data, and previously received inputs. RNNs can memorize previous
inputs due to their internal memory.
7. What is an RNN?
RNN stands for Recurrent Neural Networks. These are the artificial neural networks which
are designed to recognize patterns in sequences of data such as handwriting, text, the
spoken word, genomes, and numerical time series data. RNN use backpropagation
algorithm for training because of their internal DEEP
memory. RNN can remember important
LEARNING
things about the input they received, which enables them to be very precise in predicting
what's coming next.
8. What are the issues faced while training in Recurrent Networks?
Recurrent Neural Network uses backpropagation algorithm for training, but it is applied on
every timestamp. It is usually known as Back-propagation Through Time (BTT).
There are two significant issues with Back-propagation, such as:
Vanishing Gradient
When we perform Back-propagation, the gradients tend to get smaller and smaller
because we keep on moving backward in the Network. As a result, the neurons in the
earlier layer learn very slowly if we compare it with the neurons in the later layers.Earlier
layers are more valuable because they are responsible for learning and detecting simple
patterns. They are the building blocks of the network.
If they provide improper or inaccurate results, then how can we expect the next layers o
and complete network to perform nicely and provide accurate results. The training
procedure tales long, and the prediction accuracy of the model decreases.
9. Advantages and disadvantages of RNN
Advantages of Recurrent Neural Networks
1. Model sequential data where each sample can be assumed to be dependent on
historical ones is one of the advantage.
2. Used with convolution layers to extend the pixel effectiveness.
Disadvantages of Recurrent Neural Networks
1. Gradient vanishing and exploding problems
2. Training recurrent neural nets could be a difficult task
3. Difficult to process long sequential data using ReLU as an activation function.

16 Marks

1. What is Recurrent Neural Networks explain its detailed concepts?

2. Give a detail note on Bidirectional Rnn. Discuss the Difference Between BRNN’s and Recurrent Neural
Networks?
3. Explain the concept of Encoder Decoder sequence to sequence architectures.
4. Discuss the Deep Recurrent Networks with a neat sketch?
 DEEP LEARNING

UNIT 5 ADVANCED DEEP LEARNING TECHNIQUES

1. Why are generative adversarial networks (GANs) so popular?

Generative adversarial networks are used for a variety of purposes. In the case of working with
images, they have a high amount of traction and efficient working.
• Creation of art: GANs are used to create artistic images, sketches, and paintings.
• Image enhancement: They are used to greatly enhance the resolution of the input images.
• Image translation: They are also used to change certain aspects, such as day to night and
summer to winter, in images easily.

2. Why were GANs developed in the first place?

It has been noticed most of the mainstream neural nets can be easily fooled into misclassifying
things by adding only a small amount of noise into the original data. Surprisingly, the model after
adding noise has higher confidence in the wrong prediction than when it predicted correctly. The
reason for such adversary is that most machine learning models learn from a limited amount of
data, which is a huge drawback, as it is prone to overfitting. Also, the mapping between the input
and the output is almost linear. Although, it may seem that the boundaries of separation between
 the various classes are linear, but in reality, they are composed of linearities and even a small
change in a point in the feature space might lead to misclassification of data.

4. Different types of GANs:

GANs are now a very active topic of research and there have been many different types of GAN
implementation. Some of the important ones that are actively being used currently are described
below:
1. Vanilla GAN: This is the simplest type GAN. Here, the Generator and the Discriminator are
simple multi-layer perceptrons. In vanilla GAN, the algorithm is really simple, it tries to optimize the
mathematical equation using stochastic gradient descent.
2. Conditional GAN (CGAN): CGAN can be described as a deep learning method in which
some conditional parameters are put into place. In CGAN, an additional parameter ‘y’ is added to
the Generator for generating the corresponding data. Labels are also put into the input to the
Discriminator in order for the Discriminator to help distinguish the real data from the fake generated
data.
3. Deep Convolutional GAN (DCGAN): DCGAN is one of the most popular also the most
successful implementation of GAN. It is composed of ConvNets in place of multi-layer perceptrons.
The ConvNets are implemented without max pooling, which is in fact replaced by convolutional
stride. Also, the layers are not fully connected.
4. Laplacian Pyramid GAN (LAPGAN): The Laplacian DEEP pyramid
LEARNING is a linear invertible image
representation consisting of a set of band-pass images, spaced an octave apart, plus a low-
frequency residual. This approach uses multiple numbers of Generator and Discriminator networks
and different levels of the Laplacian Pyramid. This approach is mainly used because it produces
very high-quality images. The image is down-sampled at first at each layer of the pyramid and then
it is again up-scaled at each layer in a backward pass where the image acquires some noise from
the Conditional GAN at these layers until it reaches its original size.
5. Super Resolution GAN (SRGAN): SRGAN as the name suggests is a way of designing a
GAN in which a deep neural network is used along with an adversarial network in order to produce
higher resolution images. This type of GAN is particularly useful in optimally up-scaling native low-
resolution images to enhance its details minimizing errors while doing so.

5. Application of GANs
• With the help of DCGANs, you can train images of cartoon characters for generating faces
of anime characters as well as Pokemon characters.

• GANs can be trained on the images of humans to generate realistic faces. The faces that
you see below have been generated using GANs and do not exist in reality.

• GANs can build realistic images from textual descriptions of objects like birds, humans, and
other animals. We input a sentence and generate multiple images fitting the description.

6. What is Deepfake?
They constitute content that is fake in the form of media like videos and forms like pictures or
audios. They are created using artificial intelligence where data is fed on a computer to find a new
face.

7. What is the technology used?

1. It is called Generative Adversarial Networks or GAN. It uses two AI algorithms one of which
generates fake content and other grades the efforts teaching the system to be better.
2. GAN comes up with new faces of humans which is also available on the website
'www.thispersondoesnotexist.com'
3. It is however virtually not possible to differentiate the real image from a fake one.
3. How does GANs work?
Generative Adversarial Networks (GANs) can be broken down into three parts:
• Generative: To learn a generative model, which describes how data is generated in terms
of a probabilistic model.
• Adversarial: The training of a model is done in an adversarial setting.
• Networks: Use deep neural networks as the artificial intelligence (AI) algorithms for training
purpose.

16 Marks

1. What are Boltzmann Machines? How do Restricted Boltzmann Machines work?

2. Explain Generative Neural Networks in detail.
3. Discuss the recent trends on designing deep learning solutions for identifying fake fingerprints.
4. Describe the design principles for identifying fake images.
5. Discuss the detail steps carried out for identify fake video.

DEEP LEARNING