Roll No.

: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Amrita Vishwa Vidyapeetham
Amrita School of Engineering, Coimbatore
B.Tech. Degree Examinations – May 2022
Sixth Semester
Computer Science and Engineering
19CSE456 Neural Networks & Deep Learning
Duration: Three hours Maximum: 100 Marks

CO Course Outcomes
CO01 Understand the learning components of neural networks and apply standard neural
network models to learning problems.
CO02 Analyze the learning strategies of deep learning – regularization, generalization,
optimization, bias, and variance.
CO03 Analyze regular deep learning models for training, testing, and validation in standard
datasets.
CO04 Apply neural networks for deep learning using standard tools.
CO05 Understand the mathematics for Deep learning.

Answer all questions

PART A (10*3 =30 Marks)

1) The XOR function (exclusive or) returns true only when one of the arguments is true and another
is false. Otherwise, it returns always false. Do you think it is possible to implement this function
using a single perceptron? Justify your answer graphically. What about a network of several units?
[CO1][BTL2]
2) Consider a feedforward neural network with 100 layers designed for a binary classification task,
using a sigmoid activation in the final layer, and a mixture of tanh and ReLU activations for all
other layers. You notice your weights to you’re a subset of your layers stop updating after the first
epoch of training, even though your network has not yet converged. A deeper analysis reveals the
gradients to these layers completely, or almost completely, go to zero very early on in training.
Which of the following fixes could help? (You also note that your loss is still within a reasonable
order of magnitude). Justify your answer. [CO3][BTL3]
i. Increase the size of your training set
ii. Switch the ReLU activations with leaky ReLUs everywhere
iii. Add Batch Normalization before every activation
iv. Increase the learning rate
3) Which of the following would you consider to be valid activation functions (elementwise non-
linearities) to train a neural net in practice? Justify your selection. [CO1][BTL3]

R Page 1 of 6
 4) List out the layers of a CNN architecture? You are benchmarking runtimes for layers commonly
encountered in CNNs. Which of the following: Conv layer (convolution operation + bias addition),
Max pooling, Average pooling, and Batch Normalization would you expect to be the fastest (in
terms of floating-point operations)? [CO1][BTL2]
5) Consider a nonlinear function that passes 1 if its input is nonnegative, else evaluates to 0, i.e.

Whether this nonlinear function is applicable in convolutional neural network architecture with the
Adam optimizer. Justify [CO2][BTL2]
6) Consider a deep neural network model to diagnose chest cancer using X-ray images. What do you
think might be the most appropriate evaluation metric and why: Accuracy, Precision, Recall, F1
score. [CO2][BTL2]

7) Explain the forward and backward propagation algorithm for a multi-layer perceptron.
[CO2][BTL1]

8) A neuron with 4 inputs has the weight vector w = [1 2 3 4] . The activation function is linear, that
is, the activation function is given by f(net) = 3 *net. If the input vector is X = [5 6 7 8] , then find
the output of the neuron. Justify your answer [CO5][BTL2]

9) Consider the given plot.

[CO2][BTL3]

R Page 2 of 6
 10) The architecture of a simple neural network is given below. The two neurons f,q perform the
following operations.

q=x+y
f=q*z
What is the gradient of F with respect to x, y, and z? Hint: Calculate (df/dx), (df/dy) and (df/dz)
[CO5][BTL3]

PART B (5*6=30 Marks)

1) (a) Logical operators (i.e. NOT, AND, OR, XOR, etc) are the building blocks of any computational
device. Logical functions return only two possible values, true or false, based on the truth or false
values of their arguments. For example, the operator AND returns true only when all its arguments are
true, otherwise (if any of the arguments is false) it returns false. Design a single perceptron with 2
inputs to implement logical AND and test how the neural AND function works (Specify the weights
and the activation function) [3 Marks][CO1][BTL3]
(b) Suggest how to change either the weights or the threshold level of this single unit in order to
implement the logical OR function (true when at least one of the arguments is true).
[3 Marks] [CO1][BTL3]

2) Explain L1 and L2 regularization and their weight updation in detail. [CO2][BTL1]

3) (a) Consider the perceptron given below, with bias b=5. Provide values for the two weights w1
and w2 that allow you to compute the NAND function (which is 0 if and only if both inputs are 1).
Show the complete computation with the chosen w’s for two input combinations (x1, x2)
[3 Marks] [CO1] [BTL3]

(b) What do you mean by hyperparameters in the context of deep learning? Give some examples
[3 Marks][CO3][BTL1]
4) (a) Briefly explain Autoencoders and their applications with a diagram [3 Marks][CO4][BTL2]

R Page 3 of 6
 (b) In terms of Dimensionality Reduction, how does Autoencoder differ from PCAs?
[3 Marks][CO4][BTL3]
5) Which of the following functions can be exactly represented by a neural network with one hidden
layer which uses linear and/or steps activation functions? For each case, justify your answer.
a) Polynomials of degree one
b) Hinge loss (h(x) = max(1-x,0))
c) Polynomials of degree two [CO5][BTL3]

PART C (4*10 =40 Marks)

1) The architecture of a convolutional neural network is given below. Fill in the shape of the output
volume and the number of parameters at each layer. You can write the activation shapes in the
format (H;W;C), where H;W;C are the height, width, and channel dimensions, respectively.
assume padding 1, stride 1 where appropriate. [CO4][BTL3]

2) Consider the following neural network architecture with fully connected layers and ReLU
activations, including two input units (i1; i2), four hidden units (h1; h2) and (h3; h4). The output
units are indicated as (o1; o2) and their targets are indicated as (t1; t2). The weights and bias of the
fully connected layer are represented as w and b with specific sub-descriptors. [CO5][BTL3]

R Page 4 of 6
 The values of variables are given in the following table:

Update the weight w21 using gradient descent with a learning rate of 0.1 as well as the loss computed
previously. (Please write down all your computations.). Assume that o1=0 and o2=-1.5 and the loss
function is MSE.

3) The architecture of a convolutional neural network is given below. It converts a 13x13 image into
4 output values. The network has the following layers/operations from input to output: convolution
with 3 filters, max pooling, ReLu, and finally a fully-connected layer. Assume that no bias is
associated with this architecture. Answer the following questions. [CO1][BTL3]

a) How many weights in the convolutional layer do we need to learn? Justify

R Page 5 of 6
 d) A fully-connected neural network with the same size layers as the above network (13x13 →
3x10x10 → 3x5x5 → 4x1) can represent any classifier that the above convolutional network
can represent. Is this statement true or false? Justify your answer
4) (a) What is the need of regularization in deep learning? [5 Marks] [CO2] [BTL2]
(b) Explain L1 & L2 regularization techniques in detail. [5 Marks] [CO2] [BTL2]

*******

Course Outcome /Bloom’s Taxonomy Level (BTL) Mark Distribution Table

CO Marks BTL Marks

CO01 28 BTL 1 12

CO02 28 BTL 2 28

CO03 6 BTL 3 60

CO04 16 BTL 4

CO05 22 BTL 5

CO06 BTL 6

R Page 6 of 6