Q-1.

Linear classifiers and stochastic gradient descent

Task 1 : Information is sent over a noisy underwater acoustic channel where the noisy signal
T
[ x 1 , ... , x D ] is collected as input vector X and the transmitted signal y is predicted using a linear
regression filter, i.e., y=w T X + b . Consider the following function as stochastic loss,

2
(
l ( w , X i , y i )=log 1+‖ y i −wT X i‖ . )

Derive the SGD update for vector w that minimizes the loss.
(Write your answer here. For equations/formulae, you can math editor or write on paper and then paste a scanned image,
however, the text of answer should be typewritten).
 Task 2

Assume the transmitted signal is in binary where y ∈ [ −1,1 ] rendering the problem into a
classification task. The prediction f ( X ) is given by:

f ( X )=sign ( wT X ) while the loss function is now:

l ( w , X i , y i )=log ( 1+ max ( 0 ,− y i wT X i ) ). 

1. Find the SGD update for w.

(Write your answer here. For equations/formulae, you can math editor or write on paper and then paste a scanned image, however, the text of
answer should be typewritten)
 Q-2 : Nonlinear classification using MLP

Let build a multi-layer perceptron to implement XOR function (as shown in figure below)

b1
x1 w11
Σ1 sign w31
Output
w32 Σ sign
w21 w12
x2 b3
Σ2 sign x1 x2 output
w22
1 b2 1 0 0 0
0 1 1
1 0 1
1 1 0

Task 1: Prove that the above problem could not be solved by a single layer (linear)
perceptron.
Task 2: Find the weights wij and bj for the two layer network shown above.
(Note: You can either use training with SGD update or find the weights by careful
inspection)
Task 3: By looking at the two-layer perceptron implementation, it can be shown that XOR
is have equivalence to a combination of logic AND and OR as per the following relation.
Show (briefly) by identifying the components of your two-layer network that implement
these operations.
x 1 ⊕ x 1=( x 1 ∨ x2 ) ∧¬ ( x 1 ∧ x 2 )
Q-3 : Convolutional Neural Networks and back-propagation

Suppose you work at a botanical garden where part your job description is to find areas in
the garden with unwanted dandelions ( ). You are provided with a drone that flies over
the garden and capture (200x200) images. Lets you have 1000 images some containing
dandelions and others not, and all these 1000 images are labeled. The shape, size, and
location of the flower can be different in different images. You are to design and train a
scanning network that can “decide” on the existence or absence of dandelion in the image
irrespective of its location in the image.
Task 1 : Provide a step-by-step guide on to construct such model, including number of
hidden layers, activation, number of hidden units in each layer along-with justification.
(You should draw the diagram of network here. Again, if it’s difficult to draw digitally, you
can draw it on paper and paste the scanned image here, text should be typewritten)
Task 2 : Do the dimensional analysis at each stage of the network starting from the input
layer till the output layer.

We'll use the VGG-16 architecture to analyse the images, but since the raw data is in
200*200 dimensions instead of 224*224, we'll add an additional padding of 26 (13 on each
side) to compensate. The rest of the network is built around the VGG-16 protocol. The
initial convolution filter is 4*4*3, and the remaining convolution filters are 3*3*previous
layers. The swimming pool. After each conv layer, the pooling layer is used to collapse the
image's w and h, speeding up the process of reaching the completely bound, one-
dimensional layer. The completely linked layers function in the same way as a conventional
neural network, with the number of hidden units decreasing as they progress, and an
additional layer added at the end for classification of ‘dandelion' or ‘no dandelion.'. The
network's depth means that the classification of the dandelion is more accurate. The VGG-
16 architecture is used to increase the number of hidden units, which provides better
performance.
Task 3 : Briefly describe the effect of scanning window (size) on the training and
performance in case of using an undistributed giant MLP.

The effect of scanning window with large size can be good or bad as well. The performance
of the model would become worst because some features will be affected due to the large
size which is not good but training would become faster. If we reduce the size training
become slower and performance gets better.