ORIGINAL NATIONAL UNIVERSITY OF SINGAPORE FACULTY OF ENGINEERING EXAMINATION FOR (Semester I: 2017/2018) EE5904/ME5404 — NEURAL NETWORKS, April/May 2018 — Time Allowed: 2.5 Hours INSTRUCTIONS TO CAND) . This paper contains FOUR (4) questions and comprises FOUR (4) printed pages. pos ‘The examination paper carries 100 marks in total. All questions are compulsory. Answer ALL questions. This is a closed book examination. But the candidate is permitted to bring into the examination hall a single A4 size data sheet. ‘The candidate may refer to this sheet during the examination Programmable calculators aré not allowed.EES904/MES404 Neural Networks / Page 2 Q.1 Consider the following two-dimensional pattern recognition problem. Class I contains four points: (1, 1), (0, 1), 1, 1) and (0, -1). Class II contains two points: (I, -1), (1, -1). (a) Is this two-class pattern classification problem linearly separable or nonlinearly separable? Please supply a rigorous mathematical proof for your answer. (8 marks) (b) Design a MLP to separate these two classes completely. You are free to choose any number of hidden layers as well as any number of hidden neurons in each layer. The only constraint is that the activation functions for all the neurons, including the hidden neurons and output neurons, are hard limiters, ic. step functions. (12 marks) (©) Explain why the MLP designed in (b) can solve this pattern classification problem. (S marks) Q2 (a) The Radial Basis Function Networks (RBFN) can be used to solve regression problems. The output of the radial basis function network is described by y= w.9,(x-1, +> * Suppose that you are given a set of pairs of sampling points and desired outputs {Gx(i), d@), iF1.....N}, ‘Assume that the parameters (ie,, the centers and widths) of the radial basis units, @, have already been determined by unsupervised learning. Derive the formula to compute the optimal weights w,and the bias b. When you solve this problem, please try to cope with the issue of over-fitting, Please note that very few marks will be awarded if only the formula is supplied without any mathematical justifications. (15 marks) (b) If the size of the training set is huge (for instance, N>I million), it might be difficult to use the formula obtained in part (a). Suggest an alternative training algorithm for determining the weights of RBEN to deal with large training set. (5 marks) (©) Give one specific application example where SOM can be used. You cannot use any of the examples discussed in the lecture and the assignment. There is no need to supply the detailed algorithm for SOM, but you need to explain why SOM may be suitable for the particular example you have chosen. (5 marks)BES904/MES404 Neural Networks / Page 3 Q3 A support vector machine is to be designed for the training set as shown in Figure 3.1, where the label for the points denoted by a plus sign is +1, while the label for the points denoted by triangle is —1. 4 * Figure 3.1 (@ For the hyperplane defined by the fine x, = 3, determine the functional margins of the examples and the functional margin of the training set. (5 marks) (b) Determine the margin of this training set. (5 marks) (© Which data point(s) can be removed without affecting the optimal hyperplane associated with the original data set? (Justify your answer.) (5 marks) (@ Suppose that a new data point (1, 3) with a label of ~1 is added to the original training set. Express explicitly the dual problem associated with a soft-margin SVM that uses the kernel K(x y) = (1+ x7y)? and C= 5. (10 marks)1BES904/MES404 Neural Networks / Page 4 Q4 Consider a reinforcement learning problem with a four-state grid environment as shown in Figure 4.1. Suppose that at each state an agent can take one of two actions, namely, ai (left or right) and a2 (up or down), and in doing so can only either remain at the same state or move to one of its bordering (i.c., non-diagonally adjacent) states, depending on how the transition function is defined. However, the movement of the agent is restricted such that the agent stays, in the grid when it takes an action (i.e., the action will not cause it to leave the grid). The reward function is given as: p(s,a,,s')= 1if's’ = s + 1, and p(s, a,,s") = 0 otherwise, where s and s' are states. Let y = 0.9. o;}i 3) 2 Figure 4.1 (a) Determine the Q-values for the policy: (0) = ay, #(1) = aa, (2) = ay, (3) = ap, with F(0,a;) = 1, F(1,a2) = 2, F(2,a1) = 3, and F(3,a2) = 0. (10 marks) (b) Suppose that the state transitions become non-deterministic. with the probabilities as shown in Table 4.1. Using the value-iteration algorithm of Dynamic Programming, determine the values of Q(0, a1) and Q(0, a2) for the first iteration, with the assumption that the initial value of any state-action pair is equal numerically to the state label; for example, Qinie(O, a1) = 0. From state with a1 _From state with a 0 if 2 3 0 1 2 3 3 O02 [08 of o7[ 04 0 0 1 fa{ ox} oi} ot 0 o| oi[ 06 0 a of o1[ 02] 02 oO; o9[ 03 0 s 0 o| 07] or] 06 o| 01 0 Table 4.1 (10 marks) (©) Discuss the role of exploration in reinforcement learning. (5 marks) END OF PAPER