Department of Electrical Engineering

Maulana Azad National Institute of Technology

AI TECHNIQUES
ASSIGNMENT 2

Submitted By - Submitted To -

Name - AYUSH CHOKHANI Dr. Shiwani Rai

Scholar No - 211113038
Q1. Derive the mathematical expression to update weights for three layer back
propagation neural networks.

Backpropagation—short for ‘backward propagation of errors’—is an optimization

algorithm used to improve the accuracy of artificial neural networks. It’s an essential
component of the gradient descent optimization process.

The deep neural network above contains an input layer, a hidden layer, and an output
layer. It contains two inputs (x1 and x2), three hidden units (nodes)(ℎ1,ℎ2, and ℎ3), and
two outputs (y1 and y2).

Backpropagation Concepts Explained

This section introduces key concepts and formulas to understand the more complex
backpropagation computations below.

The Sigmoid Function

The sigmoid (logistic function) is one of the most common non-linearities. We represent it
using the following equation.

σ(x)=1/1+e^−x

The sigmoid’s derivative formula is the following:

σ′(x)=σ(x) (1−σ(x))

The sigmoid activation function transforms the input values to obtain a new vector with
values comprising the next layer.

The L-2 Norm

We obtain this by calculating the sum of the squared differences between the outputs y and
the targets t. Its mathematical expression is the following:

L2-norm loss: L=1/2 ∑(yi−ti)^2

The linear model function equals:

f(x) = xw + b

where:

x – input

w – coefficient (weight)

b – intercept (bias)
Here, we’ll use a for the linear combination before activation, where:

a(1)=xw+b(1) and

a(2)=hu+b(2)

With this notation, the output y equals the activated linear combination. Since we cannot
exhaust all activation and loss functions, we focus on the most common
ones: sigmoid activation and L2-norm loss. Therefore, for the output layer, we have
y=σ(a(2)) while for the hidden layer we obtain h=σ(a(1)).

Backpropagation for the Output Layer

We obtain the update rule using the following function:

u←u−η∇uL(u)

Where η (eta) is the ML algorithm's learning rate.

How do we calculate ∇uL(u)?

Let's take a single weight uij. The partial derivative of the loss w.r.t. uij equals:))

(∂L/∂uij ) = (∂L/∂yj) (∂yj/∂a(2)j) (∂a(2)j/∂uij)

Where:

i corresponds to the previous layer (input layer for this transformation) and

j corresponds to the next layer (output layer of the transformation).

We compute the partial derivatives following the chain rule.

The first one is the L2-norm loss derivative:

∂L/∂yj = (yj−tj)

The second one is the sigmoid derivative:

∂yj/∂a(2)j = σ(a(2)j) (1−σ(a(2)j)) = yj (1−yj)

Finally, the third one is the derivative of a(2)=hu+b(2), which equals:

∂a(2)j/∂uij = hi
replacing the partial derivatives in the expression above, we get:
∂L/∂uij = ∂L/∂yj ∂yj/∂a(2)j ∂a(2)j/∂uij = (yj−tj) yj (1−yj) hi = δjhi
Therefore, we obtain the update rule for a single weight for the output layer using the
following:
 uij←uij−η δj hi

Backpropagation for a Hidden Layer

Similarly to the backpropagation of the output layer, the update rule for a single
weight, wij, is the following:

∂L/∂wij = ∂L/∂hj ∂hj/∂a(1)j ∂a(1)j/∂wij

Again, we compute backpropagation following the chain rule.

We use the sigmoid activation and linear model formulas to obtain the following:

∂hj/∂a(1)j = σ(a(1)j) (1−σ(a(1)j)) = hj (1−hj)

∂a(1)j/∂wij = xi
Let’s take the solution for weight w11 as an example:

∂L/∂h1 = ∂L/∂y1 ∂y1/∂a(2)1 ∂a(2)1/∂h1 + ∂L/∂y2 ∂y2/∂a(2)2 ∂a(2)2/∂h1

=(y1−t1)y1(1−y1)u11+(y2−t2)y2(1−y2)u12

Now, we can calculate ∂L/∂w11, which was the only missing part of the update rule for the
hidden layer. The final expression is:

∂L/∂w11 = [(y1−t1) y1 (1−y1) u11+(y2−t2) y2 (1−y2) u12] h1 (1−h1) x

The generalized form of this equation is:

∂L/∂wij = ∑k(yk−tk) yk (1−yk) ujk hj (1−hj) xi

Backpropagation Generalization

We combine the backpropagation for the output and the hidden layers to obtain the general
backpropagation formula with the L2-norm loss and sigmoid activations.

∂L/∂wij = δj xi
where for a hidden layer:

Δj = ∑kδk wjk yj (1−yj) xi

Q2. Explain different types of activation functions with help of mathematical
modelling and its characteristics?

Solution:
Activation functions play a crucial role in artificial neural networks by introducing
non-linearity into the model. They help neural networks approximate complex, non-linear
relationships in the data. In this explanation, we will discuss some common activation
functions, their mathematical modelling, and their characteristics.

1. Step Function:
- Mathematical Model:
f(x) = 1, if x > 0
f(x) = 0, if x <= 0
- Characteristics:
- Output is binary: 1 or 0.
- Discontinuous and non-differentiable at x = 0.
- Mainly used in binary classification tasks.

2. Sigmoid Function:
- Mathematical Model:
f(x) = 1 / (1 + e^(-x))
- Characteristics:
- Output is in the range (0, 1).
- Smooth, continuous, and differentiable.
- Has a problem of vanishing gradients for extreme input values.
- Commonly used in the output layer for binary classification tasks.

3. Hyperbolic Tangent (tanh) Function:

- Mathematical Model:
f(x) = (e^x - e^(-x)) / (e^x + e^(-x))
- Characteristics:
- Output is in the range (-1, 1).
- Smooth, continuous, and differentiable.
- Solves the vanishing gradient problem better than sigmoid.
- Symmetric around the origin.

4. Rectified Linear Unit (ReLU) Function:

- Mathematical Model:
f(x) = max(0, x)
- Characteristics:
- Output is x for x > 0 and 0 for x <= 0.
 - Simple, computationally efficient, and faster convergence.
- Prone to the vanishing gradient problem for x < 0.
- Commonly used in hidden layers for deep neural networks.

5. Leaky ReLU:
- Mathematical Model:
f(x) = x, if x > 0
f(x) = αx, if x <= 0
(α is a small positive constant)
- Characteristics:
- Similar to ReLU but allows a small gradient for x < 0.
- Mitigates the dying ReLU problem.
- Commonly used in deep neural networks.

6. Parametric ReLU (PReLU):

- Mathematical Model:
f(x) = x, if x > 0
f(x) = αx, if x <= 0
(α is a learnable parameter)
- Characteristics:
- Similar to Leaky ReLU, but α is learned from the data.
- Helps the network to adaptively decide the slope for each neuron.

7. Exponential Linear Unit (ELU):

- Mathematical Model:
f(x) = x, if x > 0
f(x) = α(e^x - 1), if x <= 0
(α is a positive constant)
- Characteristics:
- Similar to ReLU but differentiable and avoids the vanishing gradient problem.
- α determines the value at which ELU transitions from linear to exponential.

8. Scaled Exponential Linear Unit (SELU):

- Mathematical Model:
f(x) = λx, if x > 0
f(x) = λα(e^x - 1), if x <= 0
(α and λ are constants)
- Characteristics:
- An extension of ELU with additional scaling.
- Helps stabilise training and encourages self-normalising properties.
9. Swish Function:
- Mathematical Model:
f(x) = x / (1 + e^(-x))
- Characteristics:
- Smooth, non-monotonic, and differentiable.
- Performs well in practice and has shown promising results.

Each activation function has its own characteristics and is suitable for different scenarios.
Choosing the right activation function depends on the specific problem, network
architecture, and the challenges of training. It's common to experiment with different
activation functions to find the one that works best for a given task.

Question 3. Explain the application of artificial intelligence in early fault detection of

transmission lines?
Solution. Artificial intelligence (AI) has numerous applications in early fault detection in
transmission lines, contributing to the efficient and reliable operation of electrical power
systems. Here are several ways AI can be applied:

1. Predictive Maintenance:
- AI models, particularly machine learning algorithms, can analyse historical data from
transmission lines, such as voltage, current, temperature, and weather conditions.
- These models can predict when a component, such as an insulator or conductor, is
likely to fail by identifying patterns and anomalies in the data.
- Early detection of potential faults allows utilities to schedule maintenance or repairs,
minimising downtime and reducing the risk of catastrophic failures.

2. Real-time Monitoring:
- AI systems can continuously monitor transmission lines in real-time, using sensors and
IoT devices.
- Machine learning models process data from these sensors to detect any unusual
behaviour or deviations from normal operating conditions.
- When irregularities are detected, automated alerts can be generated, allowing operators
to take immediate action.

3. Fault Classification and Localization:

- AI algorithms can classify and localise faults in transmission lines by analysing data
from various sources, including sensors, GPS coordinates, and weather information.
- They can differentiate between types of faults such as short circuits, open circuits, and
insulation breakdowns.
 - Accurate localization helps maintenance crews pinpoint the exact location of the fault
for quick repairs.

4. Diagnostics and Trend Analysis:

- AI can analyse trends and patterns in historical data to identify the root causes of
recurrent issues or faults.
- By diagnosing the underlying problems, utilities can implement preventive measures to
reduce the occurrence of similar faults in the future.

5. Weather-Related Fault Prediction:

- AI can incorporate weather data into fault detection models since weather conditions
can affect transmission lines.
- Machine learning algorithms can predict potential faults or damage caused by severe
weather events, such as storms, lightning, or ice accumulation.

6. Image Processing and Drones:

- Drones equipped with cameras and AI-driven image processing can inspect
transmission lines from the air.
- AI can analyse images and identify defects, damaged components, or vegetation
encroachment, which might cause future faults.

7. Data Integration and Visualization:

- AI can integrate data from various sources, including sensors, weather forecasts,
maintenance records, and GIS (Geographic Information System) data.
- Visualisation tools powered by AI can provide operators with a clear overview of the
entire transmission network, highlighting potential areas of concern.

8. Continuous Learning:
- AI systems can adapt and learn from new data, improving their fault detection accuracy
over time.
- Ongoing machine learning ensures that the system remains effective in identifying
emerging fault patterns.

9. Reducing False Alarms:

- AI can help reduce false alarms by distinguishing between harmless fluctuations in data
and actual faults.
- Advanced algorithms filter out non-critical anomalies, ensuring that operators focus on
issues that genuinely require attention.
AI-powered early fault detection in transmission lines not only enhances the reliability of
electrical grids but also improves safety, minimises downtime, and ultimately leads to cost
savings for utilities and consumers.

Question 4. Write steps for designing a Fuzzy logic based voltage controller for a
synchronous generator?
Solution.
Designing a Fuzzy Logic-Based Voltage Controller for a Synchronous Generator involves
several steps to ensure that the controller effectively maintains the generator's voltage
within the desired range. Here are the steps for designing such a controller:

1. Problem Formulation:
- Define the problem and the control objectives. In this case, the objective is to maintain
the output voltage of the synchronous generator within specified limits under varying load
conditions and disturbances.

2. System Modelling:
- Develop a mathematical model of the synchronous generator. This model should
include the generator's dynamics and its response to changes in load and disturbances. The
generator model typically includes the field voltage, rotor angle, and electrical equations
governing the system.

3. Fuzzy Logic Controller Design:

- Decide on the input and output variables for the fuzzy logic controller. Common inputs
may include load deviation, rate of change of load, and error in generator voltage. The
output variable is typically the control signal for the generator excitation system.
- Determine the linguistic variables and their associated membership functions for each
input and output. These linguistic variables represent qualitative terms like "low,"
"medium," and "high" for input and output variables.
- Develop fuzzy rules that define how the input variables relate to the output variable.
Use if-then rules that express the controller's logic. For example: "If load deviation is high
and rate of change of load is increasing, then increase excitation voltage."

4. Fuzzy Inference System:

- Implement the fuzzy inference system, which interprets the fuzzy rules and generates
crisp control signals.
- Choose an inference method (e.g., Mamdani or Sugeno) and defuzzification method
(e.g., centroid, weighted average, or other) that suits the specific control problem.
5. Membership Functions and Rule Base:
- Fine-tune the membership functions and the rule base based on the generator's
dynamics and desired control performance.
- Consider incorporating expert knowledge to enhance controller performance.

6. Controller Integration:
- Integrate the fuzzy logic-based voltage controller into the generator's excitation system.
Ensure that the controller can accept input signals such as load deviation and generator
voltage error and provide control signals to the excitation system.

7. Simulation and Testing:

- Simulate the synchronous generator and its voltage controller using specialised
software tools. Validate the controller's performance by subjecting the system to various
load and disturbance scenarios.
- Make necessary adjustments to the fuzzy logic controller based on simulation results.

8. Hardware Implementation:
- Once the controller design is validated through simulation, implement it in the physical
synchronous generator system. This may involve programming a microcontroller or digital
signal processor to execute the controller algorithm.

9. Testing on Real System:

- Perform extensive testing on the real synchronous generator system to ensure that the
controller performs as expected and maintains voltage within the desired range under
various operating conditions.

10. Performance Optimization:

- Continuously monitor the performance of the fuzzy logic controller in practice and
make further refinements as necessary to optimise the control system.

11. Documentation:
- Document the design process, controller parameters, and test results for future
reference and troubleshooting.

12. Maintenance and Monitoring:

- Implement a maintenance and monitoring plan to ensure the continued reliability and
effectiveness of the fuzzy logic-based voltage controller in the synchronous generator.
Designing a fuzzy logic-based voltage controller for a synchronous generator requires a
combination of mathematical modelling, control theory, and practical implementation to
achieve the desired voltage regulation and system stability.

Question 5. Explain the structure of A neural network PI controller tuner?

Solution.
A Neural Network Proportional-Integral (PI) Controller Tuner is an intelligent control
system that combines the concepts of traditional PI control with neural networks to
optimise the control parameters automatically. This type of controller tuner aims to
improve control performance in real-time without manual tuning. Below, I'll explain the
structure of a Neural Network PI Controller Tuner:

1. Neural Network Model:

- The core of the tuner is a neural network model, which acts as a function approximator.
It takes control error, integral error, and other relevant inputs as its input features and
learns to predict optimal PI controller parameters as output. The neural network is
typically a feedforward network with hidden layers.

2. Controller Parameters:
- The PI controller has two main parameters: proportional gain (Kp) and integral gain
(Ki). These gains are the target outputs of the neural network. The network learns to
predict the optimal values of Kp and Ki for the given input conditions.

3. Data Collection:
- Data is collected during the normal operation of the control system. This data includes
the control error (the difference between the reference and the process variable), the
integral of the error (cumulative error over time), and any other relevant process variables.
The data also includes the current values of Kp and Ki. This data is used to train the neural
network.

4. Training the Neural Network:

- The collected data is used to train the neural network. The training process involves
adjusting the network's weights and biases to minimise the error between the predicted
controller parameters (Kp and Ki) and the actual controller parameters used in the system.

5. Real-time Control:
- During normal control operation, the neural network takes the current error and integral
of error as input and predicts the optimal Kp and Ki values. These predicted controller
parameters are used to calculate the control output signal, which is then applied to the
system.

6. Feedback and Adaptation:

- The control system continues to operate, and feedback is collected. If the control
performance deteriorates or drifts from the desired setpoint, the neural network uses the
new data to adapt and update its predictions for Kp and Ki.

7. Performance Evaluation:
- The controller tuner may include mechanisms to evaluate the control system's
performance, such as monitoring the setpoint tracking, response time, and steady-state
error. If the performance does not meet the desired criteria, the neural network continues to
adapt its predictions to optimise the control parameters.

8. Robustness and Stability:

- The neural network PI controller tuner is designed to maintain system stability and
ensure that the control system remains robust even under changing conditions.

9. Convergence and Learning

- The neural network may use various learning algorithms to ensure convergence to
optimal control parameters. This process ensures that the controller adapts and learns from
different operating conditions.

10. User Interaction:

- Some implementations of neural network PI controller tuners may allow for user
interaction or manual overrides to provide additional input or adjustments as needed.

The primary goal of a Neural Network PI Controller Tuner is to autonomously and

adaptively optimise

the PI controller parameters in real-time, ensuring efficient and stable control of the
system, even in the presence of disturbances or variations in the controlled process. It
leverages machine learning and neural networks to continually improve control
performance without the need for manual parameter tuning.