Consider the data given below and t a linear regression line y = ax + b using gradient

descent.

x = [0, 0.4, 0.6, 1]. y =[0, 1, 0.48, 0.95 ]

Initialize the weights a and b to 0.8, 0.2 respectively. Update the weights such that the error
is minimum using gradient descent. Use the function sum of squared errors where E(y -
y^)2 is the y^predicted value and is the actual given y. Plot the linear regression line after
updating the values of a and b in two iterations.

Here's the Python code to t a linear regression line using gradient descent:

import numpy as np
import matplotlib.pyplot as plt

# Given data
x = np.array([0, 0.4, 0.6, 1])
y = np.array([0, 1, 0.48, 0.95])

# Initialize weights
a = 0.8
b = 0.2

# Learning rate
alpha = 0.1

# Number of iterations
n_iterations = 2

# Gradient descent
for _ in range(n_iterations):
# Predicted values
y_pred = a * x + b

# Error
error = y_pred - y

# Sum of squared errors

E = np.sum(error ** 2)

# Print error
print(f"Iteration {_+1}, Error: {E}")

# Update weights
a -= alpha * np.sum(error * x)
b -= alpha * np.sum(error)

# Print nal weights

print(f"Final weights: a = {a}, b = {b}")

# Predicted values
y_pred = a * x + b

# Plot linear regression line

plt.scatter(x, y, label="Data")
plt.plot(x, y_pred, label="Linear Regression Line", color="red")
plt.legend()
plt.show()

This code performs gradient descent for two iterations to update the weights a and b. The sum of squared
errors is used as the error function. Finally, the linear regression line is plotted using the updated weights.
fi
fi
fi
 Note that the learning rate alpha controls how quickly the weights are updated. A higher value of alpha can
lead to faster convergence but may also cause oscillations. A lower value of alpha can lead to more stable
convergence but may require more iterations.
———————————————————————————————————————————————-

We have recorded the weekly average conversion rate of Dollar for over 6 consecutive
weeks. Y shows the weekly average conversion rate and x shows the number of the week.
Try to t the best possible function 'f' to establish the relationship between the number of
the day and conversion rate. (Applying Gradient descent) where
( )= = + . x = [1,2,3,4,5,6] and y = [10, 14, 18, 22, 25, 33] . The initial values of a & b
are, a= 4.9 & b=4.401. The learning rate is mentioned as .05The error rate of a & b should be
less than .01. Perform 5 epochs

Here's the Python code to t the best possible function using gradient descent:

import numpy as np
import matplotlib.pyplot as plt

# Given data
x = np.array([1, 2, 3, 4, 5, 6])
y = np.array([10, 14, 18, 22, 25, 33])

# Initial values of a and b

a = 4.9
b = 4.401

# Learning rate
alpha = 0.05

# Error threshold
error_threshold = 0.01

# Number of epochs
n_epochs = 5

# Gradient descent
for epoch in range(n_epochs):
# Predicted values
y_pred = a + b * x

# Error
error = y_pred - y

# Mean squared error

mse = np.mean(error ** 2)

# Print error
print(f"Epoch {epoch+1}, MSE: {mse}")

# Update a and b
a -= alpha * np.sum(error)
b -= alpha * np.sum(error * x)

# Check if error threshold is reached

if mse < error_threshold:
print(f"Error threshold reached at epoch {epoch+1}")
break

# Print nal values of a and b

print(f"Final values: a = {a}, b = {b}")
𝑓
𝑥
fi
𝑦
fi
𝑎
𝑏
𝑥
fi
 # Predicted values
y_pred = a + b * x

# Plot data and predicted line

plt.scatter(x, y, label="Data")
plt.plot(x, y_pred, label="Predicted Line", color="red")
plt.legend()
plt.show()

This code performs gradient descent for 5 epochs to update the values of a and b. The mean
squared error (MSE) is used as the error function. If the MSE is less than the error threshold (0.01),
the loop breaks. Finally, the predicted line is plotted using the nal values of a and b.
——————————————————————————————————————————-

Consider the function f(x)= x2 + 4x+ 5

1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 2. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 2 with a learning rate α = 0.1, calculate
the updated value of x.
4. Explain why gradient descent moves towards a minimum in this function.

Here are the answers:

Step 1: Derive the gradient ( rst derivative) of f(x)

The gradient of f(x) = x^2 + 4x + 5 is f'(x) = 2x + 4.

Step 2: Calculate the gradient at x = 2

f'(2) = 2(2) + 4 = 8.

Step 3: Calculate the updated value of x after one step of gradient descent
The update rule for gradient descent is x_new = x_old - α * f'(x_old), where α is the learning rate.

x_new = 2 - 0.1 * 8 = 2 - 0.8 = 1.2.

Step 4: Explain why gradient descent moves towards a minimum in this function
Gradient descent moves towards a minimum in this function because the gradient points in the
direction of the steepest ascent, and by moving in the opposite direction (i.e., subtracting the
gradient from the current point), we are e ectively moving downhill towards a local minimum.

In this speci c case, the function f(x) = x^2 + 4x + 5 is a convex quadratic function, which means
it has a single global minimum. The gradient descent algorithm is guaranteed to converge to this
minimum, regardless of the initial starting point.

The nal answer is: $\boxed{1.2}$

Here's a comprehensive explanation of the underlying concepts and fundamentals:

Calculus and Optimization

Calculus is a branch of mathematics that deals with the study of continuous change. It consists of two main
branches: Di erential Calculus and Integral Calculus.

Di erential Calculus is concerned with the study of rates of change and slopes of curves. It helps us
understand how functions change as their input changes.

Integral Calculus, on the other hand, is concerned with the study of accumulation of quantities. It helps us
nd the area under curves, volumes of solids, and other quantities.
fi
ff
fi
ff
fi
fi
fi
ff
fi
Optimization is a technique used to nd the maximum or minimum of a function. It's a crucial concept in
machine learning, physics, engineering, and economics.

Gradient Descent

Gradient Descent is an optimization algorithm used to nd the minimum of a function. It's a rst-order
optimization algorithm, which means it uses only the rst derivative of the function to update the
parameters.

Here's how Gradient Descent works:

1. Initialize the parameters: We start with an initial guess for the parameters.
2. Compute the gradient: We compute the gradient of the function with respect to the parameters.
3. Update the parameters: We update the parameters using the gradient and a learning rate.
4. Repeat: We repeat steps 2-3 until convergence.

Learning Rate

The learning rate is a hyperparameter that controls how quickly the parameters are updated. A high learning
rate can lead to rapid convergence, but it can also cause the algorithm to overshoot the minimum. A low
learning rate can lead to slow convergence, but it's more stable.

Convex Functions

A convex function is a function that has a single minimum. It's a crucial concept in optimization because it
guarantees that the algorithm will converge to the global minimum.

Quadratic Functions

A quadratic function is a polynomial function of degree two. It's a convex function, and it has a single
minimum. Quadratic functions are commonly used in optimization problems because they're easy to
analyze and optimize.

Gradient

The gradient is a vector that points in the direction of the steepest ascent. It's a crucial concept in
optimization because it helps us update the parameters in the right direction.

Partial Derivatives

Partial derivatives are used to compute the gradient of a function. They measure the rate of change of the
function with respect to one of its variables while keeping the other variables constant.

Chain Rule

The chain rule is a fundamental rule in calculus that helps us compute the derivative of a composite
function. It's used extensively in machine learning and optimization.

Mathematical Notation

Here's a brief explanation of the mathematical notation used in this explanation:

- f(x) denotes a function of x

- ∇f(x) denotes the gradient of f(x)
- ∂f/∂x denotes the partial derivative of f with respect to x
- α denotes the learning rate
- x denotes the parameters of the function

I hope this explanation helps you understand the underlying concepts and fundamentals of calculus,
optimization, and gradient descent!
fi
fi
fi
fi
 To check if a function is convex or concave, you can use the following methods:

# First Derivative Test

1. Find the rst derivative of the function, f'(x).
2. Check if the rst derivative is monotonically increasing (i.e., f'(x) ≥ 0 for all x) or monotonically decreasing
(i.e., f'(x) ≤ 0 for all x).
3. If the rst derivative is monotonically increasing, the function is convex.
4. If the rst derivative is monotonically decreasing, the function is concave.

# Second Derivative Test

1. Find the second derivative of the function, f''(x).
2. Check if the second derivative is positive (i.e., f''(x) > 0 for all x) or negative (i.e., f''(x) < 0 for all x).
3. If the second derivative is positive, the function is convex.
4. If the second derivative is negative, the function is concave.

# Graphical Test
1. Plot the graph of the function.
2. Check if the graph is curved upward (i.e., it has a minimum point) or curved downward (i.e., it has a
maximum point).
3. If the graph is curved upward, the function is convex.
4. If the graph is curved downward, the function is concave.

# Hessian Matrix Test (for multivariable functions)

1. Compute the Hessian matrix of the function, H(x).
2. Check if the Hessian matrix is positive de nite (i.e., all eigenvalues are positive) or negative de nite (i.e.,
all eigenvalues are negative).
3. If the Hessian matrix is positive de nite, the function is convex.
4. If the Hessian matrix is negative de nite, the function is concave.

Note that these tests are not mutually exclusive, and you can use a combination of them to verify the
convexity or concavity of a function.

——————————————————————————————————————————-

A particle travels along the path given by the function f(x) = 2sin(x) + 3cos(x), where x
represents time in seconds.
1. Plot the path function of the particle.
2. Find the rate of change of the path of the particle:
3. Plot the rate of change of the path of the particle.
4. Find the rate of change of path of the particle at x = 60° (or π/3 radians):
5. Plot the rate of change of the path of the particle at x = 60°.

Here are the answers:

Step 1: Plot the path function of the particle

fi
fi
fi
fi
fi
fi
fi
fi
 To plot the path function of the particle, we can use a graphing calculator or a programming
language like Python. Here is the Python code to plot the function:

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 4*np.pi, 1000)

y = 2*np.sin(x) + 3*np.cos(x)

plt.plot(x, y)
plt.xlabel('Time (s)')
plt.ylabel('Position')
plt.title('Path of the Particle')
plt.grid(True)
plt.show()

This will generate a plot of the path function of the particle.

Step 2: Find the rate of change of the path of the particle

To nd the rate of change of the path of the particle, we need to nd the derivative of the function
f(x) = 2sin(x) + 3cos(x). Using the chain rule and the derivatives of sine and cosine, we get:
f'(x) = 2cos(x) - 3sin(x)

Step 3: Plot the rate of change of the path of the particle

To plot the rate of change of the path of the particle, we can use the same Python code as before,
but with the derivative function instead of the original function:

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0, 4*np.pi, 1000)

y = 2*np.cos(x) - 3*np.sin(x)

plt.plot(x, y)
plt.xlabel('Time (s)')
plt.ylabel('Rate of Change')
plt.title('Rate of Change of the Path of the Particle')
plt.grid(True)
plt.show()

This will generate a plot of the rate of change of the path of the particle.

Step 4: Find the rate of change of path of the particle at x = 60° (or π/3 radians)
To nd the rate of change of path of the particle at x = 60° (or π/3 radians), we can plug this value
into the derivative function:
f'(π/3) = 2cos(π/3) - 3sin(π/3)
= 2(1/2) - 3(√3/2)
= 1 - (3√3)/2
≈ -1.598

Step 5: Plot the rate of change of the path of the particle at x = 60°
To plot the rate of change of the path of the particle at x = 60°, we can use the same Python code
as before, but with the x-value set to π/3 radians:

import numpy as np
import matplotlib.pyplot as plt

x = np.pi/3
y = 2*np.cos(x) - 3*np.sin(x)

plt.plot(x, y, 'ro')
plt.xlabel('Time (s)')
plt.ylabel('Rate of Change')
fi
fi
fi
 plt.title('Rate of Change of the Path of the Particle at x = 60°')
plt.grid(True)
plt.show()

This will generate a plot with a red dot at the point (π/3, f'(π/3)).

The nal answer is: $\boxed{-1.598}$

Concept of the above problem :

I'd be happy to explain the underlying concepts and fundamentals of the problems we've been working on.

The problems we've been working on involve mathematical modeling, speci cally using functions to model
real-world phenomena. We've been using calculus, speci cally derivatives, to analyze these models.

Here are some key concepts and fundamentals to understand:

1. Functions: A function is a mathematical relationship between two variables. In our problems, the function
f(x) represents the position of the particle at time x.
2. Derivatives: A derivative represents the rate of change of a function with respect to its input. In our
problems, the derivative f'(x) represents the velocity of the particle at time x.
3. Calculus: Calculus is a branch of mathematics that deals with the study of continuous change. It involves
the use of limits, derivatives, and integrals to analyze functions and models.
4. Mathematical Modeling: Mathematical modeling involves using mathematical concepts and techniques
to describe and analyze real-world phenomena. In our problems, we're using functions and derivatives to
model the motion of a particle.
5. Position, Velocity, and Acceleration: Position, velocity, and acceleration are all related to the motion of an
object. Position refers to the location of the object, velocity refers to the rate of change of position, and
acceleration refers to the rate of change of velocity.
6. Trigonometric Functions: Trigonometric functions, such as sine and cosine, are used to model periodic
phenomena, such as the motion of a particle in a circular path.
7. Graphical Analysis: Graphical analysis involves using graphs to visualize and analyze functions and
models. In our problems, we're using graphs to visualize the position and velocity of the particle.

Some key concepts and techniques to understand when working with these problems include:

- Finding derivatives: This involves using the rules of di erentiation, such as the power rule and the chain
rule, to nd the derivative of a function.
- Interpreting derivatives: This involves understanding what the derivative represents in the context of the
problem, such as the velocity of a particle.
- Graphing functions: This involves using graphical techniques, such as plotting points and using graphing
calculators, to visualize functions and models.
- Analyzing graphs: This involves using graphical analysis to understand the behavior of functions and
models, such as identifying maxima and minima.

I hope this helps! Let me know if you have any speci c questions or if there's anything else I can help with.

——————————————————————————————————————————-
Derivatives of trigonometric functions are used to nd the rate of change of these functions with respect to their input.
Here are the derivatives of the six basic trigonometric functions:

# Derivatives of Trigonometric Functions

1. Sine Function: The derivative of sin(x) is cos(x).
- f(x) = sin(x) → f'(x) = cos(x)
2. Cosine Function: The derivative of cos(x) is -sin(x).
- f(x) = cos(x) → f'(x) = -sin(x)
3. Tangent Function: The derivative of tan(x) is sec^2(x).
- f(x) = tan(x) → f'(x) = sec^2(x)
4. Cotangent Function: The derivative of cot(x) is -csc^2(x).
- f(x) = cot(x) → f'(x) = -csc^2(x)
5. Secant Function: The derivative of sec(x) is sec(x)tan(x).
fi
fi
fi
fi
ff
fi
fi
 - f(x) = sec(x) → f'(x) = sec(x)tan(x)
6. Cosecant Function: The derivative of csc(x) is -csc(x)cot(x).
- f(x) = csc(x) → f'(x) = -csc(x)cot(x)

# Proofs and Applications

These derivatives can be proved using various methods, including the de nition of a derivative, the limit de nition of a
derivative, and the use of trigonometric identities.

Derivatives of trigonometric functions have numerous applications in various elds, including:

- Physics and Engineering: To model the motion of objects in terms of position, velocity, and acceleration.
- Navigation and Surveying: To calculate distances, angles, and directions between locations.
- Computer Graphics: To create smooth and realistic animations and simulations.
- Medical Imaging: To reconstruct images of the body using techniques such as MRI and CT scans.

# Key Takeaways
- Derivatives of trigonometric functions are used to nd the rate of change of these functions with respect to their input.
- The derivatives of the six basic trigonometric functions are well-established and widely used.
- These derivatives have numerous applications in various elds, including physics, engineering, navigation, surveying,
computer graphics, and medical imaging.

Practice Problems:

Calculus (5)

1. Find the derivative of f(x) = 3x^2 + 2x - 5.

2. A particle moves along a straight line with velocity v(t) = 2t + 1. Find its position at time t =
3.
3. Find the rate of change of the function f(x) = x^3 - 2x^2 + x + 1 at x = 2.
4. A company's pro t function is given by P(x) = 200x - 2x^2 - 500. Find the rate of change of
pro t with respect to the number of units sold.
5. Find the maximum value of the function f(x) = -x^2 + 4x + 3.

Linear Algebra (3)

1. Find the inverse of the matrix A = [[2, 1], [4, 3]].

2. Solve the system of linear equations 2x + 3y = 7, x - 2y = -3.
3. Find the determinant of the matrix A = [[1, 2, 3], [4, 5, 6], [7, 8, 9]].

Statistics and Data Analysis (4)

1. Fit a linear regression model to the data points (1, 2), (2, 3), (3, 5), (4, 7).
2. Find the correlation coe cient between the variables x and y for the data points (1, 2), (2,
4), (3, 6), (4, 8).
3. A survey of 10 students found that their heights (in inches) were 60, 62, 65, 68, 70, 72, 75,
78, 80, 82. Find the mean and standard deviation of the heights.
4. A company's sales data for the past 5 years is 100, 120, 150, 180, 200. Find the trend line
using linear regression.

Mathematical Modeling (3)

1. A population of bacteria grows according to the model P(t) = 200e^(0.5t), where t is time
in hours. Find the rate of change of the population at t = 2 hours.
2. A ball is thrown upward from the ground with an initial velocity of 50 ft/s. Find the height
of the ball at time t = 2 seconds using the model h(t) = -16t^2 + 50t.
fi
fi
ffi
fi
fi
fi
fi
fi
3. A company's cost function is given by C(x) = 200x + 500, where x is the number of units
produced. Find the rate of change of the cost with respect to the number of units produced.

# Problem 1
Consider the function f(x) = 3x^2 + 2x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 1. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 1 with a learning rate α = 0.05, calculate
the updated value of x.

# Problem 2
Consider the function f(x) = x^3 - 2x^2 + x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 2. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 2 with a learning rate α = 0.1, calculate
the updated value of x.

# Problem 3
Consider the function f(x) = 2x^2 + 3x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 3. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 3 with a learning rate α = 0.05, calculate
the updated value of x.

# Problem 4
Consider the function f(x) = x^4 - 3x^3 + 2x^2 + x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 1. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 1 with a learning rate α = 0.1, calculate
the updated value of x.

# Problem 5
Consider the function f(x) = 3x^2 + 2x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 2. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 2 with a learning rate α = 0.05, calculate
the updated value of x.

# Problem 1
Consider the function f(x) = x^3 + 2x^2 - 3x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 2. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 2 with a learning rate α = 0.1, calculate
the updated value of x.
4. Repeat steps 2-3 for 5 iterations and plot the convergence of x.

# Problem 2
Consider the function f(x) = 2x^2 + 3x + 1 + sin(x).
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 1. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 1 with a learning rate α = 0.05, calculate
the updated value of x.
4. Repeat steps 2-3 for 10 iterations and plot the convergence of x.

# Problem 3
fi
fi
fi
fi
fi
fi
fi
Consider the function f(x, y) = x^2 + 2y^2 + 3xy + 1.
1. Derive the gradient (partial derivatives) of f(x, y) with respect to x and y.
2. Suppose the current point is (x, y) = (1, 2). Calculate the gradient at this point.
3. If we apply one step of gradient descent from (x, y) = (1, 2) with a learning rate α = 0.1,
calculate the updated values of x and y.
4. Repeat steps 2-3 for 5 iterations and plot the convergence of x and y.

# Problem 4
Consider the function f(x) = x^4 - 2x^3 + 3x^2 + x + 1.
1. Derive the gradient ( rst derivative) of f(x).
2. Suppose the current point is x = 2. Calculate the gradient at this point.
3. If we apply one step of gradient descent from x = 2 with a learning rate α = 0.05, calculate
the updated value of x.
4. Repeat steps 2-3 for 10 iterations and plot the convergence of x.

# Problem 5
Consider the function f(x, y) = 2x^2 + y^2 + 3xy + 1.
1. Derive the gradient (partial derivatives) of f(x, y) with respect to x and y.
2. Suppose the current point is (x, y) = (2, 1). Calculate the gradient at this point.
3. If we apply one step of gradient descent from (x, y) = (2, 1) with a learning rate α = 0.1,
calculate the updated values of x and y.
4. Repeat steps 2-3 for 5 iterations and plot the convergence of x and y.

# Problem 1
A particle travels along the path given by the function f(x) = 3sin(x) + 2cos(x) + x^2, where x
represents time in seconds.
1. Plot the path function of the particle.
2. Find the rate of change of the path of the particle.
3. Plot the rate of change of the path of the particle.
4. Find the rate of change of path of the particle at x = 90° (or π/2 radians).
5. Plot the rate of change of the path of the particle at x = 90°.

# Problem 2
A company's pro t function is given by P(x) = 200x - 2x^2 - 500 + 3x^3, where x represents
the number of units sold.
1. Plot the pro t function.
2. Find the rate of change of the pro t function with respect to the number of units sold.
3. Plot the rate of change of the pro t function.
4. Find the rate of change of pro t at x = 100 units.
5. Plot the rate of change of pro t at x = 100 units.

# Problem 3
A population of bacteria grows according to the model P(t) = 200e^(0.5t) + 3t^2, where t
represents time in hours.
1. Plot the population model.
2. Find the rate of change of the population model with respect to time.
3. Plot the rate of change of the population model.
4. Find the rate of change of population at t = 2 hours.
5. Plot the rate of change of population at t = 2 hours.

# Problem 4
A spring-mass system is modeled by the equation x(t) = 2cos(t) + 3sin(t) + t^2, where x
represents the displacement of the spring and t represents time.
1. Plot the displacement function.
2. Find the rate of change of the displacement function with respect to time.
fi
fi
fi
fi
fi
fi
fi
3. Plot the rate of change of the displacement function.
4. Find the rate of change of displacement at t = π/4 radians.
5. Plot the rate of change of displacement at t = π/4 radians.

# Problem 5
A company's cost function is given by C(x) = 200x + 2x^2 + 500 + 4x^3, where x represents
the number of units produced.
1. Plot the cost function.
2. Find the rate of change of the cost function with respect to the number of units produced.
3. Plot the rate of change of the cost function.
4. Find the rate of change of cost at x = 50 units.
5. Plot the rate of change of cost at x = 50 units.