CVE325 Computational Methods

Part Five
Curve Fitting
Chapters 17, 18
Dr. Jamal A. Abdalla

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Objectives and Outcomes
Objectives
• Acquire sufficient knowledge to solve problems involving curve fitting.
• Master several techniques used in curve fitting such as linear, polynomial and non-linear regression
and polynomial interpolation
Outcomes
1. Understand the fundamental difference between regression and interpolation
2. Understand the derivation of linear least-squares regression and be able to assess the reliability of
the fit using graphical and quantitative assessments
3. Know how to linearize data by transformation
4. Understand situations where polynomial and nonlinear regression are appropriate
5. Be able to recognize general linear models, understand the general matrix formulation of linear
least squares
6. Understand that there is one polynomial of degree n or less that passes exactly through n + 1 points
7. Know how to derive the first-order Newton's interpolating polynomial
8. Understand the analogy between Newton's polynomial and the Taylor series expansion
9. Recognize that the Newton and Lagrange equations are merely different formulations of the same
interpolating polynomial and understand their respective advantages and disadvantages
10. Realize that more accurate results are generally obtained if data used for interpolation is centered
around and close to the unknown point
11. Realize that data points do not have to be equally spaced nor in any particular order for either the
Newton or Lagrange polynomials
12. Know why equispaced interpolation formulas have utility
13. Recognize the liabilities and risks associated with extrapolation
14. Understand why spline functions have utility for data with local areas of abrupt change
15. Recognize how the Fourier series is used to fit data with periodic functions
16. Understand the difference between the frequency and time domains

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Statistical Background

Mean y
 y i

n
St
Standard deviation
sy 
n 1
St
Variance s 2
y
n 1
Sum of the square st   ( yi  y ) 2

of the residuals

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Statistical Background

Variance s y2 
 i  i /n
y 2
 ( y ) 2

n 1

sy
Coefficient of c.v.  100%
variation (c.v.) y

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 5.1
Simple Statistic of a Sample

Compute the mean, variance, standard deviation, and coefficient

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 The Normal Distribution

A histogram used to depict the distribution of data. As the number of data points
increases, the histogram could approach the smooth, bellshaped curve called the
normal distribution
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4 Edition, 2003, McGraw-Hill.
th
 Estimation of Confidence Interval

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Estimation of Confidence Interval
A two-sided interval can be described by the statement

The standard normal estimate

α is the probability
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Comparison of Normal
Distribution with t Distribution

n= 3 and n = 6
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Curve Fitting
Three attempts to fit a “best” curve
through five data points:
(a)Least-Squares regression
(b)Linear Interpolation
(c)Curvilinear interpolation

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Curve Fitting and Engineering
Practice
Curve fitting in engineering practice is used
to:
• Determine intermediate values from
tabulated values
• For fitting experimental data: Trend
analysis and hypothesis testing
• For integration and numerical solution of
differential equations
• For approximating complicated functions

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Chapter 17

Least-Square Regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Reading Assignments

Sections: 17.1,17.2, 17.4, 17.5

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Linear Regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Criteria for Best Fit

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Least Square Fit for Straight Line

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Examples of some Criteria for “best
fit” that are inadequate for
regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 17.1

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Quantification of Error in Linear
Regression

The standard error of estimate is given by:

Sr
sy / x 
n2
It is a measure of deviation from the best fit using least square regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Residual “error” in Linear
Regression

It is the vertical distance between the data point and the straight
line
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Spread of data

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 The coefficient of determination
St  S r
r 
2

St
The correlation coefficient

n xi yi  ( xi )( yi )
r
n x  ( xi )
2
i
2
n  y  (  yi ) 2
i
2

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 17.2

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Linearization of Nonlinear
Relationships
Linear regression provides a powerful
technique for fitting a best line to data, if the
relationship between the dependent and
the independent variable is linear.
In many cases the relationship between the
dependent and the independent variable
may not be linear.
Transformation could be used to express
the nonlinear relations in a linear form and
apply linear regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Nonlinear Relationships
Exponential Equation

y  a1e b1 x

Power Equation

y  a2 x b2

Saturation-Growth-Rate Equation
x
y  a3
b3  x
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Linearization of Nonlinear
Relationships

ln y  ln a1  b1 x ln e
log y  log a2  b2 log x
1 1 b3 1
 
y a3 a3 x
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 17.4

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Polynomial Regression

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 17.5

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Chapter 18

Interpolation

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Reading Assignments

Sections: 18.1, 18.2

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Polynomial interpolation consist of determining the unique nth-order
ploynomial that fits n+1 data points. The general form of nth-order
polynomial is as:

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Newton’s Divided-Difference
Interpolating Polynomials
From similarity of triangles

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
This is a linear interpolation formula (first order interpolating
polynomial) and the following is known as finite-divided
difference approximation of the derivative.

The smaller the interval between the data points, the better the approximation

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 18.1

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Quadratic Interpolation
If three data points are available, a second order polynomial can be used

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 This can be written using divided-difference approximation
With x = x0

With x = x1

With x = x2

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 18.2

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 General Form of Newton’s
Interpolating Polynomials
The nth-order polynomial to fit n+1 data points is given by:

Data points can be used to evaluate the coefficients as follows:

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 f n ( x)  f ( x0 )  ( x  x0 ) f [ x1 , x0 ]  ( x  x0 )( x  x1 ) f [ x2 , x1 , x0 ] 
.......... .  ( x  x0 )( x  x1 )........( x  xn 1 ) f [ xn , xn 1 ,......, x0 ]

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 18.3

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Errors in Newton’s Interpolating
Polynomials
For an nth order interpolating polynomial the error can be given by

f ( n 1) ( )
Rn  ( x  x 0 )( x  x1 )( x  x 2 )..........( x  x n 1 )( x  x n )
( n  1)!

Rn  f [ x n 1, x n , x n 1 ,... x 0 ]( x  x 0 )( x  x1 )( x  x 2 )...( x  x n 1 )( x  x n )

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Lagrange Interpolating Polynomials
Lagrange Interpolating polynomials is the reformulation of
Newton’s polynomials that avoids the computation of divided
differences
n
f n ( x)   Li ( x) f ( xi )
i 0

where
n x  xj
Li ( x)  
j 0 xi  x j
j 1

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Example 18.6

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
 Comparison

Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.
Prepared by: Jamal A. Abdalla Ref. Chapra, S. C. and Canale, R. P. , Numerical methods for Engineers , 4th Edition, 2003, McGraw-Hill.