1
Introduction to CS412 - Fundamental concepts
Why study numerical analysis and scientific computing?
Lecture of:
In many instances we are faced with mathematical problems where the conventional 22 Jan 2013
solution method “on paper” does not perform as well, when implemented on the
computer. Take, for example, the quadratic equation
ax2 + bx + c = 0
In theory, we have a perfectly conclusive method for determining the roots of
this equation; simply use the quadratic formula
√
−b ± b2 − 4ac
x1,2 = (1)
2a
In fact, let us consider a very specific example:
0.0501x2 − 98.78x + 5.015 = 0
The actual roots of this equation, rounded to 10 digits of accuracy, are:
x1 = 1971.605916, x2 = 0.05077069387
We then proceed to evaluate the formula (1) on a computer. Let us assume that
the computer stores all intermediate results of this computation with 4 decimal
significant digits. Thus, we obtain the following answers:
x1 = 1972, x2 = 0.0998
Although the value x1 is reasonably accurate (it is, in fact, the correct answer
to 4 significant digits), the value x2 is off by almost 100% from the intended value,
even when rounded to 4 digits. The reason for this behavior is hiding in the in-
accurate (approximate) representation of the intermediate
√ steps of this calculation.
2
Particularly, the computer algorithm approximates b − 4ac ≈ 98.77, discarding
any subsequent digits due to its limited available precision. Substituting this value
into the quadratic formula yields:
98.78 ± 98.77
x1,2 =
0.1002
The numerator involves either a sum or a difference of two near-equal quantities.
When adding the 2 values, we compute the reasonably accurate √ x1 = 1972. When
subtracting, all the information that was discarded by truncating b2 − 4ac ≈ 98.77
2
is now dominating the computed value of the numerator. As a consequence we
obtain the compromised value x2 = 0.0998.
So, subtracting two near-equal quantities is risky ... Can we avoid doing so?
There is, in fact, an alternative formulation:
√ √
(−b ± b2 − 4ac)(−b ∓ b2 − 4ac)
x1,2 = √
2a(−b ∓ b2 − 4ac)
b2 − (b2 − 4ac) 4ac
= √ = √
2
2a(−b ∓ b − 4ac) 2a(−b ∓ b2 − 4ac)
2c
= √ (2)
−b ∓ b2 − 4ac
Equation (2) subtracts 2 quantities in the denominator where (1) adds them
and vice versa. Appying this approach we obtain:
x1 = 1003, x2 = 0.05077
Here the roles are reversed. x2 is quite accurate, but x1 is off by about 50% from
the intended value. One might suggest that we selectively use (1) or (2) to compute
x1 and x2 respectively, and avoid the subtraction-induced inaccuracies. Although
this may be realistic in this isolated example, in more complex problems that arise
in practice, performing such a case-by-case handling may prove impractical. In
many cases it may even be impossible to predict that a value lacks accuracy (not
knowing the exact value beforehand).
We do, however, have another alternative: Instead of resorting to case study, we
can use a fundamentally different algorithm. Consider the following methodology
(which is a special case of what we will later describe as Newton’s method):
• Start with an initial guess x0 of the solution
• Iterate
ax2k − c
xk+1 =
2axk + b
• After N iterations, take xN as the estimated solution
Let us try it ... start with a guess x0 = 1
x0 = 1, x1 = 0.050313 . . . , x2 = 0.0507706 . . .
After just 2 iterations, x2 is correct to more than 6 significant digits! Let us aim
for the other solution by setting x0 = 2000
x0 = 2000, x1 = 1972.003 . . . , x2 = 1971.60599 . . .
3
As is typically the case, there are trade-offs to consider: with this iterative
method, we require an “initial guess”, and there is little guidance offered on how to
pick a good one. Also, we need a few iterations before we can obtain an excellent
approximation. On the other hand, the method does not require any square roots,
thus being significantly cheaper in terms of computation cost per iteration.
The traits and trade-offs of such methods will be a point of focus for CS412,
and are central to the field we call numerical analysis and scientific computing.
How are numbers stored on the computer?
Corresponding
First, we shall review the concept of “scientific notation”, which will give us some textbook
helpful insights. For any decimal number x (we assume that x is a terminating chapter(s):
decimal number, with finite nonzero digits) we can write §1.2
x = a × 10b , where 1 ≤ |a| < 10
Exception: When x = 0, we simply set a = b = 0. For example:
x (decimal notation) x (scientific notation)
2012 2.012 × 103
412 4.12 × 102
3.14 3.14 × 100
0.000789 7.89 × 10−4
0.2091 2.091 × 10−1
Every decimal (or Base-10) number can be written
+k
X
ak ak-1 · · · a2 a1 a0 .a-1 a-2 a-3 · · · a-l = ai 10i
i=−l
For example
x a3 a2 a1 a0 . a-1 a-2 a-3
3.14 3 1 4
0.037 3 7
2012 2 0 1 2
Binary (Base-2) fractional numbers are written
+k
X
bk bk-1 · · · b2 b1 b0 .a-1 b-2 b-3 · · · b-l (2) = bi 2i
i=−l
where every digit bi is now only allowed to equal 0 or 1. For example
