Notes on Ordinary Differential and Difference Equations

by

John J. Seater
June, 2002

ORDINARY DIFFERENTIAL EQUATIONS

Ex.

Def Any function that is free of derivatives and that satisfies identically a differential equation is a solution of the differential equation. Ex. has

(an implicit function of y) as a solution. Def A differential equation that involves derivatives with respect to a single independent variable is an ordinary differential equation (ODE). One that involves derivatives with respect to more than one independent variable is a partial differential equation (PDE). Def The order of a differential equation is the order of the highest-order derivative present. Ex. is of order 2. Ex.

is of order 4. Def If a differential equation can be rationalized and cleared of fractions with regard to all derivatives present, the exponent of the highest-order derivative is called the degree of the Page 2 of 87

equation. Note: Not every differential equation has a degree. Ex.

is of degree 2. Ex. has no degree.

GENERAL REMARKS ON SOLUTIONS Three important questions arise in attempting to solve a differential equation: (1) Does a solution exist? (2) If a solution exists, is it unique? (3) If solutions exist, how do we find them? Although most effort concerns (3), questions (1) and (2) are logically prior and must be answered first. Thm (Existence) A differential equation y' = f(x,y) has at least one solution passing through any given point (x0, y0) if f is continuous. Thm (Uniqueness) A sufficient condition for the solution to the differential equation y' = f(x,y), passing through a given point (x0,y0) to be unique is that Mf/My be continuous. These theorems apply only to first-order equations. However, analagous theorems apply to higher-order equations: Thm (Existence and Uniqueness) The n-th order equation

has a unique solution passing through the point

Page 3 of 87

if f, f2, f3, ..., fn+1 all are continuous. When the last theorem applies, an n-th order differential equation has a solution with n arbitrary constants, called the general solution, and this solution is unique. Ex. has the general solution

where the three arbitrary constants are A, B, and C. Special cases of the general solution are called particular solutions; these are determined by the arbitrary initial conditions (x0, 01, ..., 0n). Ex. The equation

has the general solution

However, it also has the solution

which cannot be derived from the general solution and so is not a particular solution. It is called a singular solution. To understand this last example, note that the original differential equation is of degree 2, that is, it is a quadratic equation in the variable y' and therefore can be solved to give

Consider the first of these: Page 4 of 87

The partial derivative with respect to y is (x2 + 4y)-1/2, which is discontinuous at y = -x2/4. It therefore does not satisfy the uniqueness conditions. Note that only the region above the parabola y = -x2/4 defines a real equation. For initial points within this region, solutions are unique. [For example, the point (1,2) implies from the general solution y = cx + c2 that c2 + c - 2 = 0, so that c = 1or -2, i.e., that y = x + 1 or y = 4 - 2x. Of these, only the first satisfies the equation

in agreement with the existence theorem. The second satisfies the other equation

Note that y = x + 1 and y = 4 - 2x, and in fact y = cx + x2, are tantents to the parabola y = -x /4, which is the singular solution. The parabola is thus the envbelope of all these tangents, which in turn collect all the possible initial conditions.
2

Singular solutions, when they occur, often appear on the borders of existence regions.

FIRST-ORDER ODEs I. Exactness A first-order, first-degree ODE has the form

which may be rewritten as

If this equation satisfies the existence and uniqueness theorems, then the general solution will have Page 5 of 87

one arbitrary constant c, and the solution can be written

Take the differential of both sides

which implies

so that

or

Calling each of these ratios :, which in general may be a function of x and y, we obtain

and

Substituting these back into the equation for dU gives

an exact differential equation. Such an equation can be written

Page 6 of 87

The function : enables us to go from the original differential equation to an exact differential and then, by integration, to a function that solves the differential equation. Thus : is called an integrating factor. If the differential equation

already is exact, then by definition there is a function U(x,y) such that

But

which implies that

For a sufficiently smooth function U,

Thus the relation

Page 7 of 87

is a necessary condition for exactness. It turns out also to be sufficient. Thus we have Thm A necessary and sufficient condition for exactness of the differential equation

is

Ex.

The differential equation

is exact because

Thus, by the sufficiency part of the theorem, there exists a function U such that

or equivalently such that

We use these last two relationships to determine U. By reversing the differentiation (i.e., by integrating), we can recover U. Therefore

where f(y) is the constant of integration. So Page 8 of 87

To find f(y), use the second relationship above:

Thus

II. Solving First-Order Differential Equations A. Separation of variables. Ex. To solve

we write the equation in the form

Then integrate

which gives

Page 9 of 87

or

which is an implicit function for y. This method in fact is a shortcut way of finding an integrating factor. To see this, re-write the original differential equation in the form Mdx + Ndy = 0:

We see that

so the equation is not exact. However, the function (x3y2)-1 is an integrating factor; multiplying the differential equation by it gives

for which

so that exactness holds. B. Integrating factors involving one variable. Ex. The equation

Page 10 of 87

is not exact and not separable. However, the function y-2 is an integrating factor:

or

or

which is

so that we have, upon integrating,

How do we know y-2 is an integrating factor? (Ignore the problem of how we found it in the first place.) To see this, consider the case where Mdx + Ndy = 0 is not exact or separable. Multiply by the integrating factor : (as yet unknown) to obtain the exact equation :Mdx + :Ndy = 0. Then by definition

Consider two cases. (i) : is a function of x alone. In this case,

Page 11 of 87

or

where the arbitrary constant of integration has been set equal to zero. Thus we have the following: Thm If

where f(x) is a function of x alone, then (ii) : is a function of y alone.

is an integrating factor.

Similar reasoning leads to the following: Thm If

where g(y) is a function of y alone, then

is an integrating factor.

C. Important special case of the previous method - the linear first-order equation. An equation that can be written in the form

is called a linear equation of first order. (Note that it is only required to be linear in y, not x.) This equation has eIP(x)dx as an integrating factor, so that we may write

Page 12 of 87

which is equivalent to

which we may integrate to obtain

which is the general solution. D. Other methods. Often, equations are not immediately solvable by multiplying by an integrating factor. Numerous specialized methods exist for such equations, usually involving some convenient transformation of variables. Ex. The equation

(unfortunately called a homogeneous differential equation) can be solved by the transformation

Differentiate both sides of y = vx with respect to x:

so that

Page 13 of 87

or

which has variables separated and is easily solved.

III. Line Integrals and Differential Equations A. Basics. A line integral generalizes the concept of the integral to include integrals of vector-valued functions on Rn. For simplicity, we will restrict attention n=2. Let

> / (x, y) , R2; > = xi + yj where i, j are basis vectors N:R2 6 R2 ; N(>) = [f(x, y), g(x, y)] where f, g:R2 6 R1 D:R1 6 R2, D(t) = >, with and D(t) = [r(t), s(t)] where r,s:R1 1 6R C = graph of D(t) for t , [a, b], called a directed arc
Def The line integral of N along C, the graph of > = D(t), is the number

Ex.

Let N(>) = (x2, y2) and C be the graph of D(t) = (5cos t, 5sin t) for t , [-1/2 B, 1/2B]. Then

Page 14 of 87

Many times we integrate around a closed path, which is one that begins and ends at the same point. We must distinguish between the two directions around the path; integrating in one direction yields the opposite sign as integrating in the other direction. By convention, we integrate in the counterclockwise direction. The notation

indicates integrating around the closed path P in the counterclockwise direction. (If the direction of integration ever is ambiguous, the notation is altered by putting an arrow on the circle to show the direction of integration.) Thm (Fundamental Theorem of Calculus for Line Integrals.) Suppose N is the gradient of a scalar-valued function h:R2 6 R1. Then

B. Green's theorem; functions defined by line integrals. Page 15 of 87

Let P be the path shown in the graph, and let R be the region bounded by P. Then

To evaluate these last two integrals, we decompose P into two directed arcs. For the first integral, we consider P as the union of the arc ACB and BDA; for the second, we consider P as the union of CBD and DAC. Thus we have

Let ACB be the graph of y = r(x) and ADB be the graph of y = s(x). Then

so that

By the fundamental theorem of calculus, for each x , [a, b] we have

Page 16 of 87

so that

Similarly, if the arcs CBD and CAD are the graphs of x = q(y) and x = p(y) for y , [c, d], then

Then by substituting into the original integral, we obtain Green's Theorem (also called the fundamental theorem of calculus for double integrals): Thm

Green's theorem is important because it allows sensible definitions of functions in terms of Page 17 of 87

line integrals. To define a function F:R2 6 R1 with a line integral, choose a point " = (a, b) and define F(>), for any > = (x, y),

In general, this definition is inadequate in that the value of F will depend on the path chosen from " to >. To have a useful definition, we onlyn use line integrals to define functions when the integral is independent of path. We use Green's theorem to identify path-independent integrals. Let P1 and P2 be two directed paths in R that connect the points (a, b) and (x, y). We want F to be path-independent, so require:

where P = P1 c P2. Suppose the region D bounded by P is simply connected (essentially, has no holes). Then by Green's theorem

where (u, v) / (. The double integral is zero if, for each point of D, g1 = f2. Thus we have: Thm If R is simply connected and f2 = g1 * , R, then the equation

defines a function F:R2 6 R1. We can compute the partial derivatives for this function F as follows:

Page 18 of 87

Thm For F defined by

where f and g are continuous and satisfy f2 = g1, we have

Proof: Write

The first of these integrals is independent of x so that

We may use any path to evaluate this integral; it is convenient to use the horizontal segment joining (c, y) and (x, y), along which v is constant at the value y: u = r(t) = t, t,[c, x] v = s(t) = y Then we have

Page 19 of 87

Consequently,

The proof that DyF = g(x,y) is similar. QED The Fundamental Theorem of Calculus for Line Integrals tells us that the line integral of a gradient is independent of path. The converse is also true: Thm If line integrals of a continuous function N are independent of path in some region R, then N is the gradient of a scalar-valued function. Proof: We can take as this scalar-valued function the function F defined by

for which

QED C. Application to Differential Equations Let f(x,y) and g(x,y) satisfy f2 = g1, and let Page 20 of 87

For any c in the range of F, the set of points S satisfying F(x,y) = c constitutes a level curve (contour) of F. Suppose r is a differentiable function whose graph is a subset of this level curve, i.e.,

Then

Therefore, in the interval I, the function y = r(x) satisfies the exact differential equation

which in our earlier notation can be written

or

So we conclude that (1) level curves of functions defined by line integrals provide solutions to exact differential equations; (2) the condition for exactness of the differential equation is the same as that for path independence of the corresponding line integral.

Page 21 of 87

Ex.

The differential equation

is exact because

Therefore, we can set

and use the path shown to find that

Level curves of this function are graphs of the form

so the function

satisfies the differential equation. The value of c is determined by initial conditions. For example, if the initial condition is that y = 3 when x = 1 [that is, (x0, y0) = (1,3)], then c = 30.

LINEAR DIFFERENTIAL EQUATIONS I. General Def A linear differential equation of order n has the form

Page 22 of 87

where ai(x) and F(x) do not depend on y. It is often convenient to express a linear differential equation in terms of differential operators:

or

or

Clearly,

Any operator satisfying these properties is called a linear operator. It is easy to show that N(D) is a linear operator. We want to discover how to find the general solution to a linear differential equation. To do so, we will examine the homogeneous (or complementary) equation

Its solution provides a crucial step in finding the solution to the original equation. Thm If y = u(x) is any solution of the given equation and y = v(x) is any solution of the homogeneous equation, then y = u(x) + v(x) is a solution of the given equation. Proof: We have

Page 23 of 87

Adding, we have

Def The general solution of the homogeneous equation is called the homogeneous (or complementary) solution. A solution of the given equation in which all constants have specified values is called a particular solution. Thm The general solution of N(D)y = F(x) may be obtained by finding a particular solution yp and adding it to the homogeneous solution yc. Ex. The homogeneous equation for

is

It may be verified that the homogeneous solution is

It also may be verified that the particular solution is

Therefore, the general solution is

Obviously, two important questions must be answered: (1) How do we find the homogeneous solution? (2) How do we find the particular solution? Page 24 of 87

II. Existence and Uniqueness Thm Let ai(x) and F(x) be continuous on the interval [a,b] and suppose a0(x) 0 x , [a,b]. Then there exists a unique solution y(x) satisfying the differential equation and also the initial conditions

III. Obtaining a Homogeneous Solution Suppose we want to solve

which is the same as

This is easily solved by the methods discussed earlier:

This solution could also be found by lucky guessing. Assume a solution of the form y = emx with m to be determined. For this form to be a solution, it must satisfy (m-2) emx = 0, i.e., m=2 because emx is never zero. This solution can be generalized to ce2x. Would the same guessing method work on the equation

Letting y = emx, we obtain

Y Y Y
ex and e2x are solutions

Page 25 of 87

What is the general solution? We know a general solution must have two arbitrary constants, so let us propose

This function is easily shown to satisfy the original equation and therefore must be the general solution. Thm If y1, y2, ..., yp are solutions to N(D)y = 0, then c1y1 + c2y2 + ... + cpyp also is a solution. The equation used above for determination of m is called the characteristic (or auxiliary) equation and has the form N(m) = 0. Note that this equation applies only to differential equations with constant coefficients, that is, with ai(x) = ai. For the most part, we will be concerned only with equations that have constant coefficients. Thm If the differential equation N(D) = 0 has constant coefficients and if the roots of the characteristic equation are distinct, then the general solution is

where mi are the roots of the characteristic equation. Thm If the characteristic equation has repeated roots, then the general solution is

where d = number of distinct roots, ni = multiplicity of the ith root with n1 + ... + nd = n = the order of the equation. Ex. The differential equation

has one root equal to 3 with multiplicity 2. Therefore the general solution is

The foregoing theorem is a consequence of the following:

Page 26 of 87

Thm If y = y1 is one solution of the nth-order differential equation N(D) = F(x) [with variable or constant coefficients], then the substitution y = y1v will transform the given equation into one of the (n-1)st order in dv/dx. Ex. One solution to

is e3x. According to the theorem, let y = ve3x. Then

so that we obtain

which implies

or v = c1 + c2x. Consequently,

IV. Linear Independence and Wronskians Suppose we have the differential equation

and somehow arrive at the three solutions

Is the general solution given by

Page 27 of 87

The answer is no because we can rearrange terms to get

which has only two arbitrary constants. A basic principle in determining general solutions is the following Thm If y1, y2, ..., yn are n linearly independent solutions of the nth order linear differential equation

then all solutions have the form

which is the general solution. The problem in the foregoing example is that the three solutions are not linearly independent. A test for linear independence is given by the following: Thm The set of functions y1, y2, ..., yn is linearly independent if and only if the Wronskian

is not identically zero. Ex. The functions x2 and x3 have the Wronskian

Page 28 of 87

which is not identically zero. Note that W = 0 at x = 0, but that is different from W being identically zero.

V. Behavior of Solutions to Second-Order Equations Suppose the differential equation

has constant coefficients. Then the behavior of the solution depends on the values of the roots of the characteristic equation. A few examples will be mentioned here; detailed discussion is deferred until systems of equations have been discussed. (A) Distinct negative roots The solution has the form

with m1, m2 < 0. Therefore, y 6 0 as x 6 4. In this case, y is asymptotically stable. (B) Distinct positive roots y 6 4 as x 6 4 so y is unstable. (C) Distinct imaginary roots For the equation

Page 29 of 87

we have the characteristic equation

which has the conjugate pair of imaginary roots i. The general solution is

What does this solution mean? The functions eix and e-ix are defined to be

which are derived from the series expansion of eu

with u replaced by ix or -ix. Therefore

Because y must be real, both A and B must be real. Therefore

Page 30 of 87

Y
In this particular example, the solution

produces endless non-damping and non-explosive cycles as x 6 4. In general, imaginary roots yield cyclic solutions. The solutions may be damped, endless, or explosive, depending on the values of the real parts of the roots. Ex. The equation

has roots m = -1 2i and the general solution

which leads to damped cycles as x 6 4. VI. Obtaining a Particular Solution Our next step is to discover how to find a particular solution to the equation N(D)y = F(x). We briefly examine two methods. A. Method of Undetermined Coefficients Page 31 of 87

This method applies when F(x) consists of a polynomial in x, terms of the form sin(px), cos(px), and epx where p = constant, or combinations of sums and products of these. When products of these terms are not present, the rules for using the method are: (1) Write the homogeneous solution yH. (2) Assume a particular solution with the same kinds of terms constituting F(x): (a) For a polynomial of degree n, assume a polynomial of degree n. (b) For terms sin(px), cos(px), or sums and differences of such terms, assume a sin(px) + b cos(px). (c) For terms epx, assume aepx. (3) If any of the assumed terms already appear in the homogeneous solution, they must be multiplied by a power of x sufficiently high (but no higher) that they do not occur in the homogeneous solution. (4) Evaluate the coefficients of the assumed particular solution, thus obtaining yP. (5) Add yH and yP to obtain the general solution. When products of the basic terms are present, one first must differentiate the right side indefinitely, keeping track of all essentially different terms that arise. If these are finite in number, then the method of undetermined coefficients is applicable. The assumed particular solution is formed by multiplying each of the terms that appeared in the differentiation by an undetermined constant and adding the results. Ex. (step 2.b) The homogeneous solution of

is

To find a particular solution, we ask what functions differentiated once or twice yield sin3x or constant multiples of it. The answer is sin3x or cos3x. Therefore, we try

Page 32 of 87

Substituting in the given equation gives

The second equality will be satisfied if and only if -5a - 12b = 6 12a - 5b = 0

Y Y

a = -30/169, b = -72/169

Ex. (need to differentiate RHS) Suppose we have

The homogeneous solution is

To find a particular solution, begin differentiating the functions 1st round: yields terms of the form 2nd round: yields , , and and

Continuing in this manner, we find that no terms other than the following arise (where we ignore numerical constants):

Page 33 of 87

Therefore, we assume as a particular solution

If the RHS of the given equation were lnx, successive differentiations would give 1/x, 1/x2, ... which are infinite in number. In this case, the method of undetermined coefficients is inapplicable and some other method must be used. B. Method of Variation of Constants This method is generally applicable but can be difficult because it requires integration of functions to determine the general solution. The method is illustrated by finding the solution to the following equation:

The homogeneous solution is

The solution is the general solution to the equation

A particular solution to the homogeneous equation therefore is

We therefore can regard the homogeneous solution as the product of the vectors (cosx, sinx) and (A, B). To find a general solution to the original equation, we try generalizing yH by letting A and B be functions of x rather than constants (hence the name of the method). If we are to determine two functions A(x) and B(x), we need to have two conditions to indentify the two functions. One of these is that

satisfies the original equation. The other we are free to choose as we need. We proceed by differentiating the function

Page 34 of 87

to obtain

Clearly, further differentiation will introduce more terms, so we seek a condition we may impose that will simplify this last equation. The following is what we seek:

which then gives

Therefore

Substituting these expressions for y' and y'' into the original equation gives

This, together with the earlier condition that

gives the following expressions for A'(x) and B'(x):

Therefore, by integration,

Page 35 of 87

which gives

as the general solution to the original equation. C. Operator Methods Suppose we solve the differential equation

by writing

where N-1(D) represents an operation to be performed on F(x). What is this operation? To gain insight, consider the simpler equation Dy = x. Then we would have

However, we can solve Dy = x by the usual methods:

Page 36 of 87

so it seems natural to define

It is then straightforward to show that, analogously,

Now consider

Formally, we have

Solving the differential equation by usual methods gives

so it is natural to define

Note that this reduces to our earlier definition for p = 0. Now consider

The operator (D-p1)(D-p2) has several important properties: (1) , so that

Page 37 of 87

The reasoning used here establishes that D operators may be multiplied or factored like algebraic quantities, as long as the pi are constants. (2) One may show that the factorization

is always possible and unique when the ai are constants. (3) D operators obey the commutative, associative, and distributive laws.

This last fact allows us to solve our original differential equation by writing

By applying our earlier results, we have

In a similar manner, we may write

Expressions like the left side of the last equation are often easier tohandle if resoved into Page 38 of 87

partial fractions. We briefly review partial fractions. Def The ratio of two polynomials is called a rational expression. Thm Every rational expression can be written as a sum of the forms

To apply the method of partial fractions, first examine the rational expression. If the degree of the numerator is less than that of the denominator, we can proceed. Otherwise, we use long division (synthetic division) to write

where deg R(x) < deg D(x). Ex. If we have

we perform long division

Page 39 of 87

to obtain

Once we have a rational expression with only the degree of the numerator less than that of the denominator, we apply the method, according to which (a) each factor (x-a)r in the denominator leads to a sum of the form

(b) each factor (x2 + bx + c)r in the denominator leads to a sum of the form

Ex.

We use partial fractions to write

which can be rewritten as

Therefore, equating coefficients,

which we can solve for A, B, C, and D to obtain

Page 40 of 87

With these results on partial fractions, we can re-write expressions such as [(D-p1)(D-p2)...(D-pn)]-1 in the form

Therefore, we have

which involves only single integrations. Ex. To find the general solution of (D2 - 1)y = e-x, we write

Note that if we set the constants of integration c1 and c2 equal to zero, we obtain a Page 41 of 87

particular solution. Ex. To find a particular solution of

we write

By long division in ascending powers of D:

Therefore, formally,

which can be verified as a particular solution.

SYSTEMS OF EQUATIONS I. General We wish to analyze systems of the following type:

Page 42 of 87

where fi are functions defined in some region D of Rn+1. To proceed more compactly, we define

Thus we can write our original equation system in vector form:

To solve this system means to find a real interval I and a vector N(t) defined on I such that (i) N'(t) exists for each t in I (ii) the point (t, N(t)) lies in D for each t in I (iii) N'(t) = f(t, N(t)) for every t in I II. Existence and Uniqueness Thm If f and Mf/Myk k are continuous in D, then given any point (t0, 0) there exists a unique solution N of the system

satisfying the initial condition N(t0) = 0. The solution exists on any interval I containing t0 for which the pints (t, N(t)) lie in D. Furthermore, the solution is a continuous function of the triple (t, t 0, 0). III. Linear Systems A. General A linear system is one which is linear in y, so that

where

Page 43 of 87

In what follows, we will need a definition of the norm (length) of a matrix. Def The absolute value norm of A is

Other definitions of norm are possible. In general, a norm is any function N:D 6 R satisfying the following three conditions: (i) N(a) $ 0 and = 0 if and only if a = 0 (ii) N(ca) = |c| N(a) for constant c (iii) N(a+b) # N(A) = N(B) Def The sequence of matrices {Ak} converges to the matrix A if and only if the sequence of real numbers {|A-Ak|} converges to zero. Because of the way the matrix norm is defined, this definition of matrix convergence means that Ak 6 A if and only if aij(k) 6 aij as k 6 4 i, j. B. Existence and Uniqueness Thm If A(t) and g(t) are continuous on an interval I, t0 , I, and |y| < 4, then the equation

has a unique solution N(t) satisfying the initial condition N(t0) = 0 and existing on I. Page 44 of 87

C. Linear Homogeneous Systems The linear homogeneous system is of the form

Note that, given the point (t0, 0), the homogeneous system has the unique solution N = 0. [By inspection, this is a solution; by the existence and uniqueness theorem, it is unique.] Note that, for any two solutions N1 and N2 and constants c1 and c2:

so that (c1N1 + c2N2) also is a solution. Consequently, the solutions to the homogeneous equation form a vector space, denoted V. Thm If A(t) is continuous on I, then the solutions to the homogeneous system form a vector space of dimension n. Proof: We already know the solutions form a vector space. We need only discover its dimension.. Let t0 be any point of I and let *1, *2, ..., *n be n linearly independent points in the space; the elementary vectors *i = (0, 0, ..., 0, 1, 0, ..., 0) are obviously such. By the existence and uniqueness theorem, the homogeneous system has n solutions Ni satisfying

To see that the Ni are linearly independent, consider

Setting t = t0, this becomes

which can only occur if the ai all are zero because of the linear independence of the *i. Therefore the Ni are linearly independent. Page 45 of 87

To see that the Ni span V, consider any element of V, say R(t). Let R(t0) = * , Rn. The *i form a basis of Rn (because they are linearly independent), so:

Now consider

This is a solution because the solutions are a vector space. Also,

Therefore N(t) and R(t) are both solutions satisfying the same initial condition. By uniqueness,

So the Ni span V, which therefore has dimension n. QED. Def An n-dimensional set of linearly independent solutions is called a fundamental set of solutions. Def An nxn matrix whose coumns are solutions of the homogeneous system is called a solution matrix. Def An nxn solution matrix whose columns are linearly independent (i.e., whose rank is n) is called a fundamental matrix. We denote the fundamental matrix formed from solutions Ni, N2, ..., Nn by M. Then the foregoing theorem's implication that every solution R is the linear combination

for some unique choice of constants ci can be restated as

where c is the column vector (ci, ..., cn)'. Page 46 of 87

Thm (Abel's formula) If M is a solution matrix of the homogeneous system on I and if t0 , I, then t,I

Proof: Let the columns of M be Nj, each having the components (N1j, N2j, ..., Nnj ). Because Nj is a solution,

It is a fact from linear algebra that

so that here

Each determinant on the RHS can be evaluated by elementary row operations. For example, in the first determinant we multiply the second row by a12, the third by a13, and so on, add these n-1 rows and then subtract the result from the first row. The resulting first row will be (a11N11, a11N12, ..., a11N1n), so that the value of the first determinant is a11(detM). Proceeding similarly with the other determinants, we obtain

Page 47 of 87

This is a first-order scalar equation for detM, whose solution is

QED

Cor Either detM(t) 0 t , I or detM(t) = 0 t , I. Thm A solution matrix M is fundamental if and only if detM(t) 0 t on I. Proof: (A) If detM(t) 0 t , I, then the Ni are linearly independent and M is fundamental. (B) If M is fundamental, then every solution has the form N(t) = M(t)c for some constant vector c. Then t0 , I and N(t0), the system N(t0) = M(t0)c has a unique solution for a given c because M is fundamental. Therefore detM(t0) 0, because any system Ax = b has a unique solution if and only if detA 0 (with the solution given by x = A-1b). So by Abel's formula, detM(t) 0 t , I. QED Note that in general a matrix may have linearly independent columns but have its determinant identically zero. For example,

According to the last theorem, this cannot happen for solutions to homogeneous systems. Note that, by Abel's formula, it is sufficient to test whether a matrix is fundamental by evaluating it at one point. This point often can be chosen to make the calculation easy. Thm If M is fundamental and C is a nonsingular constant matrix, then MC also is fundamental. Every fundamental matrix is of the form MC for some nonsingular C. Proof: Let M and Q be fundamental. The jth column Rj of Q can be written Rj = Mcj for some cj. Define C as the matrix whose columns are cj. Then Q = MC. Because detM 0 and detQ 0, Page 48 of 87

we also have detC 0. Reversing the argument gives the converse. QED D. Linear Nonhomogeneous Systems The nonhomogenous system is

where g(t) is usually called the forcing function. Suppose M is fundamental for the homogeneous system. Also suppose that N1 and N2 are solutions to the nonhomogeneous system. Then N1-N2 is a solution to the homogeneous system, in which case c such that

Thus, to find any solution to the nonhomogeneous system, we need only know one, because every other solution differs from the known one by some solution to the homogeneous system:

We will now see how to construct solutions to the nonhomogeneous system by using the method of variation of constants. Let M be a fundamental matrix of the associated homogeneous system. Suppose we attempt to construct a solution to the nonhomogeneous system of the form

where v is to be determined. Suppose such a solution exists. Then

Because M is fundamental,

so that we have Page 49 of 87

Therefore

Conversely, if we define R(t) by

then

Obviously, R(t0) = 0. We therefore have proven the variation of constants formula: Thm If M is a fundamental matrix of y' = A(t)y on I, then the function

is the unique solution of y' = A(t)y + g(t) satisfying the initial condition R(t0) = 0. Page 50 of 87

We see, then, that every solution of the nonhomogeneous equation has the form

where

R Nh

= =

variation of constants function solution of homogeneous system satisfying the same initial condition at N, for example, Nh(t0) = 0.

Note that, in the variation of constants formula, M-1 acts like the integrating factor in our earlier single equation problems. Moreover, the entire theory of linear equations developed earlier is a special case of the present development, for any such equation

or

or

Note that the eigenvalues of A equal the roots of the characteristic equation. E. Linear Systems with Constant Coefficients In general, it is not easy to find the fundamental matrix M. However, when A(t) is constant, M is easy to determine. Def A series of matrices converges if and only if the sequence of partial sums

converges, where convergence of a sequence of matrices is as defined earlier in terms of the Page 51 of 87

absolute value norm. Lem A sequence {Ak} of matrices converges if and only if, given a number ,$0, there exists an integer N(,)>0 such that |Am-Ap| < , whenever m, p > N(,). Def The matrix eM is defined as

This definition is sensible only if the series on the RHS converges. To see that it does, define

Note that

Then, for m > p,

Thus |Sm-Sp| is less than the tail of the infinite series defining e|M|. Because e|M| is finite, the series that defines it converges, which means its tail vanishes as the lower index of summation is raised. Therefore, given ,>0, there exists N>0 such that |Sm-Sn| < , for m,p > N, so that {Sk} and therefore the series defining e|M| converges. Thm If M and P commute (i.e., MP = PM), then Page 52 of 87

Thm If T is a non-singular nxn matrix, then

Thm The matrix

is fundamental for the system

with M(0) = I on t , (-4, 4). Proof: M(0) = I is obvious. We have by differentiation

for t , (-4, 4). Therefore eAt is a solution matrix. Furthermore,

so by Abel's formula

Ex.

To find the fundamental matrix of y' = Ax if

Page 53 of 87

note that

These matrices commute, so

But

We next develop a general method for determining the form of eAt when A is an arbitrary matrix. Thm Let 81, ..., 8k be the distinct eigenvalues of the nxn matrix A with respective multiplicities n1, ..., nk (note that Enj = n). Then (1) there exist k subspaces Xj of En such that En = X1 r X2 r ... r Xk, i.e., such that x = x1 + x2 + ... + xk, i.e., En is the direct sum of the Xj Page 54 of 87

(2) Xj is invariant under A, i.e., Axj , Xj xj , Xj (3) A - 8jI is nilpotent un Xj of index at most nj, i.e., qj # nj such that, xj , Xj,

Our goal is to find that solution N(t) to the system y' = Ay that satisfies the initial condition N(0) = 0. We know that N(t) = eAt0, so we willknow N(t) once we find eAt. But we know that

where vj is some (as yet unknown) vector in the subspace Xj. We also know that vj satisfies

Therefore,

[Note that

] So we can write

Page 55 of 87

To find eAt, let 0 successively equal the basis vectors e1, e2, ..., en because

Ex.

Let

so that we have a general 2-dimensional system. The eigenvalues of A are found from

Page 56 of 87

In this example, we consider the case where the two roots 81 and 82 are distinct. We have

which establishes the usual result that each part of the solution vector N(t) depends on both roots and two arbitrary constants. As yet, the vectors v1 and v2 are unknown. They are determined by the initial conditions N(0) = 0. Once we determine v1 and v2, we can determine eAt, which we would need if we wanted the general form of N(t) = eAt0 or if we wanted to solve a non-homogeneous equation for which y'(t) = Ay(t) was the associated homogeneous equation. To determine v1 and v2, we use the conditions

Page 57 of 87

Solving the first equation for x2 gives

Substituting into the second equation gives

[Note that if we had solved the equations of (A-81I)x = 0 in the other order, we would have obtained

Page 58 of 87

which seems different but is not because

This last equality can be seen by cross-multiplying to obtain

which is the characteristic equation for A evaluated at the root 81 and so necessarily true.] Similarly, the condition

So v1 and v2 have the forms

Therefore

Page 59 of 87

or

Substitute these results into the expressions for v1 and v2 to obtain

We know that

Page 60 of 87

We let 0 successively equal the basis vectors (a)

and

(b)

So from the relation

we get

Page 61 of 87

Ex.

Let

Then 81 = 3, k = 1, and n1 = 2. X1 = E2.

Therefore

is the solution with N(0) = 0. To construct eAt, successively let 0 =

and

Page 62 of 87

so that

Ex.

Let

The characteristic polynomial of A is

so that the eigenvalues are

81 = 1 , n1 = 1 82 = 2 , n2 = 2
Page 63 of 87

k = 2 , X1 + X2 = E3 We use the equations (A-I)x = 0 and (A-2I)2x = 0 to determine X1 and X2.

so that X2 is spanned by the vectors v2 = (x1, x2, x3)' with x1 = x2 and x3 arbitrary. Now,

or

which leads to

Page 64 of 87

Therefore

The fundamental matrix is

Finally, we may now use the variation of constants formula to solve the nonhomogeneous equation

to get

Page 65 of 87

Two useful properties of solutions to linear systems with constant coefficients are: Thm If all eigenvalues of A have real parts negative, then every solution N(t) of y' = Ay approaches zero as t 6 4. More precisely, > 0, F > 0 such that

Thm If M > 0, T > 0, and a such that

then K > 0 and b such that every solution N(t) of y' = Ay + g(t) satisfies

and H > 0 and c such that

Notice that our earlier treatment of nth-order linear equations is a special case of the theory of linear systems. In particular, when the coefficients are constant, the fundamental matrix has the form eAt, which is a generalization of the terms in the nth-order equation's solutions of the form , where 8i was a root of the characteristic equation. Also note that such roots are the same as the eigenvalues in the system representation. Finally, the integrating factor in the 1storder linear equation was epx for the case of constant coefficients, which is a special case of the system result that the fundamental matrix eAt is the integrating factor in the variation of constants formula. F. Linearization Results for linear systems are far easier to obtain than for non-linear systems. Consequently, it often is advantageous to linearize a non-linear system by application of Taylor's formula. The expansion is performed about whatever point is of interest, usually a stable equilibrium pint when one exists. Taylor's formula for the multivariate case is

Page 66 of 87

BOUNDARY VALUE PROBLEMS These are problems that involve finding solutions of a differential equation that satisfy prescribed conditions at two given points, called the boundary conditions. Methods of solving such problems are not discussed here.

Page 67 of 87

DIFFERENCE EQUATIONS
Difference equations are the discrete-time analog of differential equations. Ex. yt = a + byt-1 + et

Aside from details, the theory behind difference equations is the same as that for differential equations. Our goal is to learn how to solve equations of the form

As with differential equations, the parameters ai may be functions of time, but here we will concentrate on the constant coefficients case because most of the results in the literature pertain to that case. I. Basics We begin with the simplest difference equation

or

which is a first-order homogeneous equation. By successive substitution, we obtain

where y0 is the value of y at time initial time t=0. Thus a general solution to this equation is

Page 68 of 87

where c is an arbitrary constant to be determined byinitial conditions. In this example, the initial condition is yt0 = y0 given, so that c = y0. We now consider the nth-order equation

Recall that, with linear differential equations, we found that cemt was a solution of the first-order equation and then searched for the set of values for m that would make cemt a solution to the nthorder equation. We then used the principle of superposition to conclude that the general solution was . We proceed analogously here. We know that cat solves the first-order equation. Would functions of the form yt = rt solve the nth-order equation? If so, for what values of r? Let us try yt = rt in the equation

Y either r = 0 or r is a root of

Thus, if this characteristic equation has n distinct roots, we have n solutions to the homogeneous equation of the form yt = rit. To obtain a general solution, we use the following Thm If the function yi(t) satisfy the homogeneous equation, then so does

where the ci are arbitrary constants. Thus our general solution to the homogeneous equation is

to deal with the case of repeated roots, we appeal to the following

Page 69 of 87

Thm If ri, i = 1, ..., p, are the distinct roots of the characteristic equation, with respective multiplicities ni (Eni = n), then the general solution of the homogeneous equation is

When complex roots occur, they always occur in conjugate pairs (a bi). Thus the expression for yt will contain terms of the form

These terms can be rewritten as

where

The ability to re-write the complex root terms this way follows from three facts: (1)

(2) (3) Ex. The second-order equation

Page 70 of 87

has the characteristic equation

If |a| < 1, this equation has roots

so that the general solution is

with

Ex.

The same equation has repeated roots at 1 if a = 1, in which case the solution is

II. Lag Operators The lag operator is defined by

Formally, the opeator Ln operates on one sequence .

to give a new sequence

The polynomial A(L) in the lag operator L is defined by Page 71 of 87

where the aj are constant. Note that A(L) is itself an operator; applying it to Xt gives

We assume henceforth that A(L) is rational, in which case it can be expressed as

The simplest example of a rational polynomial in L is

By long division (synthetic division), we can rewrite this as

If we apply this particular A(L) to Xt, we obtain

which is sometimes called the backward expansion of A(L)Xt. Note that if |8| $ 1, then if Xt = a constant, we have

Page 72 of 87

Thus sometimes it is important to restrict |8| < 1. An alternative expansion is available:

This is the forward expansion. Thus we can write

If |8| < 1, this forward expansion equals +4; if |8| > 1, then |1/8| < 1 and the forward expansion converges. Thus the forward expansion sometimes is useful when |8| > 1; indeed, it often has a natural interpretation in terms of current expectations of future values. III. Nonhomogeneous Equations We will examine two ways of solving nonhomogeneous difference equations: (1) undetermined coefficients (2) inverse operators A. Undetermined coefficients Page 73 of 87

This method can be used whenever the forcing sequence is itself the solution to some linear difference equation with constant coefficients. Given a difference equation

we seek an operator 2(L), called an annihilator operator, such that

This operator is then applied to both sides of the original equation, giving

which is a homogeneous equation that can be solved as previously discussed. The solution that emerges then is substituted into the original equation to evaluate the undetermined coefficients arising from 2(L). Ex. Consider

The forcing function is

Because this particular function is the solution to the homogeneous equation

we try

as the annihilator. In the case at hand,

Page 74 of 87

so indeed this 2(L) is an annihilator for 3t. Therefore, we apply 2(L) to the original equation:

The auxiliary equation is

which has roots

the last of which arises from 2(L). The general solution would be of the form

from which we can determine the particular solution

by letting c1 = c2 = 0. Substituting this particular solution into the original solution gives

Page 75 of 87

Therefore the general solution is

Some general notes in applying this method: (1) If the forcing functions a sum of other functions

then the annihilator is the product fo the annihilators 21(L) and 22(L) of x1 and x2:

(2) If the additional roots of the augmented characteristic equation that arise from the annihilator repeat any of the roots fo the original characteristic equation, we must follow the rules for repeated roots, as usual. (3) All that has been said applies when the forcing function is of the form B(L)xt. The annihilator is the same for xt alone; consequently the solution function is of the same form as well, with only the constants differing. (4) Gabel and Roberts has a table fo some annihilators. B. Inverse Operators An inverse operator is an operator that is the inverse of some other operator. Ex. The inverse operator corresponding to

Page 76 of 87

is

In general, for a linear operator N(L) = a0 + a1L + ... + anLn, the corresponding inverse operator is

The method of partial fractions was discussed earlier in the section on differential equations. We can use inverse operators to solve difference equations. If the given equation is

Then the solution is

To get some insight into how to express this solution in explicit terms, consider the simple equation

We know the solution is

But, by inverse operators,

Therefore, we define

In essence, inverse operation in linear difference equations is similar to inverse operation in linear Page 77 of 87

differential equations. There, the solution method was (a) multiply both sides by the integrating factor (b) integrate (c) divide out any function multiplying the dependent variable Ex. To solve, we multiply by the integrating factor ept:

We then integrate to obtain

(Note the constant of integration.) Finally, we isolate y:

Just as a constant of integration multiplied by e-pt appears in the continuous case, so does a constant of summation multiplied by at appear in the discrete case:

So now consider a slightly more complicated case:

The solution is

The first term on the RHS is the discrete analog of the integral of the forcing function that appears in the continuous case; the second term is the constant of summation. As before,

Page 78 of 87

In general, this is as far as we can go; but if xt has some special form, we may be able to write the solution in a more informative way. In particular, if xt is a constant x, then the solution is

If |a| < 1, then yt is finite for all t:

Note that, if |a| > 1, we may want to use the forward expansion of (1-aL)-1:

which is finite if c = 0 but 6 4 otherwise. IV. Systems of Equations A system of difference equations has the form

where y and x are nx1 vectors and A and B are nxn matrices. We will be concerned with the case where A and B are constant. One method of finding yt is iteration:

Page 79 of 87

or, if initial time is t0,

The problem with this type of solution is that the matrices in each term change with each iteration in a way that is difficult to foresee in general. Little insight into the structure of the solution is provided. To get around this problem, we need a closed-form solution for Ak. The method for obtaining one that is discussed here is based on the spectral decomposition of a matrix. Thm Any nxn matrix A has a representation of the form

where p is the number of distinct eigenvalues, 8i are the eigenvalues, each having multiplicity ri, Ni is a matrix such that , and the matrices Ei and Ni satisfy the following properties:

Page 80 of 87

Thm Any function f of the matrix A has the representation

Ex.

Suppose

The eigenvalues of A are 81 = 1/2, 82 = 1/4. Therefore

Now,

and

so

Page 81 of 87

and similarly

Therefore,

so

Ex.

Suppose

The eigenvalues are 81 = 82 = 1/2. Therefore

and Page 82 of 87

From this last equation, we have

so

Thm There exists a nonsingular matrix T such that

where p is the number of distinct eigenvalues of A and

Page 83 of 87

is of dimension rixri with ri the multiplicity of 8i. The importance fo this theorem is that

so that

The terms of Jk are powers of the 8i multiplied by numbers determined by k, so that the elements of Ak comprise linear combinations fo powers of the 8i. As a result, yt also comprises such linear combinations so that the asymptotic behavior of yt depends entirely on the values of these 8i. In particular, if |8i| < 1 i, then yt is asymptotically stable (if the forcing function xt is bounded). Thm Suppose the eigenvalues of A are distinct. Then

Page 84 of 87

Finally, note that any nth order difference equation can be put into system form. The usual kind of equation we will deal with has the form

which can be put into system form by defining

or

where

The more general form of the nth order equation

Page 85 of 87

is only slightly more difficult to handle. For the moment, assume m # n. Then re-write the difference equation as

Define

Then

or

If we now define

we can write the state-space system

or Page 86 of 87

where

Then

which is called the output equation. In the case where m > n, simply define enough extra early xi to bring n up to equality with m. then the first m-n elements of the nth row of A will be zeroes, and the foregoing analysis applies.

Page 87 of 87