DifferentialEquations
Chapter 8 · Section 8.3

8.3 The Method of Successive Approximations

The power series of Section 8.2 builds the solution one coefficient at a time. Picard's method builds it one function at a time. We start with a rough guess, feed it into a formula, and get a better guess; then repeat. Each new function is called a successive approximation (or an iterate).

The method matters for two reasons. In practice, the first few iterates are often good approximations near the initial point. In theory, the same iteration is the standard proof that solutions exist and are unique. So this section also explains why the existence–uniqueness theorem of Section 1.3 is true.

From a differential equation to an integral equation

The key move is to replace the derivative by an integral. Suppose solves

Integrate both sides of from to and use :

The unknown function now appears inside an integral. An equation like this is called an integral equation. The initial condition is no longer a separate statement: it is built in, because the integral vanishes at .

Theorem 8.3.1 (Equivalence)

Let be continuous on a region containing the graph of a continuous function defined on an interval that contains . Then is a solution of the initial-value problem , on if and only if

We just proved "only if". For "if": the integrand is continuous, so by the fundamental theorem of calculus the right-hand side is differentiable with derivative . Hence , and setting gives .

The iteration

The integral equation has the form , where is the rule "take a function, plug it into , integrate, add ". A solution is a function that leaves unchanged. Picard's idea: start anywhere, apply again and again, and hope the results settle down.

Definition 8.3.1 (Picard iterates)

For the initial-value problem , , the successive approximations are the functions

The constant starting function is the natural first guess: it at least satisfies the initial condition. Every later iterate satisfies the initial condition too, since each integral starts at .

Method in a nutshell: successive approximations
  1. Set .
  2. Substitute into . Simplify the integrand.
  3. Integrate from to and add ; this is .
  4. Repeat. Compare consecutive iterates: the terms that stop changing are the trustworthy part.
Example 8.3.1 (Finding the pattern)

Find , , for , . Find a formula for and the limit as .

Solution
  1. The iteration. Here , , , so .
  2. First iterates.
  3. Pattern. Write the coefficients as , , . So Induction confirms it: if this holds for , then integrating from to gives , which shifts every term up by one power.
  4. Limit. is the -th partial sum of the exponential series, so for every . That is the exact solution.
Example 8.3.2 (Watching the iterates converge)

For , , compute , , and compare with the exact solution .

Solution
  1. The iteration. , starting from .
  2. . .
  3. . The integrand is , so
  4. . The integrand is , so
  5. Compare with the exact series. is right through , through , through ; the last term of each iterate is still "under construction" (for : instead of ).
  6. How good on ? The largest error , found numerically:
    0123456
    max error0.8393970.1606030.0134220.0061870.0022690.0005080.000087
    The errors shrink steadily on the whole interval, not just at one point. Figure 8.3.1 shows the same thing graphically.
Figure 8.3.1. Picard iterates for , . Each iteration widens the interval around on which the iterate is indistinguishable from the exact solution (black).

The calculator below performs the iteration for , with a polynomial, showing each integrand and integral. It starts with Example 8.3.2. Try with to see the pattern of Example 8.3.1, then a nonlinear such as .

For linear equations the iterates stay tidy. For nonlinear equations they produce correct low-order terms together with extra high-order terms that later iterations keep correcting.

Example 8.3.3 (A nonlinear equation)

Find , , for , . The exact solution is ; compare.

Solution
  1. The iteration. , with .
  2. and .
  3. . First square : Then
  4. Compare. . So is correct through ; its term () is not yet right.
  5. Numbers. At : , , . At the error of is already , and near , where blows up, the polynomials cannot follow at all (Figure 8.3.2).
Figure 8.3.2. Iterates for , , against . The shaded band is the interval on which Picard's theorem guarantees convergence (Example 8.3.4); the true solution lives on (dashed lines).

Why the iterates converge: the Lipschitz condition

When do the iterates converge, and to what? The answer hinges on how strongly reacts to a change in .

Definition 8.3.2 (Lipschitz condition)

A function satisfies a Lipschitz condition (with respect to ) on a region if there is a constant such that for all points and of . is called a Lipschitz constant.

In words: changing by a small amount changes by at most times that amount. The easiest way to check it is through the partial derivative. If is a rectangle and is continuous on it, the mean value theorem gives, for some between and ,

so works. That is why the existence–uniqueness theorem of Section 1.3 asks for continuity of .

Theorem 8.3.2 (Picard's theorem)

Let and be continuous on the closed rectangle Let , , and . Then on the interval :

  1. every Picard iterate is defined, and its graph stays in ;
  2. the iterates converge uniformly on to a function ;
  3. is a solution of , on , and it is the only solution there;
  4. if , the error satisfies, for every in ,
Why it works

Staying in . Since in , each iterate satisfies . This is why may have to be smaller than : the iterates must not leave the rectangle where our bounds hold.

Consecutive iterates get close very fast. First, . Next, by the Lipschitz condition, Repeating the argument (induction) gives

Adding up. . The bounds on the differences are the terms of the convergent series , so the iterates converge (uniformly, by the Weierstrass test). Passing to the limit in the integral equation shows that the limit is a solution, and summing the tail of the series gives the error bound in part 4. Uniqueness follows from a similar estimate applied to the difference of two solutions.

The factorial in the denominator is the hero: it eventually beats any power , however large or may be.

Example 8.3.4 (The guaranteed interval)

Apply Theorem 8.3.2 to , on the rectangle , . What is the largest interval the theorem can guarantee? Compare with the true solution .

Solution
  1. Hypotheses. and are continuous everywhere, so any rectangle will do.
  2. Constants. On : and .
  3. Interval. . Since , we always have , with equality exactly at . Choosing and any gives the best guarantee:
  4. Compare. The solution actually exists on , about three times wider. The theorem is safe but often conservative: it guarantees an interval, not the largest one.
  5. Error bound. With , , , part 4 gives for on : The actual error at is (Example 8.3.3). Worst-case bounds like this one are valuable because they are guaranteed, not because they are sharp.
Example 8.3.5 (Errors can grow before they shrink)

For the iterates of Example 8.3.1, , tabulate and the error for , and compare with the bound .

Solution
  1. The bound. is a Taylor polynomial of evaluated at . By Taylor's theorem the remainder is for some between and . Since there, the error is at most .
  2. Table. The exact value is .
    errorbound
    03.0000002.5939946.000000
    1−3.0000003.4060066.000000
    23.0000002.5939944.000000
    3−1.0000001.4060062.000000
    41.0000000.5939940.800000
    50.2000000.2060060.266667
    60.4666670.0606610.076190
    70.3904760.0155300.019048
    80.4095240.0035180.004233
  3. Lesson. The first iterates are poor at ; the error even grows from to . But from on, each new iteration multiplies the bound by , a factor below that keeps shrinking, so the factorial soon wins (Figure 8.3.3). Far from , more iterations are needed before convergence "kicks in".
Figure 8.3.3. Error of for , (circles) and the bound (squares), on a logarithmic scale. After a slow start, the error falls faster than any geometric sequence.

When the Lipschitz condition fails

Picard's method always produces something. Without a Lipschitz condition, however, that something may be only one of several solutions, and the method gives no warning.

Example 8.3.6 (A silent choice)

Apply Picard's method to , . Is the result the only solution?

Solution
  1. Iterates. , and . Every iterate is , so the method returns , which is indeed a solution.
  2. Another solution. Let for and for . For , ; for both sides are ; and at both one-sided derivatives are . So is a second solution with .
  3. What went wrong? Near , so no Lipschitz constant exists on any rectangle around , and Theorem 8.3.2 does not apply. The iteration converged, but to just one of infinitely many solutions (the curve may leave the axis at any point ).
Common mistake

Convergence of the iterates does not by itself prove uniqueness. Check the hypotheses of Theorem 8.3.2 (continuity of and , or at least a Lipschitz condition) before claiming that the limit is the solution.

Picard's method in practice

For hand computation, the method is limited: the integrands grow messy after two or three steps, and for many (say ) the integrals cannot be done in closed form at all. Computer algebra pushes this further, but the real strength of the method is theoretical. It shows that a solution exists, that it is unique, and on which interval, and it gives an explicit bound on the error.

For actual numbers far from the initial point, engineers use step-by-step numerical methods. The simplest of these, Euler's method, is the subject of the next section.

Exercises

In Exercises 1–7, compute the indicated Picard iterates (with ).

  1. , : , , . What is ?
    Answer , , ; the limit is .
  2. , : , , . Compare with the exact solution .
    Answer , , : partial sums of the series of .
  3. , : , , , and the limit.
    Answer , , ; limit .
  4. , : , . Compare with the exact solution .
    Answer , ; agrees with through .
  5. , : , , . Compare with the exact solution .
    Answer , , ; these are the first terms of .
  6. , : , in powers of , and the limit.
    Answer , ; limit .
  7. , : , . Through which power does agree with the Taylor polynomial of Example 8.2.2?
    Answer , . It agrees with through (and agrees through ).
  8. Show that a continuous function satisfies if and only if it solves an initial-value problem. Which one?
    Answer , (by Theorem 8.3.1).
  9. Find a Lipschitz constant for on the rectangle , .
    Answer and , so .
  10. For , , use the rectangle , in Theorem 8.3.2. Show that and find the best choice of . Compare with the interval of existence of the exact solution.
    Answer . The function is largest at , giving . The exact solution exists on .
  11. Show that satisfies a Lipschitz condition with although it is not differentiable at , while satisfies no Lipschitz condition on any rectangle containing points with in its interior.
    Answer (triangle inequality). For : as .