DifferentialEquations
Chapter 8 · Section 8.8

8.8 Numerical Methods V: Higher-Order Equations and Systems

The methods of Sections 8.4–8.7 were written for a single first-order equation . Engineering models are rarely that simple. A mass on a spring, an RLC circuit or a swinging pendulum obeys a second-order equation, and a structure with several masses or a circuit with several loops leads to a system of equations. Most of these problems have no formula solution, especially when they are nonlinear. The good news is that nothing new has to be invented: every higher-order equation can be rewritten as a first-order system, and every method we know works for systems with essentially no change. This section shows how.

Because the applications are usually about motion in time, from now on we use for the independent variable and write the unknowns as , , as in Chapter 7.

From one higher-order equation to a first-order system

Consider a second-order equation solved for the highest derivative,

Introduce the velocity as a new unknown, . Then , and the single equation becomes the pair

The two problems are equivalent: if solves the second-order problem, then solves the system; conversely, the first equation of the system says , and then the second says . The two initial conditions of the original problem turn into one initial condition for each unknown.

Definition 8.8.1 (Equivalent first-order system)

For the th-order equation with (), set

Then

In vector form, , .

Example 8.8.1 (A third-order equation)

Write the initial-value problem as a first-order system.

Solution
  1. Solve for the highest derivative. .
  2. New unknowns. , , .
  3. The system. Note that the system is nonlinear (because of ); that is no obstacle for numerical methods.

Systems of several higher-order equations are handled the same way: introduce one new unknown for each derivative below the highest order of each unknown function (Exercise 2).

Numerical methods for systems

We write the general two-equation system as

The idea is simply to treat the pair as one unknown vector. Every formula for is applied componentwise, with the important rule that all components are advanced together, using values from the same stage.

Euler's method

Both right-hand sides use the old values .

The RK4 method

For each stage we compute a slope for and a slope for , and both are evaluated at the same trial point:

and then

In vector notation this is exactly Definition 8.6.1 with , and vector slopes . The same is true of Heun's method and of ABAM, and the orders do not change: Euler is still first order, Heun second, RK4 and ABAM fourth.

Method in a nutshell: solving a higher-order IVP numerically
  1. Solve the equation for its highest derivative.
  2. Introduce , (and , … for higher order); write the first-order system and its initial values.
  3. Choose a method and a step ; check that is small compared with the time scales of the problem (see the section on stability below).
  4. At every stage compute all component slopes at the same trial point, then advance all components together.
  5. Read off (the original unknown) and, as a bonus, .
Common mistake

Do not update first and then use the new while computing (or the new when computing ). Within a step, every slope of a given stage must be computed from the same point. Mixing old and new values silently produces a different method, usually with a lower order.

Worked example: a damped oscillator

A mass of kg hangs on a spring of stiffness N/m with a damper of coefficient N·s/m. It is pulled down m and released from rest. Its displacement satisfies (Chapter 5)

This linear problem has a known solution, which lets us measure the errors. The characteristic equation has roots , so . The initial conditions give and , so

With the equivalent system is

So and .

Example 8.8.2 (Euler's method for the oscillator)

Take two Euler steps with and compare with the exact values.

Solution
  1. First step from :
  2. Second step from :
  3. Compare. Exact: , ; , . After two steps the position is already off by . Notice that the first step leaves unchanged because the velocity starts at zero: Euler cannot "see" that the mass begins to accelerate.
Example 8.8.3 (RK4 for the oscillator)

Solve the same problem by RK4 with on . Show the first step in full.

Solution
  1. Stage 1 at :
  2. Stage 2 at :
  3. Stage 3 at :
  4. Stage 4 at :
  5. Combine. Compare with the exact and : errors of and , against and for Euler.
  6. All ten steps.
    exact exact
    0.10.9535830.953557−0.891333−0.891326
    0.20.8298740.829824−1.541026−1.540967
    0.30.6539970.653934−1.934477−1.934343
    0.40.4512140.451150−2.082762−2.082548
    0.50.2446270.244575−2.016982−2.016704
    0.60.0534920.053462−1.781844−1.781530
    0.7−0.107813−0.107813−1.429172−1.428857
    0.8−0.230195−0.230164−1.011964−1.011684
    0.9−0.309710−0.309648−0.579411−0.579199
    1.0−0.346979−0.346893−0.173172−0.173050
    Over the whole interval the position error stays below and the velocity error below .
Figure 8.8.1. RK4 with for , , . Left: position and velocity against time. Right: the phase plane ; the trajectory spirals into the equilibrium at the origin. The RK4 points lie on the exact curves to plotting accuracy.

Use the solver to reproduce Example 8.8.3 and to explore. Switch the method to Euler and try and then ; or change to model a different spring, for instance -25*x - 0.5*y.

A nonlinear example: the pendulum

For a linear oscillator we did not really need a computer. The power of numerical methods shows when the equation is nonlinear. A pendulum of length swinging through angle in air obeys

For small swings one replaces by and obtains a linear equation like the one above. For large swings this approximation is poor, and the nonlinear equation has no elementary solution.

Example 8.8.4 (Large swings of a pendulum)

Take (a pendulum about m long) and . The pendulum is released from rest at rad (about ). Use RK4 with to compute for , and compare with the linearized model .

Solution
  1. System. With , :
  2. First stage, as a check. , . The remaining stages follow the pattern of Example 8.8.3; the first step gives , .
  3. Linearized model. Its characteristic roots are with , and with the same initial data
  4. Results (60 RK4 steps; values every s, rounded to four decimals):
    (RK4) (RK4)
    0.50.2055−2.63570.1137
    1.0−0.8010−0.7271−0.8454
    1.5−0.38502.0657−0.2117
    2.00.59511.14240.6991
    2.50.4639−1.51300.2764
    3.0−0.4117−1.3263−0.5645
    A computation with a much smaller step agrees with these RK4 values to within .
  5. Interpretation. The two models differ by more than rad (about rad at worst) already within the first three seconds. The nonlinear pendulum swings more slowly: for a swing of rad its period is about longer than the small-angle value s, so it falls further and further behind the linear model (Figure 8.8.2).
Figure 8.8.2. Example 8.8.4. The nonlinear pendulum (solid, RK4) has a longer period than the linearized model (dashed), so the two curves drift out of phase.

Try the pendulum in the solver above: enter as -9*sin(x) - 0.3*y, , and . Then raise to (almost upside down) and see how the motion changes.

Stability: when the step is too large

Example 8.4.6 gave a first warning: a larger does not always mean just a larger error. For oscillating or decaying solutions something more dramatic can happen: beyond a certain step size the numerical solution grows without bound, even though the true solution dies out. This is called numerical instability, and for systems it needs a closer look.

The test equation

The cleanest way to understand it is the test equation with , whose exact solution decays. One step of a method multiplies by a fixed factor, which we found in Sections 8.4 to 8.6. With :

The numbers decay only if the factor has absolute value at most . For real this means

Why does this matter for systems? For a linear system the solution is built from modes , one for each eigenvalue of (Chapter 7), and a numerical method treats every mode like a separate test equation. So each eigenvalue must satisfy the stability condition. For a nonlinear system the same reasoning applies, approximately, to the eigenvalues of the linearization near the solution.

Example 8.8.5 (Euler's method blows up)

For the oscillator , of Example 8.8.2, find the largest step for which Euler's method gives decaying numbers. What happens with ?

Solution
  1. Eigenvalues. The matrix has characteristic equation , so (the same numbers as the characteristic roots of the original equation).
  2. Euler's factor. For : (The conjugate eigenvalue gives the same value.)
  3. Condition. .
  4. With . The factor is per step. After steps () Euler gives , while the true displacement is only about . With the factor is and the numbers decay, although more slowly than the true factor .
  5. RK4. Solving numerically for gives . RK4 stays stable with steps more than four times larger than Euler can use.
Figure 8.8.3. Left: Euler's method for decays with but grows with . Right: for the undamped oscillator , Euler's points spiral outward for every , while RK4 stays on the exact closed orbit.
Undamped oscillations

For the eigenvalues are and Euler's factor is for every : Euler always pumps energy into an undamped oscillator (Exercise 5). RK4's factor satisfies , which is at most when ; RK4 loses a tiny amount of energy per step instead. This is one reason why Euler's method is never used for long simulations of vibrating systems.

Stiff problems

Trouble arises when a system contains time scales of very different sizes, for example eigenvalues and . The fast mode dies out almost immediately and is invisible in the solution, yet an explicit method must still respect for RK4, for the whole computation. Such problems are called stiff. They are common in chemical kinetics and circuits, and they are solved with implicit methods (relatives of the implicit trapezoid and Adams–Moulton formulas), which are beyond this course. A practical warning sign: if halving changes the answer completely, or the numbers explode, suspect instability before blaming the model.

Summary

  • An th-order equation becomes a system of first-order equations by taking as unknowns.
  • Euler, Heun, RK4 and ABAM work for systems componentwise; all components are advanced together, stage by stage. Their orders are unchanged.
  • For decaying or oscillating problems the step must also satisfy a stability condition, for every eigenvalue . Euler requires for real ; RK4 allows about .
  • Euler's method is unstable for undamped oscillations at every step size.

Exercises

  1. Write , , , as a first-order system.
    Answer , , ; , , .
  2. Two coupled masses obey , . Write this as a system of four first-order equations.
    Answer With , , , : , , , .
  3. Take two Euler steps with for , , . Compare with .
    Answer , ; exact , .
  4. Take one RK4 step with for , , , . Show that and , and compare with and .
    Answer (), (). These are the Taylor polynomials of and .
  5. For , , show that Euler's method satisfies . What does this mean physically?
    Answer , ; expanding, the cross terms cancel and the identity follows. The (scaled) energy grows by the factor every step, so the computed oscillation spirals outward.
  6. For , find the largest step for which Euler's method gives decaying numbers.
    Answer ; .
  7. For , what is the largest stable step for Euler's method and (approximately) for RK4?
    Answer Euler: . RK4: .
  8. A predator–prey model reads , with , . Take one RK4 step with .
    Answer , , , ; , .
  9. The charge in a series RLC circuit satisfies , , . Write it as a system for and the current , and take one RK4 step with . Compare with the exact solution , .
    Answer , . , ; exact , .
  10. Write Heun's method for the system , , and take one step with for the oscillator of Example 8.8.2.
    Answer Predictor , slopes at , then , . Here , , , so (exact , ).