7.7 The Matrix Method: n Equations

Real systems rarely stop at two unknowns. Three tanks in series, a building with three floors, a circuit with three loops, a chain of radioactive decays: each leads to a system ${\mathbf{x}}^{\prime }=A\mathbf{x}$ with a $3\times 3$ (or larger) matrix. The good news is that the method of Section 7.6 extends almost word for word. Eigenvalues still give exponentials, complex pairs still give sines and cosines, and missing eigenvectors are still replaced by generalized eigenvectors. Only two things change: the characteristic polynomial now has degree $n$, and a repeated eigenvalue can be "short" by more than one eigenvector, which leads to chains.

We study

with most examples for $n=3$. At the end of the section we add two short tools that complete the picture: the matrix exponential $e^{At}$ and variation of parameters for nonhomogeneous systems.

The eigenvalue method for $n$ equations

Exactly as in Section 7.6, $\mathbf{x}=e^{\lambda t}\mathbf{v}$ solves ${\mathbf{x}}^{\prime }=A\mathbf{x}$ if and only if $A\mathbf{v}=\lambda \mathbf{v}$ (Theorem 7.6.1; the proof never used $n=2$). To get the general solution we need $n$ independent solutions. By Section 7.8, $n$ solutions are independent exactly when their Wronskian (the determinant of the matrix having the solutions as columns) is nonzero at one point. For eigenvector solutions, at $t=0$ that determinant is just $\det [{\mathbf{v}}_1\cdots {\mathbf{v}}_n]$. This gives:

Getting the eigenvalues of a $3\times 3$ matrix

The characteristic equation $\det (A-\lambda I)=0$ is now a cubic. Exam and textbook problems are designed to have simple roots, and a few habits make them easy to find:

• Expand along a row or column with zeros. If a column has only one nonzero entry, the determinant factors immediately (Example 7.7.2).
• Look for a common factor before multiplying out. Adding one row (or column) of $A-\lambda I$ to another sometimes produces a row like $(1-\lambda ,1-\lambda ,0)$, from which $(1-\lambda )$ can be pulled out (Example 7.7.3).
• Rational root test. The constant term of $\det (A-\lambda I)$ is $\det A$, and the leading coefficient is $(-1{)}^n$. So for an integer matrix every integer eigenvalue divides $\det A$. Test those divisors, then divide out the factor, as in Section 4.2.
• Check. The eigenvalues add up to $\mathrm{tr}A$ and multiply to $\det A$.

Distinct real eigenvalues

Complex eigenvalues

Theorem 7.6.3 (real and imaginary parts of a complex solution are real solutions) holds for any real $n\times n$ matrix, with the same proof. So each pair of complex conjugate eigenvalues $\alpha \pm i\beta$ with eigenvector $\mathbf{a}+i\mathbf{b}$ for $\alpha +i\beta$ contributes the two real solutions

and the conjugate eigenvalue is not used separately. Since the characteristic polynomial of a real $3\times 3$ matrix is a real cubic, it always has at least one real root; so a $3\times 3$ matrix has either three real eigenvalues or one real eigenvalue and one complex pair.

The geometry is worth a look. The real eigenvalue $-3$ is more negative than the real part $-1$ of the complex pair, so the component along ${\mathbf{v}}_1$ dies out first. Every trajectory therefore drops quickly onto the plane spanned by $\mathbf{a}$ and $\mathbf{b}$ and then spirals into the origin inside that plane (Figure 7.7.1). In the graphs against $t$ (Figure 7.7.2) the term $3e^{-3t}$ in $x_2$ is visible only at the start.

Repeated eigenvalues

Let $\lambda$ be an eigenvalue of algebraic multiplicity $m\ge 2$. Everything depends on how many independent eigenvectors it has (its geometric multiplicity, Definition 7.5.6).

Enough eigenvectors

If $\lambda$ has $m$ independent eigenvectors, nothing new is needed: each gives an exponential solution and Theorem 7.7.1 applies. To find them, solve $(A-\lambda I)\mathbf{v}=0$; the number of free unknowns equals the number of independent eigenvectors, which is $n-\mathrm{rank}(A-\lambda I)$. This always happens, for instance, when $A$ is symmetric ($A^T=A$): a real symmetric matrix has real eigenvalues and $n$ independent eigenvectors.

Not enough eigenvectors: chains

If $\lambda$ has fewer than $m$ independent eigenvectors, we extend the idea of Section 7.6. There we found ${\mathbf{x}}_2=e^{\lambda t}(t\mathbf{v}+\mathbf{p})$ with $(A-\lambda I)\mathbf{p}=\mathbf{v}$. If one more solution is still missing, we go one step further.

Why ${\mathbf{x}}_3$ works. Differentiate with the product rule:

On the other side, using $A\mathbf{v}=\lambda \mathbf{v}$, $A{\mathbf{p}}_1=\lambda {\mathbf{p}}_1+\mathbf{v}$ and $A{\mathbf{p}}_2=\lambda {\mathbf{p}}_2+{\mathbf{p}}_1$ (the chain equations rewritten),

The two expressions are identical. Independence follows at $t=0$, where the three solutions equal $\mathbf{v},{\mathbf{p}}_1,{\mathbf{p}}_2$, and the vectors of a chain are always independent (apply $(A-\lambda I{)}^2$ to a relation $c_1\mathbf{v}+c_2{\mathbf{p}}_1+c_3{\mathbf{p}}_2=0$ to get $c_3\mathbf{v}=0$, so $c_3=0$; then apply $A-\lambda I$ to get $c_2=0$; then $c_1=0$).

Are there always enough chains? Yes. A theorem of linear algebra (the Jordan canonical form) guarantees that for every $n\times n$ matrix the eigenvectors and generalized eigenvectors can be arranged in chains containing $n$ independent vectors altogether, and an eigenvalue of algebraic multiplicity $m$ always contributes exactly $m$ of them. Collecting the solutions from all chains gives a fundamental set. Moreover, solutions from different eigenvalues are automatically independent.

Although the only eigenvalue is $-1$, the solution in Example 7.7.5 does not simply decay like $e^{-t}$. The powers of $t$ produce a temporary rise: $x_2$ grows from $0$ to its maximum at $t=\sqrt{2}$ before it decays (Figure 7.7.3). Such transient growth is typical of defective matrices and matters in practice: a system can be stable in the long run and still produce a large overshoot.

A triple eigenvalue with two eigenvectors

One case needs care. If $\lambda$ has multiplicity $3$ but two independent eigenvectors, we need one chain of length $2$ and one of length $1$. The trap is that $(A-\lambda I)\mathbf{p}=\mathbf{v}$ may have no solution for a carelessly chosen eigenvector $\mathbf{v}$. The right $\mathbf{v}$ must lie in the column space of $A-\lambda I$. Since $(A-\lambda I{)}^2=0$ in this situation, any nonzero column of $A-\lambda I$ is such an eigenvector.

The solver below does all of this for $3\times 3$ matrices: characteristic polynomial, eigenvalues with multiplicities, eigenvectors, chains when they are needed, and the general solution. Enter the examples above, then try your own matrices.

Stability works as in Section 7.6. If every eigenvalue has a negative real part, all solutions tend to $0$ (the powers of $t$ from chains cannot beat a decaying exponential). If some eigenvalue has a positive real part, some solutions grow without bound. In Examples 7.7.2 and 7.7.5 the origin is asymptotically stable; in Examples 7.7.1, 7.7.3, 7.7.4 and 7.7.6 it is unstable.

A short note on the matrix exponential

For a single equation $x^{\prime }=ax$ the solution with $x(0)=x_0$ is $x=e^{at}{x}_0$. Is there a matrix version? Yes. Define, for any square matrix $A$,

One can show that this series converges for every $t$ (entry by entry) and may be differentiated term by term. Then

So each column of $e^{At}$ solves ${\mathbf{x}}^{\prime }=A\mathbf{x}$, and

is the solution with $\mathbf{x}(0)={\mathbf{x}}_0$, exactly as in the scalar case. In the language of Section 7.8, $e^{At}$ is the fundamental matrix that equals $I$ at $t=0$. Two practical ways to compute it:

1. From any fundamental matrix. If $\Phi (t)$ has $n$ independent solutions as its columns (found by the eigenvalue method), then $e^{At}=\Phi (t)\Phi (0{)}^{-1}$ (proved in Section 7.8).
2. Nilpotent shortcut. If $A=\lambda I+N$ with $N^k=0$ for some $k$, then, because $\lambda I$ and $N$ commute, $$e^{At}=e^{\lambda t}(I+tN+\frac{t^2}{2!}{N}^2+\cdots +\frac{t^{k-1}}{(k-1)!}{N}^{k-1}),$$ a finite sum. This applies whenever $A$ has a single eigenvalue $\lambda$, since then $N=A-\lambda I$ satisfies $N^n=0$ (Cayley–Hamilton).

A short note on nonhomogeneous systems

Forced systems ${\mathbf{x}}^{\prime }=A\mathbf{x}+\mathbf{f}(t)$ are handled as in Chapter 4: the general solution is the general solution of the homogeneous system plus one particular solution (Section 7.8 proves this). A particular solution can always be found by variation of parameters. Let $\Phi (t)$ be a fundamental matrix, so the homogeneous solutions are $\Phi (t)\mathbf{c}$ with a constant vector $\mathbf{c}$. Replace $\mathbf{c}$ by an unknown vector function $\mathbf{u}(t)$ and try ${\mathbf{x}}_p=\Phi \mathbf{u}$. Since ${\Phi }^{\prime }=A\Phi$,

So ${\mathbf{x}}_p$ is a solution exactly when $\Phi {\mathbf{u}}^{\prime }=\mathbf{f}$. Because $\det \Phi \ne 0$ we can solve for ${\mathbf{u}}^{\prime }$ and integrate.