Chapter 7 · Section 7.5

7.5 Matrices and Vectors

In Sections 7.1–7.4 we handled systems of two equations by hand: eliminate, or guess exponentials, and keep track of every coefficient. That works for two unknowns. For three or more it quickly becomes a bookkeeping nightmare. The cure is to collect the unknowns into a single vector and the coefficients into a single matrix. A system such as

x_{1}^{'} x_{2}^{'} x_{3}^{'} = 2 x_{1} - x_{2} + 3 x_{3}, = x_{1} + 4 x_{3}, = - x_{1} + x_{2} - 2 x_{3}

then becomes one short equation, $x^{'} = A x$ , and the whole theory of linear systems can be written, and understood, almost exactly like the theory of a single equation $x^{'} = a x$ .

This section collects the linear algebra we need for Sections 7.6–7.8. If you have taken a linear algebra course, skim it, but read the parts on eigenvalues and on vector functions carefully, because the matrix method is built on them. If you have not, everything you need is here, with examples.

Matrices: the basic vocabulary

Definition 7.5.1 (Matrix, vector)

An $m \times n$ matrix is a rectangular array of numbers with $m$ rows and $n$ columns:

A = a_{11} a_{21} ⋮ a_{m 1} a_{12} a_{22} ⋮ a_{m 2} \dots \dots \dots a_{1 n} a_{2 n} ⋮ a_{mn} .

The number $a_{ij}$ in row $i$ and column $j$ is the $(i, j)$ entry. We also write $A = (a_{ij})$ . The matrix is square if $m = n$ . An $n \times 1$ matrix is a column vector, or simply a vector; we denote vectors by bold lowercase letters, $x, v, \dots$

Two matrices are equal when they have the same size and all corresponding entries agree. So one matrix equation stands for many scalar equations at once. That is the whole point.

Some special matrices appear constantly. The zero matrix $0$ has every entry equal to $0$ . The $n \times n$ identity matrix $I$ has $1$ 's on the main diagonal (from top left to bottom right) and $0$ 's elsewhere. For example, the $3 \times 3$ identity is

I = 100010001 .

The transpose $A^{T}$ is obtained by turning rows into columns: the $(i, j)$ entry of $A^{T}$ is $a_{ji}$ . We use it mainly to save space: instead of a tall column we write $x = (x_{1}, x_{2}, x_{3})^{T}$ .

Operations with matrices

Addition and scalar multiplication work entry by entry. If $A$ and $B$ have the same size and $c$ is a number, then $A + B = (a_{ij} + b_{ij})$ and $c A = (c a_{ij})$ . These obey the familiar rules: $A + B = B + A$ , $(A + B) + C = A + (B + C)$ , $c (A + B) = c A + c B$ , and so on. Matrices of different sizes cannot be added.

Multiplication is less obvious, and it is designed so that a linear system can be written as a product.

Definition 7.5.2 (Matrix product)

Let $A$ be $m \times n$ and $B$ be $n \times p$ . The product $A B$ is the $m \times p$ matrix whose $(i, j)$ entry is

(A B)_{ij} = a_{i 1} b_{1 j} + a_{i 2} b_{2 j} + \dots + a_{in} b_{nj} = k = 1 \sum n a_{ik} b_{kj} .

In words: the $(i, j)$ entry is "row $i$ of $A$ times column $j$ of $B$ ". The product exists only when the number of columns of $A$ equals the number of rows of $B$ .

The most important special case is a square matrix times a vector. With $x = (x_{1}, \dots, x_{n})^{T}$ ,

A x = a_{11} x_{1} + a_{12} x_{2} + \dots + a_{1 n} x_{n} ⋮ a_{n 1} x_{1} + a_{n 2} x_{2} + \dots + a_{nn} x_{n} = x_{1} a_{11} ⋮ a_{n 1} + \dots + x_{n} a_{1 n} ⋮ a_{nn} .

The second form is worth remembering: $A x$ is a combination of the columns of $A$ , with weights $x_{1}, \dots, x_{n}$ . It is the key to understanding linear independence below.

Thanks to this definition, the system at the top of the page is exactly

x_{1}^{'} x_{2}^{'} x_{3}^{'} = 21 - 1 - 1 01 34 - 2 x_{1} x_{2} x_{3}, that is, x^{'} = A x .

Row $i$ of $A$ holds the coefficients of equation $i$ , column $j$ holds the coefficients of the unknown $x_{j}$ . A missing unknown means a zero entry, just as a missing derivative meant a zero coefficient in Chapter 4.

Matrix multiplication is associative, $(A B) C = A (BC)$ , and distributive, $A (B + C) = A B + A C$ . Also $A I = I A = A$ for square $A$ . But one rule of ordinary arithmetic fails:

Common mistake

In general $A B \neq = B A$ . The order of the factors matters. Also, $A B = 0$ does not imply $A = 0$ or $B = 0$ . For instance, with $N = (0010)$ we get $NN = 0$ although $N \neq = 0$ . Matrices with $N^{k} = 0$ will appear in Section 7.7.

Example 7.5.1

Let $A = (23 - 1 4)$ and $B = (1 - 2 50)$ . Compute $A + B$ , $3 A - 2 B$ , $A B$ , $B A$ and $A x$ for $x = (1, 2)^{T}$ .

Solution

Sum. Entry by entry: $A + B = (3144)$ .
Combination. $3 A = (69 - 3 12)$ , $2 B = (2 - 4 100)$ , so $3 A - 2 B = (413 - 13 12)$ .
Product $A B$ . Row times column: $A B = (2 \cdot 1 + (- 1) (- 2) 3 \cdot 1 + 4 \cdot (- 2) 2 \cdot 5 + (- 1) \cdot 0 3 \cdot 5 + 4 \cdot 0) = (4 - 5 1015) .$
Product $B A$ . $B A = (1 \cdot 2 + 5 \cdot 3 - 2 \cdot 2 + 0 \cdot 3 1 \cdot (- 1) + 5 \cdot 4 (- 2) (- 1) + 0 \cdot 4) = (17 - 4 192) .$ So $A B \neq = B A$ .
Matrix times vector. $A x = 1 (23) + 2 (- 1 4) = (011)$ .

Determinants

Every square matrix has a number attached to it, its determinant, which tells us at a glance whether the matrix can be "undone". For $2 \times 2$ and $3 \times 3$ matrices the definitions are:

det (a c b d) = a c b d = a d - b c,

a_{11} a_{21} a_{31} a_{12} a_{22} a_{32} a_{13} a_{23} a_{33} = a_{11} a_{22} a_{32} a_{23} a_{33} - a_{12} a_{21} a_{31} a_{23} a_{33} + a_{13} a_{21} a_{31} a_{22} a_{32} .

The second formula is the cofactor expansion along the first row: each entry multiplies the $2 \times 2$ determinant left after deleting its row and column, with alternating signs. The expansion can be done along any row or column, using the sign pattern

+ - + - + - + - + .

Choose the row or column with the most zeros: it saves work. The same idea defines $n \times n$ determinants recursively.

The following properties are used again and again (they hold for every $n$ ):

$det I = 1$ , and the determinant of a triangular matrix is the product of its diagonal entries.
Swapping two rows changes the sign of the determinant.
Multiplying one row by $c$ multiplies the determinant by $c$ .
Adding a multiple of one row to another row does not change the determinant.
A matrix with a zero row, or with two proportional rows, has determinant $0$ .
$det (A B) = det A \cdot det B$ and $det A^{T} = det A$ ; so everything said about rows holds for columns too.

Example 7.5.2

Evaluate $det M$ for $M = 205 14 - 2 - 3 11$ .

Solution

Choose a column. The first column contains a zero, so expand along it. The signs for column 1 are $+, -, +$ .
Expand. $det M = 2 4 - 2 11 - 0 \cdot (\dots) + 5 14 - 3 1 = 2 (4 \cdot 1 - 1 \cdot (- 2)) + 5 (1 \cdot 1 - (- 3) \cdot 4) = 2 \cdot 6 + 5 \cdot 13 = 77.$

The inverse of a matrix

For numbers, $a x = b$ is solved by dividing by $a$ , provided $a \neq = 0$ . For matrices, "dividing" means multiplying by an inverse.

Definition 7.5.3 (Inverse)

An $n \times n$ matrix $A$ is invertible (or nonsingular) if there is an $n \times n$ matrix $A^{- 1}$ with

A A^{- 1} = A^{- 1} A = I .

A square matrix with no inverse is called singular.

Theorem 7.5.1 (When does an inverse exist?)

A square matrix $A$ is invertible if and only if $det A \neq = 0$ . For a $2 \times 2$ matrix with $a d - b c \neq = 0$ ,

(a c b d)^{- 1} = \frac{1}{a d - b c} (d - c - b a) .

The $2 \times 2$ formula is easy to remember: swap the diagonal entries, change the sign of the other two, divide by the determinant. You can check it by multiplying. For larger matrices the practical method is Gauss–Jordan elimination: write $A$ and $I$ side by side as $[A ∣ I]$ , and use row operations (swap rows, multiply a row by a nonzero number, add a multiple of one row to another) until the left half becomes $I$ . The right half is then $A^{- 1}$ . If a zero row appears on the left, $A$ is singular.

Example 7.5.3

(a) Invert $(5332)$ . (b) Invert $N = 101211111$ and use the inverse to solve $N x = (3, 1, 2)^{T}$ .

Solution

(a) The determinant is $5 \cdot 2 - 3 \cdot 3 = 1$ , so the inverse is $(2 - 3 - 3 5)$ .
(b) Set up. $101211111100010001$
Clear column 1. Row 3 minus row 1: $100 21 - 1 110 10 - 1 010001$
Clear column 2. Row 1 minus $2 \times$ row 2, and row 3 plus row 2: $100010 - 1 11 10 - 1 - 2 11 001$
Clear column 3. Row 1 plus row 3, and row 2 minus row 3: $100010001 01 - 1 - 1 01 1 - 1 1 ⟹ N^{- 1} = 01 - 1 - 1 01 1 - 1 1 .$
Check. Multiplying, $N N^{- 1} = I$ . (Also $det N = 1 \neq = 0$ , consistent with Theorem 7.5.1.)
Solve. $x = N^{- 1} (3, 1, 2)^{T} = (0 - 1 + 2, 3 + 0 - 2, - 3 + 1 + 2)^{T} = (1, 1, 0)^{T}$ . Substituting back: $N (1, 1, 0)^{T} = (3, 1, 2)^{T}$ .

The next theorem is the single most useful fact from linear algebra for this chapter. Part 2 is the reason eigenvalues are computed from a determinant.

Theorem 7.5.2 (Square linear systems)

Let $A$ be an $n \times n$ matrix.

If $det A \neq = 0$ , then for every $b$ the system $A x = b$ has exactly one solution, $x = A^{- 1} b$ .
The homogeneous system $A x = 0$ has a solution $x \neq = 0$ if and only if $det A = 0$ .

Part 2 follows from part 1: if $det A \neq = 0$ , the only solution of $A x = 0$ is $A^{- 1} 0 = 0$ . If $det A = 0$ , elimination produces at least one zero row, so at least one unknown can be chosen freely, and a nonzero solution exists.

Linear independence of vectors

Definition 7.5.4 (Linear dependence and independence)

Vectors $v_{1}, \dots, v_{k}$ are linearly dependent if there are numbers $c_{1}, \dots, c_{k}$ , not all zero, with

c_{1} v_{1} + c_{2} v_{2} + \dots + c_{k} v_{k} = 0 .

Otherwise they are linearly independent: the only combination that gives $0$ is the one with $c_{1} = \dots = c_{k} = 0$ .

Dependent means that at least one vector is a combination of the others, so it brings nothing new. Two vectors are dependent exactly when one is a multiple of the other (they lie on one line through the origin).

When we have exactly $n$ vectors with $n$ components, there is a quick test. Put the vectors side by side as the columns of an $n \times n$ matrix $V = [v_{1} \dots v_{n}]$ . By the "combination of columns" view of the product, $c_{1} v_{1} + \dots + c_{n} v_{n} = V c$ with $c = (c_{1}, \dots, c_{n})^{T}$ . Part 2 of Theorem 7.5.2 now gives:

Theorem 7.5.3 (Determinant test for independence)

The $n$ vectors $v_{1}, \dots, v_{n}$ with $n$ components each are linearly independent if and only if

det [v_{1} v_{2} \dots v_{n}] \neq = 0.

Example 7.5.4

(a) Are $v_{1} = (1, 0, 2)^{T}$ , $v_{2} = (2, 1, 5)^{T}$ , $v_{3} = (- 1, 3, 1)^{T}$ independent? (b) Same question for $w_{1} = (2, 1, 0)^{T}$ , $w_{2} = (1, 1, 1)^{T}$ , $w_{3} = (0, 1, 3)^{T}$ .

Solution

(a) Determinant. Expanding along the first row, $102215 - 1 31 = 1 (1 - 15) - 2 (0 - 6) + (- 1) (0 - 2) = - 14 + 12 + 2 = 0.$ So the vectors are dependent.
Find the relation. Solve $c_{1} v_{1} + c_{2} v_{2} + c_{3} v_{3} = 0$ , i.e. $c_{1} + 2 c_{2} - c_{3} = 0$ , $c_{2} + 3 c_{3} = 0$ , $2 c_{1} + 5 c_{2} + c_{3} = 0$ . The second equation gives $c_{2} = - 3 c_{3}$ ; then the first gives $c_{1} = - 2 c_{2} + c_{3} = 7 c_{3}$ . The third is then automatic: $14 c_{3} - 15 c_{3} + c_{3} = 0$ . With $c_{3} = 1$ : $7 v_{1} - 3 v_{2} + v_{3} = 0$ .
(b) $210111013 = 2 (3 - 1) - 1 (3 - 0) + 0 = 1 \neq = 0,$ so $w_{1}, w_{2}, w_{3}$ are independent.

Eigenvalues and eigenvectors

In Section 7.4 we looked for solutions of a $2 \times 2$ system of the form $x = A e^{λ t}$ , $y = B e^{λ t}$ . In vector language this is $x = e^{λ t} v$ with a constant vector $v = (A, B)^{T}$ . Since $x^{'} = λ e^{λ t} v$ and $A x = e^{λ t} A v$ (here $A$ is the coefficient matrix), the equation $x^{'} = A x$ becomes, after cancelling $e^{λ t} \neq = 0$ ,

A v = λ v .

This little equation is the heart of the matrix method.

What it means geometrically

Multiplying by a matrix usually changes both the length and the direction of a vector. Take $A = (12 4 - 1)$ and $u = (1, 0)^{T}$ : then $A u = (1, 2)^{T}$ points in a different direction. But for a few special vectors the direction does not change; the vector is only stretched, shrunk or reversed. For this $A$ , $A (2, 1)^{T} = (6, 3)^{T} = 3 (2, 1)^{T}$ . Those special directions are the eigenvectors, and the stretching factors are the eigenvalues.

Figure 7.5.1. For

A = (12 4 - 1)

: a generic vector is turned (left), while an eigenvector stays on its own line (right).

A v_{1} = 3 v_{1}

is stretched;

A v_{2} = - 3 v_{2}

is stretched and reversed.

Definition 7.5.5 (Eigenvalue, eigenvector)

Let $A$ be an $n \times n$ matrix. A number $λ$ (real or complex) is an eigenvalue of $A$ if there is a vector $v \neq = 0$ with

A v = λ v .

Every such nonzero $v$ is an eigenvector of $A$ corresponding to $λ$ . (Older books say characteristic value and characteristic vector.)

Three remarks. First, $v = 0$ is excluded, because $A 0 = λ 0$ for every $λ$ and would tell us nothing. Second, if $v$ is an eigenvector, so is $c v$ for every $c \neq = 0$ : eigenvectors are really eigendirections, and we may scale them to have simple integer entries. Third, $λ = 0$ is allowed; it is an eigenvalue exactly when $A v = 0$ has a nonzero solution, i.e. when $det A = 0$ .

Figure 7.5.2 shows the same idea for many vectors at once. At points $x$ of the unit circle we attach the arrow $A x$ (shortened). Only along the eigenlines do the arrows point straight away from, or straight toward, the origin. In Section 7.6 these lines will become the straight-line solutions of $x^{'} = A x$ .

Figure 7.5.2. Arrows

A x

attached at points of the unit circle, for the same matrix. On the eigenline of

λ = 3

they point outward; on the eigenline of

λ = - 3

they point inward. Everywhere else they are tilted.

How to find them

Rewrite $A v = λ v$ as $A v - λ I v = 0$ , that is

(A - λ I) v = 0 .

(We need $I$ here: " $A - λ$ " makes no sense, a matrix minus a number.) This is a homogeneous linear system for $v$ . By part 2 of Theorem 7.5.2 it has a nonzero solution if and only if its determinant vanishes.

Definition 7.5.6 (Characteristic equation)

The polynomial $p (λ) = det (A - λ I)$ is the characteristic polynomial of $A$ ; it has degree $n$ . The eigenvalues of $A$ are exactly the roots of the characteristic equation

det (A - λ I) = 0.

The multiplicity of $λ$ as a root is its algebraic multiplicity. The number of linearly independent eigenvectors belonging to $λ$ is its geometric multiplicity; it is at least $1$ and at most the algebraic multiplicity.

For a $2 \times 2$ matrix the characteristic polynomial can be written down at once. With $A = (a c b d)$ ,

det (A - λ I) = (a - λ) (d - λ) - b c = λ^{2} - (a + d) λ + (a d - b c) = λ^{2} - (tr A) λ + det A,

where $tr A = a + d$ is the trace (sum of the diagonal entries). Comparing with $(λ - λ_{1}) (λ - λ_{2}) = λ^{2} - (λ_{1} + λ_{2}) λ + λ_{1} λ_{2}$ we see that the eigenvalues add up to the trace and multiply to the determinant. The same holds for $n \times n$ matrices (counting multiplicities), which gives a cheap check on your arithmetic.

Method in a nutshell

Form $A - λ I$ (subtract $λ$ from each diagonal entry) and expand $det (A - λ I)$ . For $2 \times 2$ use $λ^{2} - (tr A) λ + det A$ .
Solve $det (A - λ I) = 0$ ; check that the roots add up to $tr A$ and multiply to $det A$ .
For each eigenvalue, solve $(A - λ I) v = 0$ by elimination. At least one equation is always redundant; the free unknown(s) give the eigenvector(s).
Shortcut for $2 \times 2$ : if a row of $A - λ I$ is $(p q) \neq = (00)$ , then $v = (q, - p)^{T}$ is an eigenvector.
Check: compute $A v$ and compare with $λ v$ .

Example 7.5.5

Find the eigenvalues and eigenvectors of $A = (12 4 - 1)$ .

Solution

Characteristic equation. $tr A = 0$ and $det A = - 1 - 8 = - 9$ , so $λ^{2} - 9 = 0$ and $λ_{1} = 3$ , $λ_{2} = - 3$ . Check: $3 + (- 3) = 0 = tr A$ , $3 \cdot (- 3) = - 9 = det A$ .
Eigenvector for $λ_{1} = 3$ . $A - 3 I = (- 2 2 4 - 4) .$ Both rows say $- 2 v_{1} + 4 v_{2} = 0$ , i.e. $v_{1} = 2 v_{2}$ . Taking $v_{2} = 1$ : $v_{1} = (2, 1)^{T}$ .
Eigenvector for $λ_{2} = - 3$ . $A + 3 I = (4242),$ so $v_{1} + v_{2} = 0$ and $v_{2} = (1, - 1)^{T}$ .
Check. $A (2, 1)^{T} = (2 + 4, 4 - 1)^{T} = (6, 3)^{T} = 3 (2, 1)^{T}$ and $A (1, - 1)^{T} = (1 - 4, 2 + 1)^{T} = (- 3, 3)^{T} = - 3 (1, - 1)^{T}$ . These are the vectors of Figure 7.5.1.

Example 7.5.6

Find the eigenvalues and eigenvectors of $A = 210131012$ .

Solution

Characteristic polynomial. Expand $det (A - λ I)$ along the first row: $2 - λ 10 1 3 - λ 1 01 2 - λ = (2 - λ) [(3 - λ) (2 - λ) - 1] - 1 \cdot [1 \cdot (2 - λ) - 0] = (2 - λ) [(3 - λ) (2 - λ) - 1 - 1] = (2 - λ) (λ^{2} - 5 λ + 4) = (2 - λ) (λ - 1) (λ - 4) .$ Pulling out the common factor $(2 - λ)$ before multiplying out saved us from factoring a cubic.
Eigenvalues. $λ_{1} = 1$ , $λ_{2} = 2$ , $λ_{3} = 4$ . Check: $1 + 2 + 4 = 7 = 2 + 3 + 2$ and $1 \cdot 2 \cdot 4 = 8 = det A$ .
$λ_{1} = 1$ . $(A - I) v = 0$ reads $v_{1} + v_{2} v_{1} + 2 v_{2} + v_{3} v_{2} + v_{3} = 0, = 0, = 0.$ From the first and third, $v_{1} = - v_{2}$ and $v_{3} = - v_{2}$ ; the second is then automatic. With $v_{2} = - 1$ : $v_{1} = (1, - 1, 1)^{T}$ .
$λ_{2} = 2$ . $(A - 2 I) v = 0$ reads $v_{2} = 0$ , $v_{1} + v_{2} + v_{3} = 0$ , $v_{2} = 0$ . So $v_{2} = 0$ , $v_{3} = - v_{1}$ , and $v_{2} = (1, 0, - 1)^{T}$ .
$λ_{3} = 4$ . $(A - 4 I) v = 0$ reads $- 2 v_{1} + v_{2} = 0$ , $v_{1} - v_{2} + v_{3} = 0$ , $v_{2} - 2 v_{3} = 0$ . So $v_{2} = 2 v_{1}$ , $v_{3} = v_{1}$ , and $v_{3} = (1, 2, 1)^{T}$ .
Check one. $A (1, 2, 1)^{T} = (2 + 2, 1 + 6 + 1, 2 + 2)^{T} = (4, 8, 4)^{T} = 4 (1, 2, 1)^{T}$ .

In Example 7.5.6 the three eigenvectors are independent: the determinant of the matrix with columns $v_{1}, v_{2}, v_{3}$ equals $6 \neq = 0$ . This is no accident.

Theorem 7.5.4 (Distinct eigenvalues give independent eigenvectors)

Eigenvectors corresponding to distinct eigenvalues are linearly independent. In particular, an $n \times n$ matrix with $n$ distinct eigenvalues has $n$ linearly independent eigenvectors.

For two eigenvectors the reason is short. Suppose $A v_{1} = λ_{1} v_{1}$ , $A v_{2} = λ_{2} v_{2}$ with $λ_{1} \neq = λ_{2}$ , and $c_{1} v_{1} + c_{2} v_{2} = 0$ . Multiply by $A$ : $c_{1} λ_{1} v_{1} + c_{2} λ_{2} v_{2} = 0$ . Subtract $λ_{2}$ times the original relation: $c_{1} (λ_{1} - λ_{2}) v_{1} = 0$ . Since $λ_{1} \neq = λ_{2}$ and $v_{1} \neq = 0$ , $c_{1} = 0$ ; then $c_{2} v_{2} = 0$ gives $c_{2} = 0$ . The general case is proved in Section 7.8.

Complex and repeated eigenvalues

A real matrix can have complex eigenvalues. They come in conjugate pairs, and so do their eigenvectors: taking complex conjugates in $A v = λ v$ (with $A$ real) gives $A \overset{ˉ}{v} = \overset{ˉ}{λ} \overset{ˉ}{v}$ . So once you have the eigenvector for $α + i β$ , the one for $α - i β$ comes for free.

Example 7.5.7

Find the eigenvalues and eigenvectors of $A = (11 - 2 3)$ .

Solution

Characteristic equation. $tr A = 4$ , $det A = 3 + 2 = 5$ , so $λ^{2} - 4 λ + 5 = 0$ and $λ = \frac{4 \pm 16 - 20}{2} = 2 \pm i .$
Eigenvector for $λ = 2 + i$ . $A - (2 + i) I = (- 1 - i 1 - 2 1 - i) .$ Use the shortcut on the first row $(p, q) = (- 1 - i, - 2)$ : $v = (q, - p)^{T} = (- 2, 1 + i)^{T}$ , or, multiplying by $- 1$ , $v = (2, - 1 - i)^{T}$ .
Check with the second row (it must also be satisfied): $1 \cdot 2 + (1 - i) (- 1 - i) = 2 - (1 - i) (1 + i) = 2 - 2 = 0$ . Good.
Conjugate. For $λ = 2 - i$ an eigenvector is $\overset{ˉ}{v} = (2, - 1 + i)^{T}$ .

Example 7.5.8

Find the eigenvalues and eigenvectors of $A = (3 - 1 11)$ and compare with $B = 2 I$ .

Solution

Characteristic equation. $tr A = 4$ , $det A = 3 + 1 = 4$ : $λ^{2} - 4 λ + 4 = (λ - 2)^{2} = 0$ . So $λ = 2$ with algebraic multiplicity $2$ .
Eigenvectors. $A - 2 I = (1 - 1 1 - 1)$ , so $v_{1} + v_{2} = 0$ . Every eigenvector is a multiple of $(1, - 1)^{T}$ : the geometric multiplicity is only $1$ .
Comparison. $B = 2 I$ has the same characteristic polynomial $(λ - 2)^{2}$ , but $B - 2 I = 0$ , so every nonzero vector is an eigenvector: geometric multiplicity $2$ .

A matrix like $A$ in Example 7.5.8, with fewer independent eigenvectors than its size, is called defective. Defective matrices need the extra idea of generalized eigenvectors, which we develop in Sections 7.6 and 7.7.

Vector functions and their derivatives

The unknowns of a system depend on $t$ , so our vectors and matrices will have entries that are functions.

Definition 7.5.7 (Vector and matrix functions)

A vector function on an interval $I$ is a vector $x (t) = (x_{1} (t), \dots, x_{n} (t))^{T}$ whose components are functions defined on $I$ . It is continuous (or differentiable) on $I$ if every component is. Its derivative and integral are taken component by component:

x^{'} (t) = x_{1}^{'} (t) ⋮ x_{n}^{'} (t), \int_{a}^{b} x (t) d t = \int_{a}^{b} x_{1} (t) d t ⋮ \int_{a}^{b} x_{n} (t) d t .

Matrix functions $A (t) = (a_{ij} (t))$ are treated in the same way.

Geometrically, for $n = 2$ the point $x (t)$ traces a curve in the plane as $t$ runs, and $x^{'} (t)$ is the velocity vector, tangent to the curve (Figure 7.5.3). This picture is the phase plane of Section 7.6.

Figure 7.5.3. The vector function

x (t) = (2 cos t, sin t)^{T}

traces an ellipse. The derivative

x^{'} (t) = (- 2 sin t, cos t)^{T}

, drawn at the tip of

x (t)

, is tangent to the curve and points in the direction of motion.

Because everything is done componentwise, the usual rules of calculus carry over. For differentiable $x, y$ , a scalar function $f$ , a constant matrix $C$ and a differentiable matrix function $A (t)$ :

(x + y)^{'} (C x)^{'} = x^{'} + y^{'}, = C x^{'}, (f x)^{'} (A x)^{'} = f^{'} x + f x^{'}, = A^{'} x + A x^{'} .

In the last rule keep $A$ to the left of $x$ : order matters for matrices. A special case we use all the time: for a constant vector $v$ ,

(e^{λ t} v)^{'} = λ e^{λ t} v .

With this notation, a general linear system of $n$ first-order equations,

x_{i}^{'} = a_{i 1} (t) x_{1} + \dots + a_{in} (t) x_{n} + f_{i} (t), i = 1, \dots, n,

is written as $x^{'} = A (t) x + f (t)$ . It is homogeneous when $f = 0$ .

Example 7.5.9

(a) Differentiate $x (t) = (e^{- 2 t}, t sin t, ln (1 + t^{2}))^{T}$ and compute $\int_{0}^{1} (3 t^{2}, cos t)^{T} d t$ . (b) With $A = (12 4 - 1)$ show that $x (t) = e^{3 t} (2, 1)^{T}$ solves $x^{'} = A x$ , but $y (t) = e^{t} (1, 0)^{T}$ does not.

Solution

(a) Derivative. Componentwise, using the product rule in the second component and the chain rule in the third: $x^{'} (t) = - 2 e^{- 2 t} sin t + t cos t \frac{2 t}{1 + t ^{2}} .$
Integral. $\int_{0}^{1} 3 t^{2} d t = 1$ and $\int_{0}^{1} cos t d t = sin 1$ , so the integral is $(1, sin 1)^{T}$ .
(b) Left side. $x^{'} = 3 e^{3 t} (2, 1)^{T} = e^{3 t} (6, 3)^{T}$ .
Right side. $A x = e^{3 t} A (2, 1)^{T} = e^{3 t} (6, 3)^{T}$ (Example 7.5.5). The two sides agree for all $t$ : $x$ is a solution. This is exactly because $(2, 1)^{T}$ is an eigenvector with eigenvalue $3$ .
The other function. $y^{'} = e^{t} (1, 0)^{T}$ , while $A y = e^{t} (1, 2)^{T}$ . These differ, so $y$ is not a solution: $(1, 0)^{T}$ is not an eigenvector.

Linear dependence of vector functions

Solutions of systems are vector functions, so we need independence for them too. The definition is the same as for vectors, except that one set of constants must work for all $t$ in the interval.

Definition 7.5.8 (Dependence of vector functions)

Vector functions $x_{1} (t), \dots, x_{k} (t)$ are linearly dependent on $I$ if there are constants $c_{1}, \dots, c_{k}$ , not all zero, with $c_{1} x_{1} (t) + \dots + c_{k} x_{k} (t) = 0$ for every $t$ in $I$ . Otherwise they are linearly independent on $I$ .

Careful

Independence of vector functions is not the same as independence of their values at each fixed $t$ . Take $f_{1} (t) = (1, t)^{T}$ and $f_{2} (t) = (t, t^{2})^{T}$ . At every fixed $t$ the two vectors are parallel, since $f_{2} (t) = t f_{1} (t)$ , and $det [f_{1} f_{2}] = t^{2} - t^{2} = 0$ . Yet as functions they are independent: $c_{1} f_{1} + c_{2} f_{2} = 0$ for all $t$ means $c_{1} + c_{2} t = 0$ for all $t$ , which forces $c_{1} = c_{2} = 0$ . The multiplier $t$ is not a constant. Section 7.8 shows that this strange behaviour is impossible for solutions of a system $x^{'} = A (t) x$ .

Exercises

Let $A = (31 - 2 4)$ and $B = (0 - 1 52)$ . Compute $2 A - B$ , $A B$ and $B A$ .
Answer
$2 A - B = (63 - 9 6)$ , $A B = (2 - 4 1113)$ , $B A = (5 - 1 2010)$
Compute $A x$ for $A = 1 - 1 2 031210$ and $x = (2, - 1, 3)^{T}$ .
Answer
$(8, - 2, 3)^{T}$
Evaluate $32 - 1 104 - 2 15$ .
Answer
$- 39$
Find the inverse of $(4172)$ .
Answer
$(2 - 1 - 7 4)$
Find the inverse of $121011230$ by Gauss–Jordan elimination.
Answer
$3 - 3 - 1 - 2 21 2 - 1 - 1$ (the determinant is $- 1$ )
Show that $(1, 2, 0)^{T}$ , $(0, 1, - 1)^{T}$ , $(2, 5, - 1)^{T}$ are linearly dependent and find a dependence relation.
Answer
The determinant is $0$ ; $2 (1, 2, 0)^{T} + (0, 1, - 1)^{T} - (2, 5, - 1)^{T} = 0$
Are $(1, 1, 2)^{T}$ , $(2, - 1, 1)^{T}$ , $(0, 3, 1)^{T}$ linearly independent?
Answer
Yes: the determinant is $6 \neq = 0$ .
Find the eigenvalues and eigenvectors of $(56 - 2 - 3)$ .
Answer
$λ = 3$ : $(1, 1)^{T}$ ; $λ = - 1$ : $(1, 3)^{T}$
Find the eigenvalues and eigenvectors of $(21 - 5 - 2)$ .
Answer
$λ = \pm i$ ; for $λ = i$ : $(2 + i, 1)^{T}$ , for $λ = - i$ : $(2 - i, 1)^{T}$
Find the eigenvalues and eigenvectors of $110101011$ .
Answer
$p (λ) = - (λ - 2) (λ - 1) (λ + 1)$ ; $λ = 2$ : $(1, 1, 1)^{T}$ ; $λ = 1$ : $(1, 0, - 1)^{T}$ ; $λ = - 1$ : $(1, - 2, 1)^{T}$
Find $x^{'} (t)$ for $x (t) = (t^{2} e^{- t}, cos 3 t, 1/ (1 + t))^{T}$ , and evaluate $\int_{0}^{π /6} cos 3 t d t$ .
Answer
$x^{'} = ((2 t - t^{2}) e^{- t}, - 3 sin 3 t, - 1/ (1 + t)^{2})^{T}$ ; the integral is $\frac{1}{3}$ .
Let $A = (- 4 2 31)$ . Verify that $x (t) = e^{- 5 t} (3, - 1)^{T}$ solves $x^{'} = A x$ . What does this say about $(3, - 1)^{T}$ ? Find the other eigenvalue and an eigenvector for it.
Answer
Both sides equal $e^{- 5 t} (- 15, 5)^{T}$ , so $(3, - 1)^{T}$ is an eigenvector with eigenvalue $- 5$ . Since the trace is $- 3$ , the other eigenvalue is $2$ , with eigenvector $(1, 2)^{T}$ .