5. Determinants

The determinant assigns to every square matrix a single number that answers two questions at once: is the matrix invertible, and by what factor does the linear map $\mathbf{x} \mapsto A\mathbf{x}$ scale areas and volumes (and does it flip orientation)? This chapter defines the determinant, develops the algebraic rules that make it computable in $O(n^3)$ operations, and applies it to invertibility tests, explicit formulas for solutions and inverses, geometry, and the change of variables in multivariable integrals. Determinants reappear in Chapter 8, where the eigenvalues of $A$ are exactly the numbers $\lambda$ with $\det(A - \lambda I) = 0$.

<a id="51-definition-and-computation"></a>

5.1 Definition and Computation

This subchapter defines the determinant, first by explicit formulas for $2 \times 2$ and $3 \times 3$ matrices and then for any square matrix by cofactor expansion, and identifies triangular matrices as the case where it is trivial to compute.

Determinants of 2x2 and 3x3 Matrices

Why it matters. Section 4.3 showed that a $2 \times 2$ matrix is invertible exactly when the single number $ad - bc$ is nonzero, and Section 2.4 showed that a $3 \times 3$ array of numbers computes the signed volume spanned by three vectors. The determinant is the common generalization: one number per square matrix, which vanishes exactly when the matrix collapses space and otherwise measures how much it stretches it. Small cases come first because they can be computed by hand and their patterns explain the general definition.

The $2 \times 2$ determinant. For $A = \begin{bmatrix} a & b \ c & d \end{bmatrix}$,

$$\det A = \begin{vmatrix} a & b \ c & d \end{vmatrix} = ad - bc.$$

Vertical bars $\lvert A \rvert$ are an alternative notation for $\det A$ (not an absolute value). Two views of the same number:

• Elimination. If $a \ne 0$, subtracting $\frac{c}{a}$ times row 1 from row 2 leaves pivots $a$ and $d - \frac{bc}{a}$. Their product is $ad - bc$. The determinant is the product of the pivots, and it is zero exactly when the second pivot is missing.
• Geometry. The columns $(a, c)$ and $(b, d)$ span a parallelogram whose signed area is $ad - bc$. The sign is positive when the second column lies counterclockwise from the first, and negative otherwise.

The $3 \times 3$ determinant. For $A = [a_{ij}]$ of size $3 \times 3$,

$$\det A = a_{11}a_{22}a_{33} + a_{12}a_{23}a_{31} + a_{13}a_{21}a_{32} - a_{13}a_{22}a_{31} - a_{11}a_{23}a_{32} - a_{12}a_{21}a_{33}.$$

Geometrically, this is the signed volume of the parallelepiped spanned by the rows (or, as Section 5.2 shows, by the columns). It equals the scalar triple product of the rows from Section 2.4.

Rule of Sarrus. A mnemonic for the six terms: copy the first two columns to the right, add the products along the three down-right diagonals, and subtract the products along the three up-right diagonals.

   copy the first two columns to the right:

      a   b   c | a   b
      d   e   f | d   e
      g   h   i | g   h

   down-right diagonals (add):        a e i     b f g     c d h
   up-right diagonals (subtract):     g e c     h f a     i d b

   det = aei + bfg + cdh - ceg - afh - bdi

The pattern behind the formula. Every term is a product of $n$ entries with exactly one entry from each row and each column, and every such choice appears exactly once, with a sign $\pm 1$. For $n = 2$ there are $2! = 2$ such choices; for $n = 3$ there are $3! = 6$. This pattern defines the determinant in every size (the Leibniz formula, derived in "Multilinearity and Alternating Property"), with $n!$ terms. Sarrus's diagonal trick is only valid for $3 \times 3$: for $4 \times 4$ it produces 8 products, but the determinant has $4! = 24$ terms.

Example. For $A = \begin{bmatrix} 2 & 1 & 3 \ 0 & -1 & 4 \ 1 & 2 & 0 \end{bmatrix}$:

$$\det A = \underbrace{(2)(-1)(0) + (1)(4)(1) + (3)(0)(2)}{\text{down: } 0 + 4 + 0} - \underbrace{\big[(3)(-1)(1) + (2)(4)(2) + (1)(0)(0)\big]}{\text{up: } -3 + 16 + 0} = 4 - 13 = -9.$$

This matrix is used as a running example throughout the chapter.

import numpy as np

def det2(A):
    (a, b), (c, d) = A
    return a * d - b * c

def det3_sarrus(A):
    """Rule of Sarrus: three 'down' diagonals minus three 'up' diagonals (3 x 3 ONLY)."""
    (a, b, c), (d, e, f), (g, h, i) = A
    down = a * e * i + b * f * g + c * d * h
    up   = c * e * g + a * f * h + b * d * i
    return down - up

A = [[2, 1, 3],
     [0, -1, 4],
     [1, 2, 0]]
print(det2([[4, 7], [2, 6]]))                 # 10
print(det3_sarrus(A))                         # -9
print(round(np.linalg.det(np.array(A))))      # -9: NumPy agrees

# Signed volume: the determinant equals the scalar triple product of the rows
u, v, w = map(np.array, A)
print(u @ np.cross(v, w))                     # -9

# Sarrus does NOT extend to 4 x 4: wrapping diagonals give only 8 of the 24 terms
def fake_sarrus_4(A):
    n = 4
    down = sum(np.prod([A[i][(i + k) % n] for i in range(n)]) for k in range(n))
    up   = sum(np.prod([A[i][(k - i) % n] for i in range(n)]) for k in range(n))
    return down - up

B = [[1, 2, 0, 1], [0, 1, 3, 2], [2, 0, 1, 1], [1, 1, 0, 3]]
print(fake_sarrus_4(B), round(np.linalg.det(np.array(B))))   # 7 31: the "rule" is wrong

Key takeaways

• $\det\begin{bmatrix} a & b \ c & d \end{bmatrix} = ad - bc$: the product of the pivots, and the signed area spanned by the columns.
• A $3 \times 3$ determinant has six terms; it is the signed volume spanned by the rows, equal to the scalar triple product.
• Each term picks one entry from every row and every column; an $n \times n$ determinant has $n!$ such terms.
• The rule of Sarrus works for $3 \times 3$ matrices only.

🧪 Practice

1. Compute $\det\begin{bmatrix} 3 & -2 \ 5 & 4 \end{bmatrix}$ and $\det\begin{bmatrix} 1 & 2 & 3 \ 4 & 5 & 6 \ 7 & 8 & 10 \end{bmatrix}$. Then change the bottom-right entry of the second matrix from 10 to 9 and recompute. What happened, and why?
2. Find all real $x$ for which $\det\begin{bmatrix} x & 1 & 0 \ 1 & x & 1 \ 0 & 1 & x \end{bmatrix} = 0$.
3. Use a $2 \times 2$ determinant to compute the area of the triangle with vertices $(0, 0)$, $(4, 1)$, and $(1, 3)$. Does the sign of the determinant change if you list the last two vertices in the other order?
4. Interview: A colleague "generalizes" Sarrus's rule to $4 \times 4$ matrices by copying the first three columns and taking the 4 down-diagonals minus the 4 up-diagonals. Without computing an example, explain why this cannot be correct. Hint: count the terms of the true $4 \times 4$ determinant, and ask whether the term $a_{12}a_{21}a_{33}a_{44}$ lies on any wrapped diagonal.

Minors and Cofactors

Why it matters. The explicit formula with $n!$ terms is unusable beyond tiny matrices. Minors and cofactors break an $n \times n$ determinant into determinants of size $(n-1) \times (n-1)$, which is the basis of the recursive definition (cofactor expansion), the explicit inverse formula (the adjugate), and Cramer's rule. Cofactors also measure sensitivity: $C_{ij}$ is exactly how fast $\det A$ changes when the entry $a_{ij}$ changes.

Definitions. Let $A$ be an $n \times n$ matrix with $n \ge 2$.

• The submatrix $A_{ij}$ is the $(n-1) \times (n-1)$ matrix obtained by deleting row $i$ and column $j$ of $A$.
• The $(i, j)$ minor is $M_{ij} = \det A_{ij}$.
• The $(i, j)$ cofactor is $C_{ij} = (-1)^{i+j}M_{ij}$.

The sign $(-1)^{i+j}$ follows a checkerboard pattern that starts with $+$ in the top-left corner:

   [ +  -  +  - ]
   [ -  +  -  + ]        sign of C_ij = (-1)^(i+j)
   [ +  -  +  - ]        + on the main diagonal, alternating in every row and column
   [ -  +  -  + ]

Example. For the running example $A = \begin{bmatrix} 2 & 1 & 3 \ 0 & -1 & 4 \ 1 & 2 & 0 \end{bmatrix}$, the $(1, 2)$ minor and cofactor are:

          column 2 deleted
                v
    [  2  |   1  |  3 ]   <- row 1 deleted
    [  0  |  -1  |  4 ]          =>   A_12 = [ 0  4 ]     M_12 = 0*0 - 4*1 = -4
    [  1  |   2  |  0 ]                      [ 1  0 ]     C_12 = (-1)^(1+2) * (-4) = 4

Repeating this for all nine positions gives the cofactor matrix $C = [C_{ij}]$:

$$C = \begin{bmatrix} -8 & 4 & 1 \ 6 & -3 & -3 \ 7 & -8 & -2 \end{bmatrix}.$$

A striking fact, made precise in the next topic: multiplying any row of $A$ entrywise by the same row of $C$ and adding gives $\det A$. Row 1: $2(-8) + 1(4) + 3(1) = -9$. Row 2: $0(6) + (-1)(-3) + 4(-3) = -9$.

Principal minors. More generally, a $k \times k$ minor is the determinant of any $k \times k$ submatrix formed by choosing $k$ rows and $k$ columns. When the chosen rows and columns have the same indices, it is a principal minor; when they are $1, \dots, k$, it is the $k$-th leading principal minor, the determinant of the top-left $k \times k$ corner. Leading principal minors decide whether elimination can proceed without row swaps (Section 4.4) and whether a symmetric matrix is positive definite (Section 10.3).

import numpy as np

def minor(A, i, j):
    """M_ij: determinant of A with row i and column j deleted (0-based i, j)."""
    sub = np.delete(np.delete(A, i, axis=0), j, axis=1)
    return round(np.linalg.det(sub))

def cofactor(A, i, j):
    return (-1) ** (i + j) * minor(A, i, j)          # 0-based indices have the same parity as 1-based

def cofactor_matrix(A):
    n = A.shape[0]
    return np.array([[cofactor(A, i, j) for j in range(n)] for i in range(n)])

A = np.array([[2, 1, 3],
              [0, -1, 4],
              [1, 2, 0]])

print(minor(A, 0, 1), cofactor(A, 0, 1))            # -4 4: M_12 = -4, C_12 = +4
print(cofactor_matrix(A))
# [[-8  4  1]
#  [ 6 -3 -3]
#  [ 7 -8 -2]]

# Every row of A, paired with its own row of cofactors, gives the same number
C = cofactor_matrix(A)
print([int(A[i] @ C[i]) for i in range(3)])         # [-9, -9, -9] = det A

# Leading principal minors: determinants of the top-left k x k corners
T = np.array([[2, -1, 0], [-1, 2, -1], [0, -1, 2]])
print([round(np.linalg.det(T[:k, :k])) for k in (1, 2, 3)])   # [2, 3, 4]

Key takeaways

• The minor $M_{ij}$ is the determinant of $A$ with row $i$ and column $j$ deleted.
• The cofactor is the signed minor $C_{ij} = (-1)^{i+j}M_{ij}$; the signs form a checkerboard starting with $+$.
• $C_{ij}$ does not involve any entry of row $i$ or column $j$.
• Leading principal minors are determinants of top-left corners; they reappear in LU factorization and definiteness tests.

🧪 Practice

1. For $B = \begin{bmatrix} 1 & 0 & 2 \ 3 & -1 & 1 \ 2 & 4 & 0 \end{bmatrix}$, compute $M_{23}$, $C_{23}$, $M_{32}$, and $C_{32}$.
2. Write the checkerboard sign pattern of a $5 \times 5$ matrix and give the sign of $C_{34}$, $C_{41}$, and $C_{55}$ without drawing it.
3. Compute the three leading principal minors of $\begin{bmatrix} 1 & 2 & 0 \ 2 & 5 & 3 \ 0 & 3 & 10 \end{bmatrix}$, then run Gaussian elimination without row swaps. How are the pivots related to the minors?
4. Interview: A single entry $a_{ij}$ of a large matrix is measured with error $\varepsilon$. By how much can $\det A$ change, to first order? Express the answer using one number you could precompute for every entry. Hint: expand the determinant along row $i$ and note which terms involve $a_{ij}$; the result shows $\partial \det A / \partial a_{ij} = C_{ij}$.

Cofactor Expansion

Why it matters. Cofactor expansion (also called Laplace expansion) defines the determinant of any square matrix by reducing it to smaller determinants. It is the standard theoretical definition in many textbooks, the tool behind most proofs in this chapter, and a practical method for small or sparse matrices, where a row full of zeros makes most of the terms vanish.

Definition (recursive). For a $1 \times 1$ matrix, $\det[a] = a$. For $n \ge 2$, expand along the first row:

$$\det A = a_{11}C_{11} + a_{12}C_{12} + \cdots + a_{1n}C_{1n} = \sum_{j=1}^{n} (-1)^{1+j}a_{1j}M_{1j}.$$

Each $M_{1j}$ is an $(n-1) \times (n-1)$ determinant, which is defined by the same rule, until the matrices are $1 \times 1$. For $n = 2$ this gives $a(d) - b(c)$, and for $n = 3$ it reproduces the six-term formula.

Theorem (Laplace expansion). The determinant can be computed by expanding along any row or any column:

$$\det A = \sum_{j=1}^{n} a_{ij}C_{ij} \quad \text{(row } i\text{)}, \qquad \det A = \sum_{i=1}^{n} a_{ij}C_{ij} \quad \text{(column } j\text{)}.$$

The proof uses the characterization of the determinant in Section 5.2: each expansion is linear in every row, vanishes when two rows are equal, and gives 1 on the identity, and only one function has these three properties.

Strategy: expand along the sparsest line. Since any row or column works, choose the one with the most zeros, because each zero entry kills its whole term.

Example.

$$A = \begin{bmatrix} 1 & 0 & 2 & 0 \ 3 & 4 & 0 & 1 \ 0 & 0 & 5 & 0 \ 2 & 1 & 0 & 3 \end{bmatrix}.$$

Row 3 has a single nonzero entry, $a_{33} = 5$, with sign $(-1)^{3+3} = +$:

$$\det A = 5 \cdot \det\begin{bmatrix} 1 & 0 & 0 \ 3 & 4 & 1 \ 2 & 1 & 3 \end{bmatrix} = 5 \cdot 1 \cdot \det\begin{bmatrix} 4 & 1 \ 1 & 3 \end{bmatrix} = 5 \cdot (12 - 1) = 55,$$

where the $3 \times 3$ determinant was expanded along its first row, which also has a single nonzero entry.

Cost. For a dense matrix, expansion creates $n$ subproblems of size $n - 1$, each of which creates $n - 1$ subproblems, and so on: the work grows like $n!$. Gaussian elimination (Section 5.2) needs about $\tfrac{2}{3}n^3$ operations.

$n$	Terms in cofactor expansion ($n!$)	Elimination ($\approx \tfrac{2}{3}n^3$)
5	120	83
10	3,628,800	667
20	$2.4 \times 10^{18}$ (about 77 years at $10^9$ per second)	5,333

import math

calls = 0

def det_cofactor(A):
    """Laplace expansion along the row with the most zeros. A is a list of lists."""
    global calls
    calls += 1
    n = len(A)
    if n == 1:
        return A[0][0]
    i = max(range(n), key=lambda r: A[r].count(0))           # cheapest row to expand along
    total = 0
    for j in range(n):
        if A[i][j] == 0:
            continue                                         # zero entries contribute nothing
        sub = [row[:j] + row[j + 1:] for k, row in enumerate(A) if k != i]
        total += (-1) ** (i + j) * A[i][j] * det_cofactor(sub)
    return total

A = [[1, 0, 2, 0],
     [3, 4, 0, 1],
     [0, 0, 5, 0],
     [2, 1, 0, 3]]
print(det_cofactor(A), calls)              # 55 5: only five recursive calls thanks to the zeros

# A dense matrix has no zeros to skip: the number of calls grows faster than n!
for n in (4, 6, 8):
    calls = 0
    D = [[(i * 7 + j * 3) % 5 + 1 + (i == j) for j in range(n)] for i in range(n)]
    det_cofactor(D)
    print(n, calls, math.factorial(n))
# 4 41 24
# 6 1237 720
# 8 69281 40320

Key takeaways

• $\det A = \sum_j a_{ij}C_{ij}$ for any fixed row $i$, and $\det A = \sum_i a_{ij}C_{ij}$ for any fixed column $j$.
• Expand along the row or column with the most zeros; mixing in row operations to create zeros first is even better.
• A matrix with a zero row or zero column has determinant 0.
• Cofactor expansion costs about $n!$ operations on dense matrices: use it for proofs, small matrices, and sparse patterns, not for general computation.

🧪 Practice

1. Compute $\det\begin{bmatrix} 2 & 0 & 1 & 3 \ 1 & 0 & 0 & 2 \ 4 & 5 & 1 & -1 \ 0 & 0 & 3 & 1 \end{bmatrix}$ by choosing the best column to expand along.
2. Expand a general $3 \times 3$ determinant along its second column and check that you obtain the same six terms as the expansion along the first row.
3. Let $D_n$ be the determinant of the $n \times n$ tridiagonal matrix with 2 on the diagonal and $-1$ directly above and below it. Expand along the first row to show $D_n = 2D_{n-1} - D_{n-2}$, and deduce that $D_n = n + 1$.
4. Interview: Your recursive determinant function works for $8 \times 8$ matrices but appears to hang on $13 \times 13$ inputs. Explain why, give the recurrence for the running time, and propose a fix. Hint: $T(n) = n,T(n-1) + O(n^2)$ (the $O(n^2)$ is building the submatrices); compare $13!$ with $13^3$.

Determinants of Triangular Matrices

Why it matters. For triangular matrices, the determinant needs no expansion at all: it is the product of the diagonal entries. This is the fastest case, and it is the endpoint of every efficient algorithm, because elimination turns any matrix into a triangular one while keeping track of how the determinant changes.

Theorem. If $A$ is an $n \times n$ upper or lower triangular matrix, then

$$\det A = a_{11}a_{22}\cdots a_{nn}.$$

Proof (upper triangular). Expand along the first column. Its only possibly nonzero entry is $a_{11}$, and deleting row 1 and column 1 leaves an upper triangular matrix of size $n - 1$. By induction its determinant is $a_{22}\cdots a_{nn}$. For lower triangular matrices, expand along the first row instead. $\blacksquare$

   [ u11 u12 u13 u14 ]
   [  0  u22 u23 u24 ]    expand down column 1: only u11 survives
   [  0   0  u33 u34 ]
   [  0   0   0  u44 ]

   det U = u11 * det [ u22 u23 u24 ]  =  u11 * u22 * det [ u33 u34 ]  =  u11 * u22 * u33 * u44
                     [  0  u33 u34 ]                     [  0  u44 ]
                     [  0   0  u44 ]

Special cases.

Matrix	Determinant
Identity $I_n$	1
Diagonal $\operatorname{diag}(d_1, \dots, d_n)$	$d_1 d_2 \cdots d_n$
Scalar matrix $cI_n$	$c^n$
Unit triangular (1s on the diagonal), e.g. $L$ in $A = LU$	1
Triangular with a zero on the diagonal	0

The last row of the table matches Section 4.2: a triangular matrix is invertible exactly when its diagonal entries are all nonzero.

Block triangular matrices. The same idea works with square diagonal blocks: if $A$ and $D$ are square,

$$\det\begin{bmatrix} A & B \ O & D \end{bmatrix} = \det A \cdot \det D.$$

The off-diagonal block $B$ does not matter. Combined with the block LDU factorization of Section 4.5 and the product rule of Section 5.2, this gives $\det\begin{bmatrix} A & B \ C & D \end{bmatrix} = \det A \cdot \det(D - CA^{-1}B)$ when $A$ is invertible.

Numerical caution. A product of many numbers overflows or underflows easily. A $500 \times 500$ triangular matrix with every diagonal entry $0.1$ has determinant $10^{-500}$, which is below the smallest positive double (about $10^{-308}$) and is reported as exactly 0. Libraries provide slogdet, which returns the sign and $\log\lvert\det A\rvert = \sum_i \log\lvert a_{ii}\rvert$ instead.

import numpy as np

U = np.array([[2., 5, -1,  7],
              [0, -3,  4,  0],
              [0,  0, 0.5, 9],
              [0,  0,  0,  4]])
print(np.prod(np.diag(U)))                   # -12.0: product of the diagonal
print(round(np.linalg.det(U), 10))           # -12.0

# Block upper triangular: det [[A, B], [O, D]] = det A * det D
A = np.array([[1., 2], [3, 4]])              # det A = -2
D = np.array([[2., 1], [1, 1]])              # det D = 1
B = np.array([[7., -3], [0, 5]])             # B has no effect on the determinant
M = np.block([[A, B], [np.zeros((2, 2)), D]])
print(round(np.linalg.det(M), 10))           # -2.0

# Many small diagonal entries: the true determinant underflows to 0.0
T = np.triu(np.full((500, 500), 0.1))        # upper triangular, diagonal entries all 0.1
print(np.linalg.det(T))                      # 0.0: 0.1**500 = 1e-500 is below the float range
sign, logdet = np.linalg.slogdet(T)
print(sign, round(logdet / np.log(10), 6))   # 1.0 -500.0: det = +10^(-500), not 0

Key takeaways

• The determinant of a triangular (or diagonal) matrix is the product of its diagonal entries.
• $\det I = 1$, $\det(cI_n) = c^n$, and unit triangular matrices have determinant 1.
• Block triangular matrices: $\det\begin{bmatrix} A & B \ O & D \end{bmatrix} = \det A \det D$.
• For large matrices, work with $\log\lvert\det\rvert$ (slogdet) to avoid overflow and underflow.

🧪 Practice

1. Compute $\det\begin{bmatrix} 2 & 5 & -1 & 7 \ 0 & -3 & 4 & 0 \ 0 & 0 & \tfrac12 & 9 \ 0 & 0 & 0 & 4 \end{bmatrix}$ and the determinant of its transpose.
2. Let $A = \begin{bmatrix} 3 & 1 & 4 \ 0 & -2 & 5 \ 0 & 0 & 7 \end{bmatrix}$. For which values of $\lambda$ is $A - \lambda I$ singular? (This previews eigenvalues of triangular matrices, Chapter 8.)
3. Prove that the determinant of a lower triangular matrix is the product of its diagonal entries by expanding along the first row and using induction.
4. Interview: A Gaussian log-likelihood needs $\log\det\Sigma$ for a $10{,}000 \times 10{,}000$ covariance matrix $\Sigma$. np.log(np.linalg.det(Sigma)) returns -inf. What went wrong, and how would you compute it reliably and cheaply? Hint: factor $\Sigma = LL^T$ (Cholesky, Section 10.3); $L$ is triangular, so $\log\det\Sigma$ is a sum of logarithms.

<a id="52-properties-of-determinants"></a>

5.2 Properties of Determinants

This subchapter establishes the rules that govern determinants: how they react to row operations, products, transposes, and inverses, and the three defining properties from which all the others follow. Together these rules turn the $n!$-term definition into an $O(n^3)$ algorithm.

Effect of Row Operations

Why it matters. Gaussian elimination already reduces any matrix to triangular form, and triangular determinants are trivial. If the effect of each row operation on the determinant is known, the determinant can be computed by elimination in polynomial time. These three rules are also the most frequently used facts about determinants in proofs.

Theorem. Let $A$ be a square matrix.

Row operation	Effect on the determinant	Elementary matrix $E$	$\det E$
Replacement: $R_i \leftarrow R_i + kR_j$ ($i \ne j$)	Unchanged	Identity plus $k$ in position $(i, j)$	1
Interchange: $R_i \leftrightarrow R_j$	Sign flips	Identity with rows $i$, $j$ swapped	$-1$
Scaling: $R_i \leftarrow cR_i$	Multiplied by $c$	Identity with $c$ in position $(i, i)$	$c$

In every case, $\det(EA) = \det E \cdot \det A$.

Proof.

• Scaling. Expand along row $i$. Every term $c,a_{ij}C_{ij}$ picks up the factor $c$, and the cofactors $C_{ij}$ do not involve row $i$.
• Interchange. For $n = 2$, $\begin{vmatrix} c & d \ a & b \end{vmatrix} = cb - da = -(ad - bc)$. For $n > 2$, expand along a row $k$ not involved in the swap. Each minor $M_{kj}$ is a determinant of size $n - 1$ with two of its rows swapped, so by induction it changes sign, while the entries $a_{kj}$ and the signs $(-1)^{k+j}$ are unchanged.
• Two equal rows give 0. Swapping the two equal rows leaves the matrix unchanged but flips the sign of the determinant, so $\det A = -\det A$, and $\det A = 0$.
• Replacement. Expand along row $i$. The new row is $\mathbf{r}i + k\mathbf{r}j$, so the expansion splits as $\sum_l (a{il} + ka{jl})C_{il} = \det A + k\det A'$, where $A'$ is $A$ with row $i$ replaced by a copy of row $j$. $A'$ has two equal rows, so $\det A' = 0$. $\blacksquare$

Geometric meaning. View the rows as edges of a parallelogram (or parallelepiped):

   replacement = shear: slide the top edge along r1

        r2  ______                         r2 + k r1  ______
           |      |                                  /      /
           |      |     R2 <- R2 + k R1             /      /
           |______|     -------------->            /______/
           O   r1                                  O   r1

     base |r1|, height h                  same base, same height h  =>  same area

• A shear keeps the base and the height, so the area is unchanged.
• A swap keeps the shape but reverses the orientation (counterclockwise becomes clockwise), so the sign flips.
• Scaling one edge by $c$ scales the area by $c$.

Common pitfall. Multiplying the whole matrix by $c$ scales every row, so

$$\det(cA) = c^n \det A \quad \text{for } A \text{ of size } n \times n,$$

not $c\det A$. Doubling every side of a cube multiplies its volume by $2^3 = 8$.

Column operations. The same three rules hold for column operations, because $\det A^T = \det A$ ("Determinant of Transpose and Inverse"). In hand calculations, rows and columns can be mixed freely.

import numpy as np

det = lambda M: round(np.linalg.det(M), 6)

A = np.array([[2., 1, 3],
              [0, -1, 4],
              [1, 2, 0]])
print(det(A))                                   # -9.0

R = A.copy(); R[2] += -0.5 * R[0]               # replacement: R3 <- R3 - (1/2) R1
print(det(R))                                   # -9.0: unchanged

S = A[[1, 0, 2]]                                # interchange: swap R1 and R2
print(det(S))                                   # 9.0: sign flips

K = A.copy(); K[1] *= 5                         # scaling: R2 <- 5 R2
print(det(K))                                   # -45.0: multiplied by 5

print(det(2 * A))                               # -72.0 = 2^3 * (-9), NOT 2 * (-9)

# Elementary matrices: det E is 1, -1, or c, and det(EA) = det(E) det(A)
E_rep  = np.eye(3); E_rep[2, 0] = -0.5
E_swap = np.eye(3)[[1, 0, 2]]
E_sc   = np.diag([1., 5, 1])
for E in (E_rep, E_swap, E_sc):
    print(det(E), det(E @ A) == det(E) * det(A))   # 1.0 True / -1.0 True / 5.0 True

# Column operations have the same effects
C = A.copy(); C[:, [0, 2]] = C[:, [2, 0]]      # swap columns 1 and 3
print(det(C))                                   # 9.0

Key takeaways

• Replacement leaves $\det$ unchanged, interchange flips its sign, and scaling a row by $c$ multiplies it by $c$.
• Elementary matrices have determinants 1, $-1$, and $c$, and $\det(EA) = \det E \det A$.
• A matrix with two equal (or proportional) rows has determinant 0.
• $\det(cA) = c^n\det A$, because scaling the matrix scales all $n$ rows.
• The same rules hold for column operations.

🧪 Practice

1. A $3 \times 3$ matrix $A$ with rows $\mathbf{r}_1, \mathbf{r}_2, \mathbf{r}_3$ has $\det A = 5$. Find the determinants of the matrices with rows $(\mathbf{r}_2, \mathbf{r}_1, \mathbf{r}_3)$, $(\mathbf{r}_1, 3\mathbf{r}_2, \mathbf{r}_3)$, $(\mathbf{r}_1, \mathbf{r}_2 + 4\mathbf{r}_1, \mathbf{r}_3)$, and $(\mathbf{r}_3, \mathbf{r}_1, \mathbf{r}_2)$.
2. Without expanding, explain why $\det\begin{bmatrix} 1 & 2 & 3 \ 4 & 5 & 6 \ 7 & 8 & 9 \end{bmatrix} = 0$. Suggestion: look for a row replacement that creates a zero row.
3. Prove that if one row of $A$ is a linear combination of the other rows, then $\det A = 0$.
4. Interview: A $3 \times 3$ matrix has $\det A = 2$. A colleague claims $\det(A + A) = 4$. Find the error, give the correct value, and state the general rule for an $n \times n$ matrix. Hint: $A + A = 2A$ scales each of the three rows.

Determinant of a Product

Why it matters. Matrix multiplication composes linear maps. If $B$ scales volumes by a factor $\det B$ and $A$ scales them by $\det A$, then doing $B$ first and $A$ second should scale them by the product. This multiplicative rule is one of the most useful facts in linear algebra: it shows that similar matrices share a determinant, gives the determinant of an inverse, and lets factorizations such as $PA = LU$ turn determinants into products of diagonal entries.

Theorem. For $n \times n$ matrices $A$ and $B$,

$$\det(AB) = \det A \cdot \det B.$$

Proof. If $A$ is singular, then $AB$ is singular as well (if $AB$ had an inverse $X$, then $A(BX) = I$ and $A$ would be invertible by the Invertible Matrix Theorem). Both sides are 0, using the invertibility criterion of Section 5.3. If $A$ is invertible, write it as a product of elementary matrices, $A = E_1E_2\cdots E_k$ (Section 4.4). Applying $\det(EM) = \det E \det M$ repeatedly:

$$\det(AB) = \det(E_1)\det(E_2 \cdots E_kB) = \cdots = \det E_1 \cdots \det E_k \cdot \det B = \det A \cdot \det B,$$

where the last step applies the same reasoning with $B = I$. $\blacksquare$

   unit cube --- B ---> parallelepiped --- A ---> parallelepiped
   volume 1             volume |det B|            volume |det A| * |det B|  =  |det(AB)|

Consequences.

Statement	Reason
$\det(A^k) = (\det A)^k$	Apply the product rule $k$ times
$\det(AB) = \det(BA)$, even when $AB \ne BA$	Both equal $\det A \det B$
$\det(P^{-1}AP) = \det A$	Similar matrices have equal determinants (Section 8.2)
$\det(PA) = \pm\det A$ for a permutation matrix $P$	$\det P = \pm 1$
No real $3 \times 3$ matrix satisfies $A^2 = -I$	$(\det A)^2 \ge 0$, but $\det(-I_3) = -1$

What does not hold. The determinant is multiplicative but not additive: in general $\det(A + B) \ne \det A + \det B$. For $A = B = I_2$: $\det(2I_2) = 4$, while $\det I_2 + \det I_2 = 2$.

Non-square factors. If $X$ is $m \times n$ and $Y$ is $n \times m$ with $m \ne n$, then $\det X$ and $\det Y$ are undefined, and the rule has no direct analogue. When $m < n$, $\det(YX) = 0$ always (the $n \times n$ matrix $YX$ has rank at most $m$), while $\det(XY)$ can be anything. The correct generalization is the Cauchy-Binet formula, which expresses $\det(XY)$ as a sum of products of $m \times m$ minors.

import numpy as np

det = lambda M: round(np.linalg.det(M), 6)
rng = np.random.default_rng(0)
A = rng.integers(-3, 4, (3, 3)).astype(float)
B = rng.integers(-3, 4, (3, 3)).astype(float)

print(det(A), det(B), det(A @ B), det(A) * det(B))     # -9.0 -6.0 54.0 54.0
print(np.allclose(A @ B, B @ A), det(A @ B) == det(B @ A))   # False True

# Powers and similar matrices
P = np.array([[1., 2, 0], [0, 1, 1], [1, 0, 1]])       # invertible (det 3)
print(det(np.linalg.matrix_power(A, 3)), det(A) ** 3)  # -729.0 -729.0
print(det(np.linalg.inv(P) @ A @ P), det(A))           # -9.0 -9.0: similar matrices

# The determinant is NOT additive
I = np.eye(2)
print(det(I + I), det(I) + det(I))                      # 4.0 2.0

# Non-square factors: det(XY) can be nonzero while det(YX) is always 0
X = np.array([[1., 0, 2], [0, 1, 1]])                   # 2 x 3
Y = np.array([[1., 0], [0, 1], [1, 1]])                 # 3 x 2
print(det(X @ Y), det(Y @ X))                           # 4.0 0.0

Key takeaways

• $\det(AB) = \det A \det B$ for square matrices of the same size.
• Volume scaling factors multiply under composition of linear maps.
• $\det(AB) = \det(BA)$ and similar matrices share a determinant, although matrix multiplication is not commutative.
• $\det(A + B)$ has no simple formula in terms of $\det A$ and $\det B$.

🧪 Practice

1. Let $A$ and $B$ be $4 \times 4$ with $\det A = 3$ and $\det B = -2$. Compute $\det(AB)$, $\det(A^2B^3)$, $\det(2AB)$, and $\det(B^TA)$ (the last one using the next topic).
2. Find $2 \times 2$ matrices $A$ and $B$ with $AB \ne BA$, and verify that $\det(AB) = \det(BA)$.
3. Prove that no real $3 \times 3$ matrix satisfies $A^2 = -I$, then find a real $2 \times 2$ matrix that does. What does your $2 \times 2$ example do geometrically?
4. Interview: Let $X$ be $2 \times 3$ and $Y$ be $3 \times 2$. Is $\det(XY) = \det(YX)$? Prove that one of them is always 0, and give an example where the other is not. Hint: the rank of a product is at most the rank of each factor, and a $3 \times 3$ matrix of rank 2 is singular.

Determinant of Transpose and Inverse

Why it matters. The transpose rule says that rows and columns play symmetric roles: every statement about rows (row operations, expansion along rows, linear dependence of rows) holds equally for columns. The inverse rule says that undoing a map undoes its volume scaling. Together they pin down the determinants of orthogonal matrices, which are always $\pm 1$ and distinguish rotations from reflections.

Theorem (transpose). For every square matrix $A$, $\det A^T = \det A$.

Proof. If $A$ is singular, so is $A^T$ (Invertible Matrix Theorem, statement (k)), and both determinants are 0. Otherwise $A = E_1 \cdots E_k$ with elementary $E_i$, so $A^T = E_k^T \cdots E_1^T$. The transpose of an elementary matrix is an elementary matrix of the same type with the same determinant: a replacement matrix transposes to a replacement matrix, while interchange and scaling matrices are symmetric. By the product rule,

$$\det A^T = \det E_k^T \cdots \det E_1^T = \det E_1 \cdots \det E_k = \det A. \quad \blacksquare$$

Theorem (inverse). If $A$ is invertible, then $\det A \ne 0$ and

$$\det(A^{-1}) = \frac{1}{\det A}.$$

Proof. $AA^{-1} = I$ gives $\det A \cdot \det A^{-1} = \det I = 1$. $\blacksquare$

Geometrically: if $A$ scales volumes by 5, then $A^{-1}$ must scale them by $\tfrac15$.

Consequences.

Matrix type	Defining property	Determinant
Orthogonal $Q$	$Q^TQ = I$	$\pm 1$: $+1$ for rotations, $-1$ for reflections
Permutation $P$	Rows of $I$ reordered	$\pm 1$: the sign of the permutation
Skew-symmetric $K$, $n$ odd	$K^T = -K$	0, since $\det K = \det(-K) = (-1)^n\det K$
Integer $A$ with integer inverse	entries of $A$, $A^{-1}$ in $\mathbb{Z}$	$\pm 1$, since $\det A \cdot \det A^{-1} = 1$ with both integers
Complex $A$	conjugate transpose $A^*$	$\det A^* = \overline{\det A}$

import numpy as np

det = lambda M: round(np.linalg.det(M), 6)
A = np.array([[2., 1, 3],
              [0, -1, 4],
              [1, 2, 0]])

print(det(A), det(A.T))                         # -9.0 -9.0
print(det(np.linalg.inv(A)), round(1 / -9, 6))  # -0.111111 -0.111111

# Orthogonal matrices: rotations have det +1, reflections det -1
t = np.pi / 6
Rot = np.array([[np.cos(t), -np.sin(t)], [np.sin(t), np.cos(t)]])
Ref = np.array([[1., 0], [0, -1]])              # mirror across the x-axis
print(det(Rot), det(Ref))                       # 1.0 -1.0
print(np.allclose(Rot.T @ Rot, np.eye(2)), np.allclose(Ref.T @ Ref, np.eye(2)))   # True True

# Odd-sized skew-symmetric matrices are always singular
K = np.array([[0., 2, -1], [-2, 0, 4], [1, -4, 0]])
print(np.allclose(K.T, -K), det(K))             # True 0.0

# Complex matrices: det of the conjugate transpose is the conjugate of det
Z = np.array([[1 + 2j, 3], [1j, 2]])
print(np.round(np.linalg.det(Z), 6), np.round(np.linalg.det(Z.conj().T), 6))   # (2+1j) (2-1j)

Key takeaways

• $\det A^T = \det A$, so every row rule is also a column rule.
• $\det A^{-1} = 1/\det A$ for invertible $A$.
• Orthogonal matrices have determinant $\pm 1$: rotations $+1$, reflections $-1$.
• Skew-symmetric matrices of odd size are singular.

🧪 Practice

1. Let $A$ be $3 \times 3$ with $\det A = 4$. Compute $\det A^T$, $\det A^{-1}$, $\det(2A^{-1})$, and $\det(A^TA^{-1}A^T)$.
2. Prove that every $n \times n$ skew-symmetric matrix with $n$ odd is singular. Find a $2 \times 2$ skew-symmetric matrix that is invertible.
3. Prove that if $A$ has integer entries and $A^{-1}$ also has integer entries, then $\det A = \pm 1$.
4. Interview: A 3D graphics engine stores each object's orientation as a $3 \times 3$ orthogonal matrix. After a bug, some objects render "inside out", with their surfaces mirrored. What single-number check detects the bad matrices, and why can it only take two values? Hint: apply the product and transpose rules to $Q^TQ = I$; a mirror image corresponds to one of the two values.

Multilinearity and Alternating Property

Why it matters. So far the determinant has been defined by a recursive formula and its properties checked one at a time. A deeper view explains why the formula looks the way it does: the determinant is the only function of the rows that is linear in each row separately, vanishes when two rows coincide, and equals 1 on the identity. These three properties are exactly what a "signed volume" must satisfy. This characterization proves Laplace expansion along any row, yields the explicit Leibniz formula, and generalizes to the wedge product of exterior algebra (Section 13.1).

The three defining properties. Write $D(\mathbf{r}_1, \dots, \mathbf{r}_n)$ for a function of the $n$ rows of an $n \times n$ matrix.

1. Multilinear: $D$ is linear in each row when the other rows are held fixed: $$D(\dots, \mathbf{u} + c\mathbf{v}, \dots) = D(\dots, \mathbf{u}, \dots) + c,D(\dots, \mathbf{v}, \dots).$$
2. Alternating: if two rows are equal, then $D = 0$.
3. Normalized: $D(\mathbf{e}_1, \dots, \mathbf{e}_n) = \det I = 1$.

Properties 1 and 2 imply that swapping two rows flips the sign: expand $D(\dots, \mathbf{u} + \mathbf{v}, \dots, \mathbf{u} + \mathbf{v}, \dots) = 0$ by multilinearity in both slots; the two terms with a repeated row vanish, leaving $D(\dots, \mathbf{u}, \dots, \mathbf{v}, \dots) + D(\dots, \mathbf{v}, \dots, \mathbf{u}, \dots) = 0$.

Multilinear is not linear. Linearity in each row separately is very different from linearity in the whole matrix. Adding two matrices adds all rows at once, which is why $\det(A + B) \ne \det A + \det B$ and $\det(cA) = c^n\det A$.

Deriving the formula from the properties. For a $2 \times 2$ matrix, write the rows in the standard basis, $\mathbf{r}_1 = a\mathbf{e}_1 + b\mathbf{e}_2$ and $\mathbf{r}_2 = c\mathbf{e}_1 + d\mathbf{e}_2$, and expand using only the three properties:

   D(r1, r2) = D(a e1 + b e2,  c e1 + d e2)

             = ac D(e1, e1)  +  ad D(e1, e2)  +  bc D(e2, e1)  +  bd D(e2, e2)     multilinearity
                    0              +1                -1                0           alternating, normalized

             = ad - bc

The same expansion for $n \times n$ matrices produces one term for each way of choosing a basis vector $\mathbf{e}{\sigma(i)}$ from each row $i$. Terms where two rows choose the same basis vector vanish (alternating), so only choices where $\sigma$ is a permutation of ${1, \dots, n}$ survive. Reordering $\mathbf{e}{\sigma(1)}, \dots, \mathbf{e}_{\sigma(n)}$ back into $\mathbf{e}_1, \dots, \mathbf{e}_n$ by swaps produces the sign $\operatorname{sgn}\sigma = \pm 1$. The result is forced:

Theorem (Leibniz formula, existence and uniqueness). Exactly one function satisfies properties 1 to 3, namely

$$\det A = \sum_{\sigma \in S_n} \operatorname{sgn}(\sigma), a_{1\sigma(1)}a_{2\sigma(2)}\cdots a_{n\sigma(n)},$$

where $S_n$ is the set of all $n!$ permutations of ${1, \dots, n}$.

The sign of a permutation. An inversion of $\sigma$ is a pair $i < j$ with $\sigma(i) > \sigma(j)$. Then $\operatorname{sgn}\sigma = (-1)^{\text{number of inversions}}$, which also equals $(-1)^{\text{number of swaps}}$ in any sequence of swaps that sorts $\sigma$. Permutations with $\operatorname{sgn} = +1$ are even; those with $-1$ are odd.

$\sigma$ (as the list $\sigma(1), \sigma(2), \sigma(3)$)	Inversions	Sign	Term of $\det A$
1, 2, 3	0	$+$	$a_{11}a_{22}a_{33}$
2, 3, 1	2	$+$	$a_{12}a_{23}a_{31}$
3, 1, 2	2	$+$	$a_{13}a_{21}a_{32}$
3, 2, 1	3	$-$	$a_{13}a_{22}a_{31}$
1, 3, 2	1	$-$	$a_{11}a_{23}a_{32}$
2, 1, 3	1	$-$	$a_{12}a_{21}a_{33}$

These are exactly the six terms of the $3 \times 3$ formula and of the rule of Sarrus.

Payoffs of uniqueness. To prove that some formula equals $\det A$, it is enough to check that it has properties 1 to 3. Cofactor expansion along any row $i$ is multilinear (each term $a_{ij}C_{ij}$ is linear in row $i$, and $C_{ij}$ is multilinear in the other rows), alternating, and normalized, so it equals the Leibniz formula: this proves Laplace's theorem. Applying the Leibniz formula to $A^T$ and replacing each permutation by its inverse (which has the same sign) gives another proof of $\det A^T = \det A$.

from itertools import permutations
from math import prod
import numpy as np

def sign(p):
    """(-1)^(number of inversions): pairs i < j with p[i] > p[j]."""
    inversions = sum(p[i] > p[j] for i in range(len(p)) for j in range(i + 1, len(p)))
    return -1 if inversions % 2 else 1

def det_leibniz(A):
    n = len(A)
    # one term per permutation: pick a_{0,p(0)}, a_{1,p(1)}, ... (one entry per row AND per column)
    return sum(sign(p) * prod(A[i][p[i]] for i in range(n)) for p in permutations(range(n)))

print(sign((1, 2, 0)), sign((0, 2, 1)), sign((3, 2, 1, 0)))   # 1 -1 1 (0-based lists)

A = [[2, 1, 3], [0, -1, 4], [1, 2, 0]]
print(det_leibniz(A))                                         # -9

# Linear in one row (the others held fixed) ...
u, v = [1, 2, 3], [4, 0, -2]
r1, r3 = [5, 1, 0], [2, 2, 1]
lhs = det_leibniz([r1, [a + 3 * b for a, b in zip(u, v)], r3])
rhs = det_leibniz([r1, u, r3]) + 3 * det_leibniz([r1, v, r3])
print(lhs, rhs)                                               # 21 21

# ... alternating: two equal rows give zero ...
print(det_leibniz([r1, u, u]))                                # 0

# ... but NOT linear in the whole matrix
M = np.array(A)
print(det_leibniz((M + M).tolist()), 2 * det_leibniz(A))      # -72 -18

Key takeaways

• The determinant is the unique function of the rows that is multilinear, alternating, and equal to 1 on $I$.
• Alternating plus multilinear implies that swapping two rows flips the sign.
• Leibniz formula: $\det A = \sum_{\sigma \in S_n} \operatorname{sgn}(\sigma)\prod_i a_{i\sigma(i)}$, with $n!$ terms and $\operatorname{sgn}\sigma = (-1)^{#\text{inversions}}$.
• Multilinear (in each row separately) does not mean linear in the matrix.
• To prove a formula equals $\det$, check the three properties.

🧪 Practice

1. Count the inversions and find the sign of the permutations $(2, 3, 1)$, $(4, 3, 2, 1)$, and $(2, 1, 4, 3)$ (written as lists $\sigma(1), \sigma(2), \dots$).
2. Using only properties 1 to 3 (no formulas), prove that adding a multiple of one row to another does not change $D$.
3. Use the Leibniz formula to show that the determinant of an upper triangular matrix is the product of its diagonal entries. Suggestion: which permutations $\sigma$ satisfy $\sigma(i) \ge i$ for every $i$?
4. Interview: Over the field $\mathbb{F}_2 = {0, 1}$ (Section 1.2), consider defining "alternating" as "swapping two rows flips the sign" instead of "two equal rows give 0". Why is this weaker definition not enough to pin down the determinant over $\mathbb{F}_2$? Hint: in $\mathbb{F}_2$, $-1 = 1$; what does $D = -D$ tell you?

Determinants via Row Reduction

Why it matters. This is how determinants are actually computed. Elimination brings any matrix to triangular form in about $\tfrac{2}{3}n^3$ operations, and the rules for row operations say exactly how the determinant changed along the way. NumPy, MATLAB, and LAPACK all compute determinants this way, through the $PA = LU$ factorization of Section 4.4.

Algorithm. Reduce $A$ to an echelon form $U$ using only replacements and interchanges (no scaling). Count the interchanges $s$. Then

$$\det A = (-1)^s, u_{11}u_{22}\cdots u_{nn} = (-1)^s \times (\text{product of the pivots}).$$

If some column has no pivot, then $U$ has a zero on its diagonal and $\det A = 0$.

If you do scale a row by $c$ during hand computation, remember that the determinant of the new matrix is $c$ times the old one, so divide by $c$ at the end.

Connection to LU. If $PA = LU$ with $P$ a permutation matrix, $L$ unit lower triangular, and $U$ upper triangular, then $\det P \cdot \det A = \det L \cdot \det U = 1 \cdot \prod_i u_{ii}$. Since $\det P = (-1)^s$ and $1/(-1)^s = (-1)^s$:

$$\det A = (-1)^s\prod_{i=1}^{n} u_{ii}.$$

Example (with a row swap).

   A = [ 0  2  1 ]   R1 <-> R2    [ 1  3  2 ]   R3 <- R3 - 2 R1   [ 1   3  2 ]   R3 <- R3 + (5/2) R2   [ 1  3   2  ]
       [ 1  3  2 ]   --------->   [ 0  2  1 ]   --------------->  [ 0   2  1 ]   ------------------>   [ 0  2   1  ]
       [ 2  1  4 ]   s = 1        [ 2  1  4 ]   det unchanged     [ 0  -5  0 ]   det unchanged         [ 0  0  5/2 ]

   det A = (-1)^1 * (1 * 2 * 5/2) = -5

Mixed strategy for hand computation. Use a few row or column replacements to create zeros in one row or column, then expand along it. For example, in $\begin{bmatrix} 1 & 2 & 3 \ 2 & 5 & 8 \ 3 & 8 & 14 \end{bmatrix}$, clearing the first column leaves $\det\begin{bmatrix} 1 & 2 \ 2 & 5 \end{bmatrix} = 1$ to compute.

Numerical aspects.

• Libraries use partial pivoting (swapping the largest available pivot into place) for stability, exactly as in solving systems.
• For integer matrices, exact arithmetic with fractions gives the exact determinant; floating point gives a nearby float (for example, np.linalg.det may print -8.999999999999998 for an exact $-9$).
• Determinants of large matrices overflow or underflow easily, even for well-behaved matrices. np.linalg.slogdet returns the sign and $\sum_i \log\lvert u_{ii}\rvert$.
• A tiny or huge determinant says nothing reliable about being nearly singular; use the condition number (Section 11.3).

from fractions import Fraction as Fr
import numpy as np

def det_elimination(A):
    """Exact determinant: reduce to upper triangular form, tracking row swaps."""
    U = [[Fr(x) for x in row] for row in A]
    n, det = len(U), Fr(1)
    for k in range(n):
        p = next((r for r in range(k, n) if U[r][k] != 0), None)
        if p is None:
            return Fr(0)                          # no pivot in this column: singular
        if p != k:
            U[k], U[p] = U[p], U[k]               # interchange flips the sign
            det = -det
        for r in range(k + 1, n):                 # replacements leave det unchanged
            m = U[r][k] / U[k][k]
            U[r] = [a - m * b for a, b in zip(U[r], U[k])]
        det *= U[k][k]                            # multiply in the pivot
    return det

print(det_elimination([[0, 2, 1], [1, 3, 2], [2, 1, 4]]))    # -5
print(det_elimination([[2, 1, 3], [4, 5, 7], [-2, 3, 1]]))   # 16
print(det_elimination([[1, 2, 3], [4, 5, 6], [7, 8, 9]]))    # 0

# Libraries do the same thing via PA = LU with partial pivoting: O(n^3) work
rng = np.random.default_rng(1)
A = rng.standard_normal((200, 200))
print(np.linalg.det(A))                    # about -4.06e+185 for a 200 x 200 matrix of modest entries

# Products of many pivots overflow or underflow; slogdet returns sign and log|det| instead
B = 10 * np.eye(400)                       # det = 10^400, beyond the float range (about 1.8e308)
print(np.linalg.det(B))                    # inf (with an overflow warning)
sign, logabs = np.linalg.slogdet(B)
print(sign, round(logabs / np.log(10), 6)) # 1.0 400.0

Key takeaways

• Reduce to echelon form with replacements and interchanges; then $\det A = (-1)^{#\text{swaps}} \times$ product of pivots.
• A missing pivot means $\det A = 0$.
• With $PA = LU$: $\det A = \det P \cdot \prod_i u_{ii}$, costing $O(n^3)$ instead of $O(n!)$.
• Use exact arithmetic for exact answers, and slogdet for large matrices.

🧪 Practice

1. Compute $\det\begin{bmatrix} 1 & 2 & 3 \ 2 & 5 & 8 \ 3 & 8 & 14 \end{bmatrix}$ by row reduction.
2. Compute $\det\begin{bmatrix} 0 & 1 & 2 & 3 \ 1 & 0 & 1 & 2 \ 2 & 1 & 0 & 1 \ 3 & 2 & 1 & 0 \end{bmatrix}$ by row reduction, keeping track of the row interchanges.
3. Suppose elimination on $A$ needs no row swaps and produces pivots $p_1, \dots, p_n$. Show that the $k$-th leading principal minor of $A$ equals $p_1p_2\cdots p_k$, and conclude that $p_k = \Delta_k / \Delta_{k-1}$ in terms of the leading principal minors $\Delta_k$ (with $\Delta_0 = 1$).
4. Interview: Explain how np.linalg.det works, its time complexity, and why np.linalg.slogdet exists. Then explain why "the determinant is close to zero" is a poor test for "the matrix is close to singular". Hint: LU with partial pivoting; consider $10^{-3}I_{100}$, which is perfectly conditioned but has determinant $10^{-300}$.

<a id="53-applications-of-determinants"></a>

5.3 Applications of Determinants

This subchapter puts determinants to work: deciding invertibility, writing solutions and inverses in closed form, measuring areas, volumes, and orientation, and changing variables in multivariable integrals.

Invertibility Criterion

Why it matters. The determinant condenses the question "is this matrix invertible?" into "is this number nonzero?". Because the determinant is a polynomial in the entries, this turns questions about matrices that depend on a parameter into polynomial equations: for which values of $k$ does a system lose its unique solution, for which $t$ do vectors become dependent, for which $\lambda$ does $A - \lambda I$ become singular (the eigenvalue problem of Chapter 8).

Theorem. A square matrix $A$ is invertible if and only if $\det A \ne 0$.

Proof. Reduce $A$ to an echelon form $U$ by elementary row operations: $U = E_k \cdots E_1 A$. By the product rule, $\det U = \det E_k \cdots \det E_1 \cdot \det A$, and every $\det E_i$ is nonzero (it is $1$, $-1$, or a nonzero scale $c$). So $\det A \ne 0$ exactly when $\det U \ne 0$. The square echelon matrix $U$ is upper triangular, so $\det U$ is the product of its diagonal entries, which is nonzero exactly when every diagonal entry is a pivot, that is, when $A$ has $n$ pivots. By the Invertible Matrix Theorem (Section 4.3), that is exactly when $A$ is invertible. $\blacksquare$

This is statement (m) of the Invertible Matrix Theorem. In particular, for $n$ vectors in $\mathbb{R}^n$:

$$\mathbf{v}_1, \dots, \mathbf{v}_n \text{ are linearly independent} \iff \det[,\mathbf{v}_1\ \cdots\ \mathbf{v}_n,] \ne 0.$$

Matrices with parameters. Since $\det A$ is a polynomial in the entries, the set of parameter values where $A$ is singular is the set of roots of a polynomial. For example,

$$\det\begin{bmatrix} 1 & k & 0 \ k & 1 & k \ 0 & k & 1 \end{bmatrix} = 1(1 - k^2) - k(k - 0) + 0 = 1 - 2k^2,$$

so the matrix is singular exactly when $k = \pm\tfrac{1}{\sqrt{2}}$, and invertible for every other $k$.

Question	Determinant test
Does $A\mathbf{x} = \mathbf{b}$ have a unique solution for every $\mathbf{b}$?	$\det A \ne 0$
Does $A\mathbf{x} = \mathbf{0}$ have a nontrivial solution?	$\det A = 0$
Are $n$ vectors in $\mathbb{R}^n$ a basis?	$\det$ of the matrix with these columns $\ne 0$
Is $\lambda$ an eigenvalue of $A$?	$\det(A - \lambda I) = 0$ (Chapter 8)

Limits of the test. The criterion is exact in exact arithmetic. In floating point, use rank or the condition number to judge whether a matrix is nearly singular (Sections 4.3 and 11.3); the size of $\det A$ alone is misleading, since $\det(cA) = c^n \det A$ can be made arbitrarily small or large by rescaling.

import sympy as sp

k, lam, t = sp.symbols('k lambda t')

# For which k is the matrix singular?
A = sp.Matrix([[1, k, 0],
               [k, 1, k],
               [0, k, 1]])
d = A.det()
print(d, sp.solve(d, k))                       # 1 - 2*k**2 [-sqrt(2)/2, sqrt(2)/2]

# Are three vectors in R^3 linearly independent? Put them in the columns.
V = sp.Matrix([[1, 1, 1], [t, 2, 3], [t**2, 4, 9]])
print(sp.factor(V.det()))                      # (t - 3)*(t - 2): dependent only for t = 2, 3

# Preview of Chapter 8: A - lambda*I is singular exactly at the eigenvalues
B = sp.Matrix([[4, 1], [2, 3]])
p = (B - lam * sp.eye(2)).det()
print(sp.expand(p), sp.solve(p, lam))          # lambda**2 - 7*lambda + 10 [2, 5]

Key takeaways

• $A$ is invertible if and only if $\det A \ne 0$; this joins the Invertible Matrix Theorem as statement (m).
• $n$ vectors in $\mathbb{R}^n$ are linearly independent exactly when the determinant of the matrix they form is nonzero.
• For matrices depending on parameters, singular values of the parameter are roots of the polynomial $\det A$.
• In floating point, test near-singularity with rank or condition number, not with the size of $\det A$.

🧪 Practice

1. Use determinants to decide which matrices are invertible: $\begin{bmatrix} 2 & 6 \ 1 & 3 \end{bmatrix}$, $\begin{bmatrix} 1 & 0 & 2 \ 3 & 1 & 0 \ 0 & 2 & 1 \end{bmatrix}$, $\begin{bmatrix} 1 & 2 & 3 \ 0 & 0 & 4 \ 2 & 4 & 7 \end{bmatrix}$.
2. For which values of $t$ are the vectors $(1, t, t^2)$, $(1, 2, 4)$, and $(1, 3, 9)$ linearly independent? Interpret the answer in terms of fitting a parabola through three points.
3. Find all $\lambda$ for which $\begin{bmatrix} 4 & 1 \ 2 & 3 \end{bmatrix} - \lambda I$ is singular, and for each such $\lambda$ find a nonzero $\mathbf{x}$ with $(A - \lambda I)\mathbf{x} = \mathbf{0}$.
4. Interview: A bipartite graph has $n$ "left" and $n$ "right" vertices. Build the $n \times n$ matrix with a variable $x_{ij}$ in position $(i, j)$ when left vertex $i$ is joined to right vertex $j$, and 0 otherwise. Why is its determinant a nonzero polynomial exactly when the graph has a perfect matching, and how could you test this quickly with random numbers? Hint: each Leibniz term corresponds to a perfect matching, and distinct matchings give distinct monomials; then substitute random values from a large finite field (Schwartz-Zippel lemma).

Cramer's Rule

Why it matters. Cramer's rule writes each unknown of a square system as an explicit ratio of two determinants. It is rarely the right way to compute a solution, but it is the right way to reason about solutions: it shows that the solution depends smoothly (in fact, rationally) on the data, gives closed forms for small symbolic systems, and explains when integer systems have integer solutions.

Theorem (Cramer's rule). Let $A$ be an invertible $n \times n$ matrix and $\mathbf{b} \in \mathbb{R}^n$. For each $i$, let $A_i(\mathbf{b})$ be $A$ with its $i$-th column replaced by $\mathbf{b}$. Then the unique solution of $A\mathbf{x} = \mathbf{b}$ is

$$x_i = \frac{\det A_i(\mathbf{b})}{\det A}, \qquad i = 1, \dots, n.$$

Proof. Let $\mathbf{a}_1, \dots, \mathbf{a}_n$ be the columns of $A$. Since $A\mathbf{x} = \mathbf{b}$, we have $\mathbf{b} = x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n$. Substitute this into column $i$ and use multilinearity in that column (columns behave like rows, by the transpose rule):

$$\det A_i(\mathbf{b}) = \det[,\mathbf{a}_1 \cdots \textstyle\sum_j x_j\mathbf{a}_j \cdots \mathbf{a}_n,] = \sum_j x_j \det[,\mathbf{a}_1 \cdots \mathbf{a}_j \cdots \mathbf{a}_n,].$$

For $j \ne i$, the matrix in the sum has $\mathbf{a}_j$ in two columns, so its determinant is 0. Only $j = i$ survives: $\det A_i(\mathbf{b}) = x_i \det A$. $\blacksquare$

Example. Solve $3x + 2y = 7$, $x - y = -1$.

$$\det A = \begin{vmatrix} 3 & 2 \ 1 & -1 \end{vmatrix} = -5, \quad x = \frac{\begin{vmatrix} 7 & 2 \ -1 & -1 \end{vmatrix}}{-5} = \frac{-5}{-5} = 1, \quad y = \frac{\begin{vmatrix} 3 & 7 \ 1 & -1 \end{vmatrix}}{-5} = \frac{-10}{-5} = 2.$$

   A = [ 3   2 ]     A_1(b) = [  7   2 ]     A_2(b) = [ 3   7 ]
       [ 1  -1 ]              [ -1  -1 ]              [ 1  -1 ]
                               ^ b replaces column 1           ^ b replaces column 2

Geometric view ($2 \times 2$). $\det A_1(\mathbf{b})$ is the area of the parallelogram spanned by $\mathbf{b}$ and $\mathbf{a}_2$. Writing $\mathbf{b} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2$ and sliding $\mathbf{b}$ parallel to $\mathbf{a}_2$ (a shear, which keeps the area) leaves $x_1\mathbf{a}_1$, so that area is $x_1$ times the area spanned by $\mathbf{a}_1$ and $\mathbf{a}_2$.

When to use it, and when not to.

Use Cramer's rule for	Avoid it for
$2 \times 2$ and small $3 \times 3$ systems by hand	Numerical systems larger than about $3 \times 3$
Symbolic systems, to get a closed-form solution	Anything performance-sensitive: $n + 1$ determinants cost about $\tfrac{2}{3}n^4$ even with elimination
Proofs: $x_i$ is a rational function of the entries of $A$ and $\mathbf{b}$	Ill-conditioned systems: it is less stable than LU with pivoting
Finding a single unknown $x_i$ without the others

Integer solutions. If $A$ and $\mathbf{b}$ have integer entries and $\det A = \pm 1$, then every $\det A_i(\mathbf{b})$ is an integer, so the solution $\mathbf{x}$ is an integer vector.

import sympy as sp

def cramer(A, b):
    A, b = sp.Matrix(A), sp.Matrix(b)
    D = A.det()
    if D == 0:
        raise ValueError("Cramer's rule needs det A != 0")
    x = []
    for i in range(A.cols):
        Ai = A.copy()
        Ai[:, i] = b                       # replace column i by the right-hand side
        x.append(Ai.det() / D)
    return x

print(cramer([[3, 2], [1, -1]], [7, -1]))                  # [1, 2]

A = [[2, 1, 3], [0, -1, 4], [1, 2, 0]]
x = cramer(A, [1, 2, 3])
print(x)                                                   # [-25/9, 26/9, 11/9]
print(sp.Matrix(A) * sp.Matrix(x))                         # Matrix([[1], [2], [3]])

# Symbolic: the solution is a ratio of polynomials in the data
a, b_, c, d, e, f = sp.symbols('a b c d e f')
print(cramer([[a, b_], [c, d]], [e, f]))
# [(-b*f + d*e)/(a*d - b*c), (a*f - c*e)/(a*d - b*c)]

Key takeaways

• $x_i = \det A_i(\mathbf{b}) / \det A$, where $A_i(\mathbf{b})$ replaces column $i$ of $A$ by $\mathbf{b}$; valid only when $\det A \ne 0$.
• The proof is one line of multilinearity: $\det A_i(\mathbf{b}) = x_i\det A$.
• Each unknown is a rational function of the data, which is useful in theory and for symbolic work.
• For numerical computation, use LU-based solvers instead.

🧪 Practice

1. Solve with Cramer's rule: $2x - y = 4$, $3x + 5y = -7$; and $x + z = 3$, $2x + y = 5$, $y + 3z = 7$.
2. Consider $\begin{bmatrix} k & 1 \ 1 & k \end{bmatrix}\mathbf{x} = \begin{bmatrix} 1 \ 0 \end{bmatrix}$. Use Cramer's rule to find $x_1$ and $x_2$ as functions of $k$. For which $k$ do they fail to exist, and how do they behave as $k$ approaches those values?
3. Prove that if $A$ has integer entries and $\det A = \pm 1$, then $A\mathbf{x} = \mathbf{b}$ has an integer solution for every integer vector $\mathbf{b}$.
4. Interview: In a code review you find Cramer's rule used to solve $100 \times 100$ systems inside a simulation loop, with each determinant computed by LU. Estimate the cost relative to a single LU solve, and explain what to replace it with. Hint: count the determinants needed and the cost of each, then compare with $\tfrac{2}{3}n^3 + 2n^2$.

Adjugate Matrix and Inverse Formula

Why it matters. Section 4.3 gave the $2 \times 2$ inverse formula "swap, negate, divide by the determinant". The adjugate generalizes it to every size: the inverse is the transposed matrix of cofactors divided by the determinant. The formula is impractical for large numerical work but invaluable in theory: it shows that every entry of $A^{-1}$ is a ratio of determinants, it explains exactly when integer matrices have integer inverses, and it works even for matrices whose entries are polynomials or symbols.

Definition. The adjugate (or classical adjoint) of an $n \times n$ matrix $A$ is the transpose of its cofactor matrix:

$$\operatorname{adj} A = [C_{ij}]^T, \qquad (\operatorname{adj} A){ij} = C{ji}.$$

The transpose is easy to forget; it is essential.

Theorem. For every square matrix $A$,

$$A \operatorname{adj}(A) = \operatorname{adj}(A), A = (\det A), I.$$

Consequently, if $\det A \ne 0$,

$$A^{-1} = \frac{1}{\det A}\operatorname{adj} A.$$

Proof. The $(i, k)$ entry of $A\operatorname{adj}(A)$ is $\sum_j a_{ij}C_{kj}$: the entries of row $i$ multiplied by the cofactors of row $k$.

• If $i = k$, this is the cofactor expansion of $\det A$ along row $i$.
• If $i \ne k$, it is the cofactor expansion along row $k$ of the matrix obtained from $A$ by replacing row $k$ with a copy of row $i$. The cofactors of row $k$ do not involve row $k$, so they are unchanged. That matrix has two equal rows, so the sum is 0. (These are called expansions by alien cofactors.)

So $A\operatorname{adj}(A)$ has $\det A$ on the diagonal and 0 elsewhere. The product in the other order is handled the same way with columns. $\blacksquare$

Example. For the running example $A = \begin{bmatrix} 2 & 1 & 3 \ 0 & -1 & 4 \ 1 & 2 & 0 \end{bmatrix}$, the cofactor matrix was computed in "Minors and Cofactors". Transposing it:

$$\operatorname{adj} A = \begin{bmatrix} -8 & 6 & 7 \ 4 & -3 & -8 \ 1 & -3 & -2 \end{bmatrix}, \qquad A^{-1} = -\frac{1}{9}\begin{bmatrix} -8 & 6 & 7 \ 4 & -3 & -8 \ 1 & -3 & -2 \end{bmatrix}.$$

For $2 \times 2$ matrices, the cofactors of $\begin{bmatrix} a & b \ c & d \end{bmatrix}$ are $C_{11} = d$, $C_{12} = -c$, $C_{21} = -b$, $C_{22} = a$, so $\operatorname{adj} A = \begin{bmatrix} d & -b \ -c & a \end{bmatrix}$, recovering Section 4.3.

Consequences.

Fact	Explanation
$(A^{-1}){ij} = C{ji}/\det A$	A single entry of the inverse costs one minor
Cramer's rule	$\mathbf{x} = A^{-1}\mathbf{b} = \frac{1}{\det A}\operatorname{adj}(A)\mathbf{b}$, and $(\operatorname{adj}(A)\mathbf{b})i = \sum_j C{ji}b_j = \det A_i(\mathbf{b})$
$\det(\operatorname{adj} A) = (\det A)^{n-1}$	Take determinants of $A\operatorname{adj}(A) = (\det A)I$
Integer $A$ has an integer inverse $\iff \det A = \pm 1$	Cofactors of an integer matrix are integers; see Section 5.2 for the converse
Singular $A$: $A\operatorname{adj}(A) = O$	Every column of $\operatorname{adj} A$ solves $A\mathbf{x} = \mathbf{0}$

Integer matrices with determinant $\pm 1$ are called unimodular. They are exactly the invertible changes of coordinates on the integer lattice $\mathbb{Z}^n$, and they appear in cryptography (the Hill cipher, Section 13.3) and in integer programming.

import sympy as sp

A = sp.Matrix([[2, 1, 3],
               [0, -1, 4],
               [1, 2, 0]])

C = A.cofactor_matrix()
adj = A.adjugate()                       # = C.T
print(adj)                               # Matrix([[-8, 6, 7], [4, -3, -8], [1, -3, -2]])
print(adj == C.T)                        # True
print(A * adj)                           # Matrix([[-9, 0, 0], [0, -9, 0], [0, 0, -9]]) = det(A) I
print(adj / A.det() == A.inv())          # True

# A single entry of the inverse: (A^-1)_{23} = C_32 / det A
print(C[2, 1] / A.det(), A.inv()[1, 2])  # 8/9 8/9

# Singular A: A * adj(A) = O, so the columns of adj(A) lie in the null space of A
S = sp.Matrix([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print(S.adjugate())                      # Matrix([[-3, 6, -3], [6, -12, 6], [-3, 6, -3]])
print(S * S.adjugate())                  # zero matrix

# det(adj A) = (det A)^(n-1)
print(adj.det(), A.det() ** 2)           # 81 81

# Integer matrix with det = 1: the inverse has integer entries
U = sp.Matrix([[2, 3], [1, 2]])
print(U.det(), U.inv())                  # 1 Matrix([[2, -3], [-1, 2]])

Key takeaways

• $\operatorname{adj} A$ is the transpose of the cofactor matrix.
• $A\operatorname{adj}(A) = \operatorname{adj}(A)A = (\det A)I$, so $A^{-1} = \operatorname{adj}(A)/\det A$ when $\det A \ne 0$.
• Each entry of $A^{-1}$ is a ratio of determinants: $(A^{-1}){ij} = C{ji}/\det A$.
• An integer matrix has an integer inverse exactly when its determinant is $\pm 1$.
• The formula costs far more than elimination for large $n$; use it for theory, symbolic work, and small matrices.

🧪 Practice

1. Compute $\operatorname{adj} A$ and $A^{-1}$ for $A = \begin{bmatrix} 1 & 2 & 0 \ 0 & 1 & 3 \ 2 & 0 & 1 \end{bmatrix}$, and verify that $A\operatorname{adj}(A) = (\det A)I$.
2. Prove that $\det(\operatorname{adj} A) = (\det A)^{n-1}$ for every invertible $n \times n$ matrix $A$.
3. For the singular matrix $S = \begin{bmatrix} 1 & 2 & 3 \ 4 & 5 & 6 \ 7 & 8 & 9 \end{bmatrix}$, compute $\operatorname{adj} S$ and use it to find a nonzero solution of $S\mathbf{x} = \mathbf{0}$ without row reduction.
4. Interview: You need a $3 \times 3$ integer matrix, other than a permutation or triangular matrix, whose inverse also has integer entries (for example, to scramble and unscramble integer coordinates). How would you construct one, and how would you verify it without computing the inverse? Hint: products of integer elementary matrices of replacement type have determinant 1; then use the adjugate formula.

Area, Volume, and Orientation

Why it matters. The determinant is not only an algebraic test; it is the volume scaling factor of a linear map. This makes determinants the standard tool for areas and volumes computed from coordinates (triangles in meshes, polygons in maps, tetrahedra in simulations) and for orientation tests, which decide whether three points make a left or a right turn. These predicates are the core of computational geometry: convex hulls, segment intersection, point-in-polygon tests, and triangulations.

Theorem. Let $A$ be an $n \times n$ matrix.

1. The parallelogram ($n = 2$) or parallelepiped ($n = 3$, and its analogue in general) spanned by the columns of $A$ has area or volume $\lvert\det A\rvert$.
2. For every region $S$ with a well-defined area or volume, the image $A(S) = {A\mathbf{x} : \mathbf{x} \in S}$ satisfies $$\operatorname{vol}(A(S)) = \lvert\det A\rvert \cdot \operatorname{vol}(S).$$

Why. For part 1, the statement holds for elementary matrices: a shear preserves volume and $\lvert\det\rvert = 1$, a swap only relabels axes, and scaling one axis by $c$ scales volume by $\lvert c\rvert$. An invertible $A$ is a product of elementary matrices, and both volume factors and determinants multiply. A singular $A$ flattens space into a lower-dimensional subspace, of volume 0. For part 2, fill $S$ with a fine grid of tiny cubes. Each cube maps to a tiny parallelepiped whose volume is $\lvert\det A\rvert$ times that of the cube, and refining the grid gives the result.

   unit square                         image under A = [ a  b ]
                                                       [ c  d ]
   e2 ^                                            (b,d)
      |____                                          ^ _______
      |    |          ------ A ------>              / /      /
      |____|____> e1                               / /      /      area = |ad - bc|
      O                                           O/_______/
                                                      (a,c)
   every region's area is multiplied by the same factor |det A|

Orientation. The sign of $\det A$ records whether $A$ preserves orientation ($\det A > 0$: counterclockwise stays counterclockwise, right-handed stays right-handed) or reverses it ($\det A < 0$: the map includes a reflection). Rotations have determinant $+1$; reflections have $-1$.

Formulas from coordinates.

Shape	Formula
Triangle $P, Q, R$ in $\mathbb{R}^2$	$\tfrac12\lvert\det[,Q - P\ \ R - P,]\rvert$
Polygon $(x_1, y_1), \dots, (x_m, y_m)$, vertices in order (shoelace formula)	$\tfrac12\left\lvert\sum_{k=1}^m (x_ky_{k+1} - x_{k+1}y_k)\right\rvert$, indices mod $m$
Tetrahedron $P_0, P_1, P_2, P_3$ in $\mathbb{R}^3$	$\tfrac16\lvert\det[,P_1 - P_0\ \ P_2 - P_0\ \ P_3 - P_0,]\rvert$
Simplex with $n + 1$ vertices in $\mathbb{R}^n$	$\tfrac{1}{n!}\lvert\det[,P_1 - P_0\ \cdots\ P_n - P_0,]\rvert$

Without the absolute value, the shoelace sum is a signed area: positive for vertices listed counterclockwise and negative for clockwise.

The orientation predicate. For points $p, q, r$ in the plane,

$$\operatorname{orient}(p, q, r) = \det\begin{bmatrix} q_x - p_x & r_x - p_x \ q_y - p_y & r_y - p_y \end{bmatrix}$$

is positive if $p \to q \to r$ turns left (counterclockwise), negative if it turns right, and zero if the three points are collinear.

import numpy as np

def orient(p, q, r):
    """> 0: p -> q -> r turns left (counterclockwise); < 0: right turn; 0: collinear."""
    return (q[0] - p[0]) * (r[1] - p[1]) - (q[1] - p[1]) * (r[0] - p[0])

print(orient((0, 0), (4, 0), (1, 3)))      # 12: left turn
print(orient((0, 0), (1, 3), (4, 0)))      # -12: same points, opposite order
print(orient((0, 0), (1, 1), (3, 3)))      # 0: collinear

def shoelace(poly):
    """Signed area of a simple polygon: half the sum of 2 x 2 determinants of consecutive vertices."""
    s = 0
    for (x1, y1), (x2, y2) in zip(poly, poly[1:] + poly[:1]):
        s += x1 * y2 - x2 * y1             # det [[x1, x2], [y1, y2]]
    return s / 2

L_shape = [(0, 0), (4, 0), (4, 1), (1, 1), (1, 3), (0, 3)]
print(shoelace(L_shape))                   # 6.0: positive, so the vertices are counterclockwise
print(shoelace(L_shape[::-1]))             # -6.0: clockwise

# A linear map scales every area by |det A|: the unit disk (area pi) becomes an ellipse
A = np.array([[3., 1], [1, 2]])            # det A = 5
theta = np.linspace(0, 2 * np.pi, 100_000, endpoint=False)
ellipse = (A @ np.vstack([np.cos(theta), np.sin(theta)])).T
print(round(shoelace(ellipse.tolist()), 4), round(5 * np.pi, 4))   # 15.708 15.708

# Tetrahedron volume = |det| / 6 of the three edge vectors from one vertex
P0, P1, P2, P3 = map(np.array, [(0, 0, 0), (2, 0, 0), (0, 3, 0), (1, 1, 4)])
print(round(abs(np.linalg.det(np.array([P1 - P0, P2 - P0, P3 - P0]))) / 6, 10))   # 4.0

Key takeaways

• $\lvert\det A\rvert$ is the area or volume spanned by the columns of $A$, and the factor by which $A$ scales every area or volume.
• The sign of $\det A$ tells whether $A$ preserves ($+$) or reverses ($-$) orientation.
• Triangle area is $\tfrac12\lvert\det\rvert$, tetrahedron volume is $\tfrac16\lvert\det\rvert$, and polygon area follows from the shoelace formula.
• The orientation predicate, a $2 \times 2$ determinant, is the basic building block of computational geometry.

🧪 Practice

1. Find the area of the triangle with vertices $(1, 1)$, $(4, 2)$, and $(2, 5)$, and decide whether this listing order is clockwise or counterclockwise.
2. Find the volume of the tetrahedron with vertices $(0, 0, 0)$, $(2, 0, 0)$, $(0, 3, 0)$, and $(1, 1, 4)$, and of the parallelepiped spanned by the same three edge vectors.
3. The ellipse $x^2/a^2 + y^2/b^2 \le 1$ is the image of the unit disk under a diagonal matrix. Use this to derive its area $\pi ab$. What is the area of the image of the unit disk under $\begin{bmatrix} 3 & 1 \ 1 & 2 \end{bmatrix}$?
4. Interview: A GIS file stores land parcels as lists of polygon vertices, and the specification says outer boundaries must be counterclockwise. Write (in words or code) a check that detects violating polygons, and explain how to fix them. Hint: the sign of the shoelace sum; reversing the vertex list flips it.

Jacobian and Change of Variables

Why it matters. Linear maps scale all areas by the same factor $\lvert\det A\rvert$. Nonlinear maps, such as the change from polar to Cartesian coordinates, scale different regions by different amounts, but locally every smooth map looks linear, and its local scaling factor is the determinant of its derivative matrix, the Jacobian. This is why integrals pick up the factor $r$ in polar coordinates, why probability densities change when random variables are transformed, and why normalizing-flow models in machine learning are built from layers whose Jacobian determinants are cheap to compute.

The Jacobian matrix. For a differentiable map $F : \mathbb{R}^n \to \mathbb{R}^n$, $F(\mathbf{u}) = (F_1(\mathbf{u}), \dots, F_n(\mathbf{u}))$, the Jacobian matrix at $\mathbf{u}$ is

$$J_F(\mathbf{u}) = \begin{bmatrix} \dfrac{\partial F_1}{\partial u_1} & \cdots & \dfrac{\partial F_1}{\partial u_n} \ \vdots & \ddots & \vdots \ \dfrac{\partial F_n}{\partial u_1} & \cdots & \dfrac{\partial F_n}{\partial u_n} \end{bmatrix}.$$

Near $\mathbf{u}$, the map is approximately linear: $F(\mathbf{u} + \mathbf{h}) \approx F(\mathbf{u}) + J_F(\mathbf{u})\mathbf{h}$. So a tiny box of volume $dV$ near $\mathbf{u}$ maps to an approximate parallelepiped of volume $\lvert\det J_F(\mathbf{u})\rvert,dV$. The number $\det J_F$ is often called simply "the Jacobian".

Change of variables formula. If $F$ maps a region $D$ one-to-one onto $F(D)$ and $\det J_F \ne 0$ on $D$, then

$$\int_{F(D)} f(\mathbf{x}),d\mathbf{x} = \int_D f(F(\mathbf{u})),\lvert\det J_F(\mathbf{u})\rvert,d\mathbf{u}.$$

It is the multivariable version of the substitution rule $\int f(x),dx = \int f(g(u)),\lvert g'(u)\rvert,du$, with $\lvert\det J_F\rvert$ in place of $\lvert g'(u)\rvert$.

Polar coordinates. $x = r\cos\theta$, $y = r\sin\theta$:

$$J = \begin{bmatrix} \cos\theta & -r\sin\theta \ \sin\theta & r\cos\theta \end{bmatrix}, \qquad \det J = r\cos^2\theta + r\sin^2\theta = r,$$

so $dx,dy = r,dr,d\theta$. A small polar cell is approximately a rectangle with sides $dr$ and $r,d\theta$:

                   r d theta
              .---------------.          the cell between radii r and r + dr
             /                 \         and angles theta and theta + d theta
            /       cell        \  dr
           '---------------------'       area ~ (r d theta) * dr = r dr d theta
                                         cells far from the origin are larger:
          ^ angle d theta at the origin  that is the factor |det J| = r

Common Jacobians.

Coordinates	Map	$\lvert\det J\rvert$
Linear	$\mathbf{x} = A\mathbf{u}$	$\lvert\det A\rvert$ (constant)
Polar	$(r\cos\theta, r\sin\theta)$	$r$
Cylindrical	$(r\cos\theta, r\sin\theta, z)$	$r$
Spherical	$(\rho\sin\varphi\cos\theta, \rho\sin\varphi\sin\theta, \rho\cos\varphi)$	$\rho^2\sin\varphi$

Inverse function theorem. If $\det J_F(\mathbf{u}) \ne 0$, then $F$ is invertible near $\mathbf{u}$, and the Jacobian of the inverse map is $J_F^{-1}$, with determinant $1/\det J_F$. Where $\det J_F = 0$ (for polar coordinates, at $r = 0$), the map folds or collapses and local invertibility can fail.

Transforming probability densities. If $\mathbf{Y} = F(\mathbf{X})$ with $F$ invertible and smooth, the probability in a tiny region is preserved while its volume is scaled by $\lvert\det J_F\rvert$, so

$$p_{\mathbf{Y}}(F(\mathbf{x})) = \frac{p_{\mathbf{X}}(\mathbf{x})}{\lvert\det J_F(\mathbf{x})\rvert}, \qquad \log p_{\mathbf{Y}}(\mathbf{y}) = \log p_{\mathbf{X}}(\mathbf{x}) - \log\lvert\det J_F(\mathbf{x})\rvert.$$

Normalizing flows model complicated distributions by pushing a simple Gaussian through many invertible layers, and need $\log\lvert\det J\rvert$ for every layer. A general $n \times n$ determinant costs $O(n^3)$, so layers are designed with triangular Jacobians, whose log-determinant is a sum of $n$ logarithms of diagonal entries.

import numpy as np
import sympy as sp

r, th, rho, phi = sp.symbols('r theta rho phi', positive=True)

# Polar coordinates: dA = r dr dtheta
polar = sp.Matrix([r * sp.cos(th), r * sp.sin(th)])
J = polar.jacobian([r, th])
print(J)                                    # Matrix([[cos(theta), -r*sin(theta)], [sin(theta), r*cos(theta)]])
print(sp.simplify(J.det()))                 # r

# Spherical coordinates: dV = rho^2 sin(phi) drho dphi dtheta
sph = sp.Matrix([rho * sp.sin(phi) * sp.cos(th),
                 rho * sp.sin(phi) * sp.sin(th),
                 rho * sp.cos(phi)])
print(sp.simplify(sph.jacobian([rho, phi, th]).det()))    # rho**2*sin(phi)

# The Gaussian integral over the plane, done in polar coordinates (note the factor r)
print(sp.integrate(sp.exp(-r**2) * r, (r, 0, sp.oo), (th, 0, 2 * sp.pi)))   # pi

# Numerical check: a tiny square at (r, theta) = (2, 0.5) maps to a region of area ~ r * dr * dtheta
F = lambda p: np.array([p[0] * np.cos(p[1]), p[0] * np.sin(p[1])])
h = 1e-4
corners = [F(np.array([2, 0.5]) + d) for d in ([0, 0], [h, 0], [h, h], [0, h])]
x, y = np.array(corners).T
image_area = 0.5 * abs(np.dot(x, np.roll(y, -1)) - np.dot(y, np.roll(x, -1)))   # shoelace formula
print(round(image_area / h**2, 4))          # 2.0 = |det J| at r = 2

# Affine coupling layer (normalizing flows): y1 = x1, y2 = x2 * exp(s(x1)) + t(x1)
s = lambda x1: np.tanh(x1)                  # s and t can be arbitrary functions (e.g. neural networks)
t = lambda x1: x1 ** 2
G = lambda v: np.array([v[0], v[1] * np.exp(s(v[0])) + t(v[0])])

v = np.array([0.7, -1.3])
eps = 1e-6
J_num = np.column_stack([(G(v + eps * e) - G(v - eps * e)) / (2 * eps) for e in np.eye(2)])
print(np.round(J_num, 4))                   # [[1, 0], [-0.1101, 1.8301]]: lower triangular
print(round(np.log(abs(np.linalg.det(J_num))), 6), round(s(v[0]), 6))   # 0.604368 0.604368: log|det J| = s(x1)

Key takeaways

• The Jacobian matrix $J_F$ is the best linear approximation of $F$ near a point; $\lvert\det J_F\rvert$ is the local volume scaling factor.
• Change of variables: $\int_{F(D)} f,d\mathbf{x} = \int_D (f \circ F),\lvert\det J_F\rvert,d\mathbf{u}$.
• Polar: $dA = r,dr,d\theta$; spherical: $dV = \rho^2\sin\varphi,d\rho,d\varphi,d\theta$.
• $\det J_F \ne 0$ guarantees local invertibility (inverse function theorem).
• Densities transform by dividing by $\lvert\det J_F\rvert$; triangular Jacobians make this cheap in normalizing flows.

🧪 Practice

1. For the change of variables $u = x + y$, $v = x - y$, compute $\det\dfrac{\partial(u, v)}{\partial(x, y)}$ and $\det\dfrac{\partial(x, y)}{\partial(u, v)}$. Use the result to find the area in the $xy$-plane of the region where $0 \le u \le 2$ and $0 \le v \le 4$.
2. Verify that the spherical-coordinates Jacobian determinant is $\rho^2\sin\varphi$ by computing the $3 \times 3$ determinant by hand, and use it to derive the volume $\tfrac43\pi R^3$ of a ball.
3. Use polar coordinates to show that $\iint_{\mathbb{R}^2} e^{-(x^2 + y^2)},dx,dy = \pi$, and deduce that $\int_{-\infty}^{\infty} e^{-x^2},dx = \sqrt{\pi}$.
4. Interview: In a normalizing flow with data dimension $n = 3{,}072$ (a $32 \times 32$ color image), why are coupling layers designed so that each layer's Jacobian is triangular rather than allowing a general invertible map? Quantify the saving per layer. Hint: compare the cost of $\log\lvert\det J\rvert$ via LU, about $\tfrac23 n^3$ operations, with summing $n$ logarithms of diagonal entries.

<a id="6-vector-spaces"></a>