2. Vectors in Euclidean Space

Vectors are the basic objects of linear algebra. A vector is a list of numbers that you can also picture as an arrow with a length and a direction. This chapter develops both views in $\mathbb{R}^n$. It defines the operations that combine vectors (addition, scaling, and the dot and cross products) and uses them to measure lengths, angles, areas, and volumes and to describe lines and planes. Later chapters, from linear systems to eigenvectors, are written in terms of these objects, and the geometric intuition built here is what makes those chapters readable.

2.1 Vector Fundamentals

This subchapter defines vectors in the plane and in space, extends them to any number of dimensions, and separates the two ways of reading them: as arrows and as lists of numbers.

Vectors in R^2 and R^3

Why it matters. Many quantities need more than one number. A displacement has an east part and a north part, a velocity in space has three parts, and a force has a strength and a direction. A vector bundles these numbers into one object that you can add, scale, and measure.

Definition. A vector in $\mathbb{R}^2$ is an ordered pair of real numbers, and a vector in $\mathbb{R}^3$ is an ordered triple:

$$\mathbf{v} = \begin{pmatrix} v_1 \ v_2 \end{pmatrix} \in \mathbb{R}^2, \qquad \mathbf{w} = \begin{pmatrix} w_1 \ w_2 \ w_3 \end{pmatrix} \in \mathbb{R}^3.$$

The numbers $v_i$ are the components (or entries) of $\mathbf{v}$. This guide writes vectors as columns by default and as rows $(v_1, v_2)$ in running text to save space. Two vectors are equal exactly when all their corresponding components are equal. Order matters: $(1, 2) \ne (2, 1)$.

The arrow picture. Draw $\mathbf{v} = (3, 2)$ as an arrow from the origin to the point $(3, 2)$. The components are the instructions "move 3 along the $x$-axis, then 2 along the $y$-axis", like the directions "3 blocks east, 2 blocks north".

  y
  ^
2 |                 * (3, 2)
  |              _/
1 |          _/          v = (3, 2)
  |      _/              3 units along x, 2 units along y
  |  _/
0 O-----+-----+-----+---> x
  0     1     2     3

Coordinate axes in $\mathbb{R}^3$. Space uses three perpendicular axes arranged in a right-handed system: point the fingers of your right hand along $x$ and curl them toward $y$, and your thumb points along $z$.

          z
          ^
          |
          |
          O------> y
         /
        /
       v
      x         right-handed: x curled toward y gives z

Special vectors.

Vector	$\mathbb{R}^2$	$\mathbb{R}^3$	Role
Zero vector $\mathbf{0}$	$(0, 0)$	$(0, 0, 0)$	The "do not move" displacement
Standard basis $\mathbf{e}_1$ (or $\mathbf{i}$)	$(1, 0)$	$(1, 0, 0)$	One unit along $x$
Standard basis $\mathbf{e}_2$ (or $\mathbf{j}$)	$(0, 1)$	$(0, 1, 0)$	One unit along $y$
Standard basis $\mathbf{e}_3$ (or $\mathbf{k}$)	n/a	$(0, 0, 1)$	One unit along $z$

Every vector can be rebuilt from its components and the standard basis: $(3, 2) = 3\mathbf{e}_1 + 2\mathbf{e}_2$. This is the first example of a linear combination (Section 2.2).

Notation. Textbooks write vectors as $\mathbf{v}$ (bold), $\vec{v}$ (arrow), or $\underline{v}$ (handwriting). This guide uses bold for vectors and plain italic letters for scalars.

import numpy as np

v = np.array([3, 2])                  # a vector in R^2
w = np.array([1, -2, 4])              # a vector in R^3

print(v[0], v[1])                     # 3 2   (NumPy indexes components from 0)
print(w.shape)                        # (3,)  three components
print(np.array_equal(v, [2, 3]))      # False: order matters

e1, e2 = np.eye(2)                    # standard basis vectors of R^2
print(3 * e1 + 2 * e2)                # [3. 2.]  v rebuilt from its components

Key takeaways

A vector in $\mathbb{R}^2$ or $\mathbb{R}^3$ is an ordered list of 2 or 3 real components.
Equality is componentwise, and the order of components matters.
Geometrically, a vector is an arrow whose components say how far to move along each axis.
$\mathbb{R}^3$ uses right-handed axes, and every vector is a combination of the standard basis vectors.

🧪 Practice

Sketch $\mathbf{u} = (2, -1)$, $\mathbf{v} = (-3, 2)$, and $\mathbf{w} = (0, 4)$ as arrows from the origin.

Find all real $a, b$ with $(a + b,\ 2a) = (5,\ 4)$.

Write $(4, -1, 7)$ in terms of $\mathbf{i}, \mathbf{j}, \mathbf{k}$. Which coordinate plane contains the point $(0, 3, -2)$?

Interview: Why is a vector in $\mathbb{R}^2$ defined as an ordered pair $(v_1, v_2)$ and not as the set ${v_1, v_2}$? Hint: compare ${1, 2}$ with ${2, 1}$, and think about what happens to the vector $(1, 1)$.

Vectors in R^n

Why it matters. Real data rarely fits in three numbers. A $28 \times 28$ grayscale image has 784 pixel values, a customer's ratings can span thousands of products, and a system of equations can have hundreds of unknowns. You cannot draw these objects, but the algebra of vectors works the same in every dimension.

Definition. $\mathbb{R}^n$ is the set of ordered $n$-tuples of real numbers, the Cartesian product $\mathbb{R} \times \cdots \times \mathbb{R}$ from Chapter 1:

$$\mathbb{R}^n = {(v_1, v_2, \dots, v_n) : v_i \in \mathbb{R}}.$$

The number $n$ is the dimension of the space (made precise in Chapter 6). All definitions in this chapter, except the cross product, work componentwise for any $n$. You get them by writing "for $i = 1, \dots, n$" where the 2D and 3D versions write out each component.

Vectors as data. A vector in $\mathbb{R}^n$ represents anything described by $n$ numbers in a fixed order:

Object	Vector	$n$
RGB color	$(255, 128, 0)$	3
Polynomial $a_0 + a_1 x + a_2 x^2$	$(a_0, a_1, a_2)$	3
$28 \times 28$ grayscale image, read row by row	pixel intensities	784
Word counts of a document over a vocabulary	one count per word	vocabulary size
Unknowns of a linear system	$(x_1, \dots, x_n)$	number of unknowns

The fixed order is a contract. Adding two image vectors only makes sense if pixel 17 means the same location in both.

Other scalars. Replacing $\mathbb{R}$ with any field $\mathbb{F}$ from Chapter 1 gives $\mathbb{F}^n$: $\mathbb{C}^n$ for quantum mechanics and signal processing, or $\mathbb{F}_2^n$ (bit strings) for coding theory. The operations in Section 2.2 are defined the same way over any field. Lengths and angles (Section 2.3) need real scalars or the complex modifications covered in Chapter 9.

Intuition in high dimensions. Two vectors in $\mathbb{R}^{1000}$ still lie in a single plane through the origin, and a triangle formed by three points is still a flat triangle. Many 2D sketches stay faithful in high dimensions, because most statements involve only two or three vectors at a time.

import numpy as np

img = np.array([[0, 255],
                [128, 64]])          # a tiny 2x2 grayscale image
x = img.flatten()                    # read row by row: a vector in R^4
print(x, x.size)                     # [  0 255 128  64] 4

n = 784                              # a 28x28 handwritten digit
e1 = np.zeros(n)
e1[0] = 1.0                          # the first standard basis vector of R^784
print(e1[:3], e1.size)               # [1. 0. 0.] 784

p = np.array([1, 0, 2])              # 1 + 2x^2
q = np.array([0, 3, -1])             # 3x - x^2
print(p + q)                         # [1 3 1]  ->  1 + 3x + x^2: adding vectors adds polynomials

Key takeaways

$\mathbb{R}^n$ is the set of ordered $n$-tuples of reals; every operation in this chapter except the cross product is defined componentwise for any $n$.
Any object described by $n$ numbers in a fixed order is a vector in $\mathbb{R}^n$, and that order must be consistent.
Replacing $\mathbb{R}$ with another field $\mathbb{F}$ gives $\mathbb{F}^n$ with the same algebraic operations.
You cannot draw $\mathbb{R}^n$, but you can reason about two or three of its vectors at a time with 2D or 3D pictures.

🧪 Practice

How many components does a vector representing a $1920 \times 1080$ RGB image have?

Encode $p(x) = 3 - x + 2x^3$ and $q(x) = x^2 + x^3$ as vectors in $\mathbb{R}^4$. Which polynomial corresponds to the vector $2\mathbf{p} - \mathbf{q}$?

List all vectors in $\mathbb{F}_2^3$. How many are there, and how many are in $\mathbb{F}_2^n$?

Interview: Why can we still use 2D sketches to reason about two vectors in $\mathbb{R}^{1000}$? Hint: think about the smallest flat surface through the origin that contains both vectors.

Geometric vs. Algebraic Interpretation

Why it matters. Every vector concept has two faces. The geometric face (arrows) gives intuition and tells you what to expect. The algebraic face (lists of numbers) gives proofs, computation, and a way to work beyond three dimensions. Switching between them is a core skill in linear algebra.

The geometric view. A vector is an arrow with a length and a direction. Where you draw it does not matter: all arrows with the same length and direction represent the same vector. "Walk 3 east and 2 north" is the same instruction whether you start at home or at work.

  ^ y
  |            _>              _>
  |          _/              _/
  |        _/              _/          two copies of the same vector (3, 2),
  |      _/              _/            drawn from different starting points
  |     /               /
  +-----------------------------------> x

The algebraic view. A vector is a tuple $(v_1, \dots, v_n)$, and every operation is a formula on components. The arrow from a point $P$ to a point $Q$ has components $Q - P$ (head minus tail), whatever $P$ is.

A translation dictionary.

Concept	Geometric view	Algebraic view
Vector	Arrow: length and direction	Tuple $(v_1, \dots, v_n)$
Equality	Same length and direction	Same components
Addition	Tip-to-tail / parallelogram	Add components
Scaling by $c$	Stretch by $\lvert c \rvert$, flip if $c < 0$	Multiply each component
Length	Length of the arrow	$\sqrt{v_1^2 + \cdots + v_n^2}$
Perpendicular	Right angle between arrows	Dot product is 0
Zero vector	Arrow of length 0 (a point)	$(0, \dots, 0)$
Works in	$n \le 3$ (drawable)	Any $n$

How to use both. Use the picture to guess a result ("the diagonals of a rhombus are perpendicular") and the algebra to prove it ("expand $(\mathbf{u} + \mathbf{v}) \cdot (\mathbf{u} - \mathbf{v})$"). When the algebra gives a surprising answer, sketch the 2D case to check it.

import numpy as np

def arrow(tail, head):
    """Algebraic content of the arrow from tail to head: head minus tail."""
    return np.array(head) - np.array(tail)

print(arrow((0, 0), (3, 2)))      # [3 2]
print(arrow((1, 1), (4, 3)))      # [3 2]  same vector, different starting point
print(arrow((-2, 5), (1, 7)))     # [3 2]

Key takeaways

Geometrically, a vector is an arrow defined by length and direction, not by where it is drawn.
Algebraically, a vector is a tuple, and operations are formulas on components.
The arrow from $P$ to $Q$ is the vector $Q - P$ (head minus tail).
Use pictures to form conjectures and algebra to prove them, especially beyond three dimensions.

🧪 Practice

Find the vector from $P = (1, 4)$ to $Q = (-2, 0)$, and draw it twice: once from the origin and once from $P$.

The arrow from $A = (1, 2)$ to $B = (4, 6)$ and the arrow from $C$ to $D = (0, 1)$ represent the same vector. Find $C$.

Show that the quadrilateral with vertices $A, B, C, D$ (in order) is a parallelogram exactly when $B - A = C - D$.

Interview: Name one property of vector addition that is obvious from the picture but needs a calculation in components, and one that is easy in components but impossible to draw. Hint: consider why $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$ follows from a parallelogram, and what happens to pictures in $\mathbb{R}^{100}$.

Position Vectors and Points

Why it matters. Points (locations) and vectors (displacements) are both written as tuples, but they are different things. Mixing them up causes real bugs. For example, a graphics engine that translates a surface's normal direction along with its vertices produces broken lighting.

Points and position vectors. A point $P$ is a location. Once you fix an origin $O$, every point $P$ has a position vector $\mathbf{p} = \overrightarrow{OP}$, the arrow from the origin to $P$, with the same coordinates as $P$. This one-to-one match is why $\mathbb{R}^n$ can serve as both "the set of points" and "the set of vectors".

                 * Q
                /^
               / |
           q  /  |  q - p : displacement from P to Q
             /   |
            /    * P
           /  _/
          / _/  p
         O         (not to scale)
   p, q: position vectors, arrows from the origin O

Which operations make sense. The test is whether the result depends on where you put the origin. Meaningful operations give the same geometric answer for any origin.

Operation	Result	Meaningful?	Example
point $-$ point	vector	Yes	$Q - P$ is the displacement from $P$ to $Q$
point $+$ vector	point	Yes	$P + \mathbf{d}$ is $P$ translated by $\mathbf{d}$
vector $\pm$ vector	vector	Yes	Combining displacements
point $+$ point	none	No	Shifts when the origin moves
$\sum c_i P_i$ with $\sum c_i = 1$	point	Yes	Midpoint, centroid

Why point + point fails: if you move the origin so that every position vector shifts by $\mathbf{c}$, then $\mathbf{p} + \mathbf{q}$ becomes $\mathbf{p} + \mathbf{q} + 2\mathbf{c}$. That describes a different location, while the true points did not move. A weighted average with weights summing to 1 shifts by exactly $\mathbf{c}$, so it describes the same location.

Useful formulas.

Midpoint of $PQ$: $M = \tfrac{1}{2}(P + Q) = P + \tfrac{1}{2}(Q - P)$.
Point a fraction $t$ of the way from $P$ to $Q$: $P + t(Q - P) = (1 - t)P + tQ$.
Centroid of triangle $ABC$: $G = \tfrac{1}{3}(A + B + C)$.

Homogeneous coordinates. Computer graphics stores the difference explicitly by appending a fourth coordinate: 1 for points and 0 for directions. A translation matrix then moves points and leaves directions unchanged.

import numpy as np

P = np.array([1.0, -2.0, 3.0])
Q = np.array([5.0, 0.0, -1.0])
print((P + Q) / 2)                # [ 3. -1.  1.]  midpoint
print(P + (Q - P) / 4)            # [ 2.  -1.5  2. ]  a quarter of the way from P to Q

# 2D homogeneous coordinates: (x, y, 1) for points, (x, y, 0) for directions
point = np.array([2.0, 1.0, 1.0])
direction = np.array([1.0, 0.0, 0.0])
T = np.array([[1, 0, 5],          # translate by (5, -3)
              [0, 1, -3],
              [0, 0, 1]])
print(T @ point)                  # [ 7. -2.  1.]  the point moves
print(T @ direction)              # [1. 0. 0.]     the direction is unchanged

Key takeaways

A point is a location, a vector is a displacement, and an origin turns each point into a position vector.
Point minus point is a vector, and point plus vector is a point; point plus point has no geometric meaning.
Weighted averages of points with weights summing to 1 (midpoints, centroids) do not depend on the origin.
Homogeneous coordinates (last entry 1 for points, 0 for directions) make the difference automatic in graphics code.

🧪 Practice

Find the midpoint of $A = (1, -2, 3)$ and $B = (5, 0, -1)$.

Find the point one third of the way from $P = (0, 0)$ to $Q = (6, 9)$, and the centroid of the triangle with vertices $(0, 0), (6, 0), (0, 3)$.

Suppose every position vector is shifted by $\mathbf{c}$ (a change of origin). Show that the midpoint formula gives the same geometric point, while $\mathbf{p} + \mathbf{q}$ does not.

Interview: In a game engine, should vertex positions and surface normals share the same data type? What goes wrong when a translated object's normals are transformed like its vertices? Hint: apply a translation to both and ask which one should change.

2.2 Vector Operations

Two operations, addition and scalar multiplication, generate almost everything else in linear algebra; this subchapter defines them, lists the rules they obey, and combines them into linear combinations.

Vector Addition and Subtraction

Why it matters. Displacements, velocities, and forces combine by addition. If you walk $(3, 1)$ and then $(1, 2)$, your total displacement is $(4, 3)$. Two forces acting on an object have the same effect as one resultant force equal to their sum.

Definition. For $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$,

$$\mathbf{u} + \mathbf{v} = (u_1 + v_1, \dots, u_n + v_n), \qquad \mathbf{u} - \mathbf{v} = (u_1 - v_1, \dots, u_n - v_n).$$

Both vectors must have the same number of components; $\mathbb{R}^2 + \mathbb{R}^3$ is undefined. The negative $-\mathbf{v} = (-v_1, \dots, -v_n)$ has the same length as $\mathbf{v}$ and points the opposite way, and $\mathbf{u} - \mathbf{v} = \mathbf{u} + (-\mathbf{v})$.

Geometry: tip-to-tail. Draw $\mathbf{u}$, then draw $\mathbf{v}$ starting at the tip of $\mathbf{u}$. The arrow from the start of $\mathbf{u}$ to the end of $\mathbf{v}$ is $\mathbf{u} + \mathbf{v}$. Doing it in the other order traces the other two sides of a parallelogram, which is the parallelogram rule.

                        * u + v
                     _/ ^
                  _/    |
       u + v   _/       |  v   (drawn starting at the tip of u)
            _/          |
         _/             |
        O-------------->*
                u

Geometry of subtraction. In the parallelogram spanned by $\mathbf{u}$ and $\mathbf{v}$, one diagonal is $\mathbf{u} + \mathbf{v}$. The other diagonal, from the tip of $\mathbf{v}$ to the tip of $\mathbf{u}$, is $\mathbf{u} - \mathbf{v}$: it is "what you add to $\mathbf{v}$ to reach $\mathbf{u}$".

         v *-----------------* u + v
           ^ \               ^
           |   \             |
         v |     \  u - v    | v
           |       \         |
           |         v       |
           O---------------->* u
                   u

Example. With $\mathbf{u} = (3, 1)$ and $\mathbf{v} = (1, 2)$: $\mathbf{u} + \mathbf{v} = (4, 3)$ and $\mathbf{u} - \mathbf{v} = (2, -1)$.

Relative motion. If car $A$ has velocity $\mathbf{v}_A$ and car $B$ has velocity $\mathbf{v}_B$, then $A$'s velocity as seen from $B$ is $\mathbf{v}_A - \mathbf{v}_B$.

import numpy as np

u = np.array([3, 1])
v = np.array([1, 2])
print(u + v)                      # [4 3]
print(u - v)                      # [ 2 -1]  arrow from the tip of v to the tip of u

try:
    u + np.array([1, 2, 3])       # R^2 + R^3 is undefined
except ValueError as e:
    print("error:", e)

print(u + 10)                     # [13 11]  NumPy "broadcasting": convenient, but NOT a vector operation

The last line is a common pitfall. NumPy accepts it and adds 10 to each component, but "vector plus number" is not defined in linear algebra. It is shorthand for $\mathbf{u} + 10 \cdot (1, 1)$.

Key takeaways

Addition and subtraction are componentwise and require the same number of components.
Geometrically, $\mathbf{u} + \mathbf{v}$ is tip-to-tail, and $\mathbf{u} + \mathbf{v}$ and $\mathbf{u} - \mathbf{v}$ are the two diagonals of the parallelogram.
$\mathbf{u} - \mathbf{v}$ is the arrow from the tip of $\mathbf{v}$ to the tip of $\mathbf{u}$.
Libraries may "broadcast" a scalar over a vector; that is a convenience, not vector addition.

🧪 Practice

For $\mathbf{u} = (2, -1, 4)$ and $\mathbf{v} = (-3, 0, 5)$, compute $\mathbf{u} + \mathbf{v}$, $\mathbf{u} - \mathbf{v}$, and $\mathbf{v} - \mathbf{u}$. How are the last two related?

A plane's velocity relative to the air is $(400, 0)$ km/h, and the wind is $(0, -50)$ km/h. Find its velocity relative to the ground. What air velocity would give a ground velocity of exactly $(400, 0)$?

Solve for $\mathbf{x}$: $\mathbf{x} + (1, 2, 3) = (4, 0, -1)$.

Interview: Given points $P_1, \dots, P_k$, what is the sum of the displacement vectors $P_1 \to P_2,\ P_2 \to P_3,\ \dots,\ P_k \to P_1$, and why? Hint: write each displacement as a difference of points and look for cancellation.

Scalar Multiplication

Why it matters. Scaling a vector stretches, shrinks, or reverses it while keeping it on the same line. Doubling a velocity, converting a displacement from meters to centimeters, and reversing a force are all scalar multiplications.

Definition. For a scalar $c \in \mathbb{R}$ and $\mathbf{v} \in \mathbb{R}^n$,

$$c\mathbf{v} = (c v_1, c v_2, \dots, c v_n).$$

Geometry.

Scalar $c$	Effect on the arrow
$c > 1$	Stretch, same direction
$0 < c < 1$	Shrink, same direction
$c = 0$	Collapse to $\mathbf{0}$
$-1 < c < 0$	Shrink and reverse
$c = -1$	Reverse, same length ($-\mathbf{v}$)
$c < -1$	Stretch and reverse

       2v :  O------------------->
        v :  O--------->
     0.5v :  O---->
       -v :  <---------O
   every multiple of v lies on one line through the origin

Parallel vectors. Nonzero vectors $\mathbf{u}$ and $\mathbf{v}$ are parallel if $\mathbf{u} = c\mathbf{v}$ for some scalar $c$. They point the same way if $c > 0$ and opposite ways if $c < 0$. By convention, $\mathbf{0} = 0\mathbf{v}$ counts as parallel to every vector. All multiples of a nonzero $\mathbf{v}$ form a line through the origin, the first example of a span.

import numpy as np

v = np.array([2, -1])
for c in (2, 0.5, -1, 0):
    print(c, c * v)
# 2 [ 4 -2]          stretched
# 0.5 [ 1.  -0.5]    shrunk
# -1 [-2  1]         reversed
# 0 [0 0]            collapsed to the zero vector

def parallel(u, v):
    """True if u and v lie on one line through the origin."""
    return np.linalg.matrix_rank(np.vstack([u, v])) <= 1   # at most one independent row

print(parallel(np.array([1, 2, 3]), np.array([-2, -4, -6])))   # True  (c = -2)
print(parallel(np.array([1, 2, 3]), np.array([2, 4, 7])))      # False

Testing parallelism by dividing components ($u_1 / v_1 = u_2 / v_2 = \cdots$) breaks when a component is zero. The rank test above handles every case. Rank is defined in Chapter 3.

Key takeaways

$c\mathbf{v}$ multiplies every component by $c$; geometrically it stretches by $\lvert c \rvert$ and flips when $c < 0$.
Nonzero vectors are parallel exactly when one is a scalar multiple of the other.
All multiples of a nonzero vector form a line through the origin.
Avoid ratio-based parallel tests in code; zero components break them.

🧪 Practice

For $\mathbf{u} = (4, -2, 6)$, compute $3\mathbf{u}$, $-\tfrac{1}{2}\mathbf{u}$, and $0\mathbf{u}$.

Find all $c$ with $c(1, 2) = (-3, -6)$, and all $c$ with $c(1, 2) = (3, 5)$.

Are $(2, -4, 6)$ and $(-3, 6, -9)$ parallel? If so, do they point the same way or opposite ways?

Interview: Prove that if $c\mathbf{v} = \mathbf{0}$, then $c = 0$ or $\mathbf{v} = \mathbf{0}$. Hint: if $c \ne 0$, multiply both sides by $1/c$; note which field property this uses.

Algebraic Properties of Vector Operations

Why it matters. These rules let you manipulate vector expressions the way you manipulate numbers: regroup, expand, and cancel. In Chapter 5 they become the axioms of an abstract vector space. Anything that satisfies them, including polynomials, matrices, and functions, automatically inherits every theorem proved from them.

The eight properties. For all $\mathbf{u}, \mathbf{v}, \mathbf{w} \in \mathbb{R}^n$ and scalars $c, d$:

#	Property	Statement
1	Commutativity of addition	$\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$
2	Associativity of addition	$(\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$
3	Additive identity	$\mathbf{u} + \mathbf{0} = \mathbf{u}$
4	Additive inverse	$\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$
5	Distributivity over vector addition	$c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$
6	Distributivity over scalar addition	$(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$
7	Compatibility of scaling	$c(d\mathbf{u}) = (cd)\mathbf{u}$
8	Scalar identity	$1\mathbf{u} = \mathbf{u}$

Why they hold. Each property reduces to the same property of real numbers, applied one component at a time. For property 5:

$$c(\mathbf{u} + \mathbf{v}) = \big(c(u_1 + v_1), \dots, c(u_n + v_n)\big) = (cu_1 + cv_1, \dots, cu_n + cv_n) = c\mathbf{u} + c\mathbf{v}.$$

Geometrically, property 1 is the parallelogram rule (both orders reach the same corner), and property 5 says that scaling a whole triangle of arrows scales each side.

Consequences proved from the axioms alone. These follow without looking at components, so they hold in every vector space:

$0\mathbf{u} = \mathbf{0}$. Proof: $0\mathbf{u} = (0 + 0)\mathbf{u} = 0\mathbf{u} + 0\mathbf{u}$; add $-(0\mathbf{u})$ to both sides.
$(-1)\mathbf{u} = -\mathbf{u}$ and $c\mathbf{0} = \mathbf{0}$.
Cancellation: $\mathbf{u} + \mathbf{w} = \mathbf{v} + \mathbf{w} \implies \mathbf{u} = \mathbf{v}$.

What is missing. There is no general way to multiply two vectors to get another vector in $\mathbb{R}^n$, and there is no division by a vector. The dot product (Section 2.3) returns a scalar, and the cross product (Section 2.4) exists only in $\mathbb{R}^3$.

import numpy as np

rng = np.random.default_rng(0)
u, v, w = rng.normal(size=(3, 4))      # three random vectors in R^4
c, d = rng.normal(size=2)              # two random scalars

checks = {
    "commutative":      np.allclose(u + v, v + u),
    "associative":      np.allclose((u + v) + w, u + (v + w)),
    "distrib. vectors": np.allclose(c * (u + v), c * u + c * v),
    "distrib. scalars": np.allclose((c + d) * u, c * u + d * u),
    "compatibility":    np.allclose(c * (d * u), (c * d) * u),
}
print(all(checks.values()))            # True

# In floating point, associativity holds only approximately
a, b, e = np.array([0.1]), np.array([0.2]), np.array([0.3])
print((a + b) + e == a + (b + e))      # [False]: 0.6000000000000001 vs 0.6

Use np.allclose instead of == when comparing floating-point vectors. The algebra is exact, but the hardware rounds.

Key takeaways

Vector operations satisfy eight rules inherited componentwise from real arithmetic.
These rules are the axioms of a vector space, and anything that satisfies them inherits the theory.
Facts like $0\mathbf{u} = \mathbf{0}$ and cancellation can be proved from the axioms without components.
Floating-point arithmetic only approximates these laws, so compare with a tolerance.

🧪 Practice

Prove $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$ for $\mathbf{u} \in \mathbb{R}^n$ using components.

Using only the eight properties (no components), prove $(-1)\mathbf{u} = -\mathbf{u}$.

Solve $2\mathbf{x} + 3(1, -1, 0) = \mathbf{x} - (0, 2, 4)$ for $\mathbf{x}$, naming the property used at each step.

Interview: On the positive reals, define "addition" as $x \oplus y = xy$ and "scaling" as $c \odot x = x^c$. Do all eight properties hold? Hint: first work out which positive number plays the role of the zero vector, and what the "negative" of $x$ is.

Linear Combinations of Vectors

Why it matters. Linear combinations are the central construction of linear algebra. Solving a linear system, describing a plane, multiplying a matrix by a vector, and changing coordinates all ask the same question: which weighted sums of some given vectors produce a target? An analogy: the vectors are ingredients, the weights are amounts, and the combination is the dish.

Definition. Given vectors $\mathbf{v}_1, \dots, \mathbf{v}_k$ and scalars $c_1, \dots, c_k$, the vector

$$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k$$

is a linear combination of $\mathbf{v}_1, \dots, \mathbf{v}_k$ with weights $c_1, \dots, c_k$.

Example. Is $\mathbf{b} = (7, 4)$ a linear combination of $\mathbf{v}_1 = (1, 1)$ and $\mathbf{v}_2 = (1, -1)$? We need $c_1(1, 1) + c_2(1, -1) = (7, 4)$, that is,

$$c_1 + c_2 = 7, \qquad c_1 - c_2 = 4 \implies c_1 = \tfrac{11}{2},\ c_2 = \tfrac{3}{2}.$$

So the question "is $\mathbf{b}$ a combination?" is a linear system, which Chapter 3 solves in general.

Geometry: what combinations can reach. The integer combinations $c_1 \mathbf{u} + c_2 \mathbf{v}$ form a slanted grid. Allowing all real weights fills in the whole plane between the grid lines.

              *-------*-------*-------*
             /       /       /       /
            *-------*-------*-------*
           /       /  b    /       /       b = 2u + v
          *-------*---*---*-------*
         /       /       /
        ^ v     /       /
       /       /       /
      O------>*-------*
          u

Vectors	All linear combinations (the span)
One nonzero $\mathbf{v}$	A line through the origin
Two non-parallel vectors in $\mathbb{R}^2$	All of $\mathbb{R}^2$
Two non-parallel vectors in $\mathbb{R}^3$	A plane through the origin
Three vectors in $\mathbb{R}^3$, none in the plane of the others	All of $\mathbb{R}^3$

Restricted combinations. Restricting the weights produces familiar shapes:

Weights	Name	Combinations of two points $P, Q$
Any real	Linear	The plane through $O$, $P$, $Q$
Sum to 1	Affine	The line through $P$ and $Q$
Sum to 1, all $\ge 0$	Convex	The segment from $P$ to $Q$

import numpy as np

v1 = np.array([1, 1])
v2 = np.array([1, -1])
b = np.array([7, 4])

V = np.column_stack([v1, v2])           # vectors as the columns of a matrix
c = np.linalg.solve(V, b)               # weights with c1*v1 + c2*v2 = b
print(c)                                # [5.5 1.5]
print(c[0] * v1 + c[1] * v2)            # [7. 4.]  check

# In R^3, two vectors reach only a plane: is b3 on it?
W = np.column_stack([[1, 0, 1], [0, 1, 1]])   # combinations look like (a, b, a + b)
b3 = np.array([2, 3, 4])                      # would need a + b = 5, not 4
coef, residual, *_ = np.linalg.lstsq(W, b3, rcond=None)
print(residual)                         # [0.33333333]  nonzero: b3 is NOT a linear combination

Key takeaways

A linear combination is a weighted sum $c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k$.
Deciding whether $\mathbf{b}$ is a linear combination of given vectors means solving a linear system for the weights.
The combinations of one vector form a line, and those of two non-parallel vectors form a plane through the origin.
Requiring the weights to sum to 1 gives lines and planes through the given points; also requiring them to be nonnegative gives segments and triangles.

🧪 Practice

Compute $2(1, 0, -1) - 3(2, 1, 1) + (0, 4, 5)$.

Write $(5, 1)$ as a linear combination of $(1, 2)$ and $(3, -1)$.

Decide whether $(1, 2, 3)$ and $(1, 2, 4)$ are linear combinations of $(1, 0, 1)$ and $(0, 1, 1)$. Describe the result geometrically.

Interview: Given $k$ vectors in $\mathbb{R}^n$, how would you decide whether every vector in $\mathbb{R}^n$ is a linear combination of them, and what is the smallest $k$ for which this is possible? Hint: count how many new independent directions each added vector can contribute.

2.3 Dot Product and Geometry

The dot product turns two vectors into a single number, and that number encodes length, angle, and perpendicularity; this subchapter defines it and derives the geometric tools built on it.

Dot Product Definition

Why it matters. Addition and scaling cannot measure anything: they say nothing about how long a vector is or how two vectors relate. The dot product adds measurement. A shopping analogy: if $\mathbf{q}$ lists the quantities you buy and $\mathbf{p}$ lists the prices, the total bill is $q_1 p_1 + q_2 p_2 + \cdots$, which is the dot product $\mathbf{q} \cdot \mathbf{p}$.

Definition. For $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$,

$$\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = \sum_{i=1}^{n} u_i v_i.$$

The result is a scalar, not a vector. In matrix notation (Chapter 4) it is written $\mathbf{u}^T \mathbf{v}$.

Example. $(1, 2, -1) \cdot (3, 0, 4) = 3 + 0 - 4 = -1$.

Geometric meaning. Section "Angle Between Vectors" proves that

$$\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert , \lVert \mathbf{v} \rVert \cos\theta,$$

where $\theta$ is the angle between the vectors. So the sign of the dot product tells you about the angle:

$\mathbf{u} \cdot \mathbf{v}$	Angle $\theta$	Meaning
$> 0$	$0 \le \theta < 90^\circ$	Roughly the same direction
$= 0$	$\theta = 90^\circ$	Perpendicular
$< 0$	$90^\circ < \theta \le 180^\circ$	Roughly opposite directions

Another reading: $\mathbf{u} \cdot \mathbf{v}$ equals the length of $\mathbf{v}$ times the signed length of $\mathbf{u}$'s "shadow" on the line of $\mathbf{v}$ (see "Vector Projection").

Properties. For all vectors $\mathbf{u}, \mathbf{v}, \mathbf{w}$ and scalars $c$:

Property	Statement
Commutative	$\mathbf{u} \cdot \mathbf{v} = \mathbf{v} \cdot \mathbf{u}$
Distributive	$\mathbf{u} \cdot (\mathbf{v} + \mathbf{w}) = \mathbf{u} \cdot \mathbf{v} + \mathbf{u} \cdot \mathbf{w}$
Scalars factor out	$(c\mathbf{u}) \cdot \mathbf{v} = c(\mathbf{u} \cdot \mathbf{v}) = \mathbf{u} \cdot (c\mathbf{v})$
Positivity	$\mathbf{v} \cdot \mathbf{v} \ge 0$, with equality only for $\mathbf{v} = \mathbf{0}$

What does not hold.

No associativity: $(\mathbf{u} \cdot \mathbf{v})\mathbf{w}$ is a multiple of $\mathbf{w}$, and $\mathbf{u}(\mathbf{v} \cdot \mathbf{w})$ is a multiple of $\mathbf{u}$. The expression $\mathbf{u} \cdot \mathbf{v} \cdot \mathbf{w}$ is meaningless.
No cancellation: $\mathbf{u} \cdot \mathbf{v} = \mathbf{u} \cdot \mathbf{w}$ does not imply $\mathbf{v} = \mathbf{w}$. It only says $\mathbf{u} \cdot (\mathbf{v} - \mathbf{w}) = 0$.

import numpy as np

qty    = np.array([3, 1, 2])            # 3 apples, 1 bread, 2 milk
prices = np.array([0.5, 2.0, 1.25])     # price per item
print(qty @ prices)                     # 6.0 = 3*0.5 + 1*2.0 + 2*1.25

u = np.array([1, 2, -1])
v = np.array([3, 0, 4])
print(np.dot(u, v), u @ v)                       # -1 -1   two spellings of the same thing
print(sum(int(a * b) for a, b in zip(u, v)))     # -1      the definition, written out

Key takeaways

$\mathbf{u} \cdot \mathbf{v} = \sum u_i v_i$ takes two vectors of the same size and returns a scalar.
$\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \cos\theta$, so its sign tells you whether the angle is acute, right, or obtuse.
It is commutative, distributive, and positive on nonzero vectors.
It is not associative, and $\mathbf{u}$ cannot be cancelled from $\mathbf{u} \cdot \mathbf{v} = \mathbf{u} \cdot \mathbf{w}$.

🧪 Practice

Compute $(2, -1, 3) \cdot (4, 5, -1)$ and $(1, 1, 1, 1) \cdot (1, 2, 3, 4)$. What does the first result say geometrically?

Find $k$ such that $(k, 2, -1) \cdot (3, k, 4) = 0$.

Give an example where $\mathbf{u} \cdot \mathbf{v} = \mathbf{u} \cdot \mathbf{w}$ but $\mathbf{v} \ne \mathbf{w}$, and state what the equation does tell you.

Interview: A recommender system scores how well an item suits a user by the dot product of their embedding vectors. Why is this a natural score, and what is its weakness compared with cosine similarity? Hint: see what happens to the score when an item's vector is scaled by 10.

Norm and Length of a Vector

Why it matters. Distances are everywhere: how far a robot is from its goal, how different two images are, how large an error is. The dot product of a vector with itself gives its length, and length gives distance.

Definition. The norm (Euclidean length) of $\mathbf{v} \in \mathbb{R}^n$ is

$$\lVert \mathbf{v} \rVert = \sqrt{\mathbf{v} \cdot \mathbf{v}} = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}.$$

In $\mathbb{R}^2$ this is the Pythagorean theorem. In $\mathbb{R}^3$ you apply it twice: first across the floor of a box, then up its height.

            +-----------* (3, 4, 12)
           /|          /|
          / |         / |        floor diagonal: sqrt(3^2 + 4^2) = 5
         +-----------+  | 12     space diagonal: sqrt(5^2 + 12^2) = 13
         |  |     _/ |  |
         |  +--_/----|--+
         | /_/  5    | /  4
         O-----------+
              3

Properties.

$\lVert \mathbf{v} \rVert \ge 0$, and $\lVert \mathbf{v} \rVert = 0$ only for $\mathbf{v} = \mathbf{0}$.
$\lVert c\mathbf{v} \rVert = \lvert c \rvert , \lVert \mathbf{v} \rVert$. Scaling by $-3$ triples the length.
$\lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert$ (the triangle inequality, proved later in this subchapter).

Distance. The distance between $\mathbf{u}$ and $\mathbf{v}$ is the length of the arrow between them:

$$d(\mathbf{u}, \mathbf{v}) = \lVert \mathbf{u} - \mathbf{v} \rVert.$$

Unit vectors and normalization. A unit vector has norm 1. Dividing a nonzero vector by its norm keeps its direction and sets its length to 1:

$$\hat{\mathbf{v}} = \frac{\mathbf{v}}{\lVert \mathbf{v} \rVert}.$$

This separates a vector into "how much" ($\lVert \mathbf{v} \rVert$) and "which way" ($\hat{\mathbf{v}}$): $\mathbf{v} = \lVert \mathbf{v} \rVert , \hat{\mathbf{v}}$.

Other norms. The Euclidean norm is not the only reasonable notion of size. Chapter 9 generalizes it, and these alternatives are common in practice:

Norm	Formula	Unit "circle" in $\mathbb{R}^2$	Typical use
$\ell^2$ (Euclidean)	$\sqrt{\sum v_i^2}$	Circle	Geometry, least squares
$\ell^1$ (Manhattan)	$\sum \lvert v_i \rvert$	Diamond	Grid travel, sparse models (lasso)
$\ell^\infty$ (max)	$\max_i \lvert v_i \rvert$	Square	Worst-case error, chessboard moves

import math
import numpy as np

v = np.array([3.0, 4.0, 12.0])
print(np.linalg.norm(v))                 # 13.0 = sqrt(9 + 16 + 144)
unit = v / np.linalg.norm(v)
print(unit.round(4), np.linalg.norm(unit))   # [0.2308 0.3077 0.9231] 1.0

p, q = np.array([1.0, 2.0]), np.array([4.0, 6.0])
print(np.linalg.norm(p - q))             # 5.0  distance between two points

print(np.linalg.norm(v, 1), np.linalg.norm(v, np.inf))   # 19.0 12.0  (l1 and l-infinity norms)

big = np.array([1e200, 1e200])
print(np.sqrt(np.sum(big**2)))           # inf (with an overflow warning): the squares overflow first
print(math.hypot(*big))                  # 1.414213562373095e+200: rescales internally to avoid overflow

Key takeaways

$\lVert \mathbf{v} \rVert = \sqrt{\mathbf{v} \cdot \mathbf{v}}$ is the Pythagorean length in any dimension.
Distance is the norm of the difference: $d(\mathbf{u}, \mathbf{v}) = \lVert \mathbf{u} - \mathbf{v} \rVert$.
$\mathbf{v} / \lVert \mathbf{v} \rVert$ is a unit vector with the same direction as $\mathbf{v}$.
Other norms ($\ell^1$, $\ell^\infty$) measure size differently and suit different problems.

🧪 Practice

Compute $\lVert (1, -2, 2) \rVert$ and the unit vector in the direction of $(1, -2, 2)$.

Find the distance between $(1, 0, -1, 2)$ and $(3, 1, 1, 0)$ in $\mathbb{R}^4$.

Prove $\lVert c\mathbf{v} \rVert = \lvert c \rvert , \lVert \mathbf{v} \rVert$ from the definition.

Interview: Sketch the set of vectors in $\mathbb{R}^2$ with norm 1 in each of the $\ell^1$, $\ell^2$, and $\ell^\infty$ norms. Which norm measures travel distance for a delivery robot in a city with a rectangular street grid? Hint: think about which directions the robot is allowed to move in.

Angle Between Vectors

Why it matters. Angle measures how similar two directions are, independent of length. Document similarity, the brightness of a surface in 3D rendering (the angle between the surface and the light), and correlation in statistics are all angle computations.

Deriving the formula. Place $\mathbf{u}$ and $\mathbf{v}$ tail to tail with angle $\theta$ between them. The third side of the triangle is $\mathbf{u} - \mathbf{v}$.

            * v
           / \
          /   \
     v   /     \  u - v
        /       \
       / θ       \
      O-----------* u
            u

The law of cosines gives $\lVert \mathbf{u} - \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2 - 2\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \cos\theta$. Expanding the left side with the dot product gives

$$(\mathbf{u} - \mathbf{v}) \cdot (\mathbf{u} - \mathbf{v}) = \lVert \mathbf{u} \rVert^2 - 2,\mathbf{u} \cdot \mathbf{v} + \lVert \mathbf{v} \rVert^2.$$

Comparing the two expressions:

$$\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert , \lVert \mathbf{v} \rVert \cos\theta.$$

Definition in $\mathbb{R}^n$. In higher dimensions there is no picture to measure, so this formula becomes the definition. For nonzero $\mathbf{u}, \mathbf{v}$, the angle $\theta \in [0, \pi]$ is given by

$$\cos\theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{u} \rVert , \lVert \mathbf{v} \rVert}.$$

For this to make sense the right side must lie in $[-1, 1]$. The Cauchy-Schwarz inequality, proved later in this subchapter, guarantees it.

Example. For $\mathbf{u} = (1, 0, 1)$ and $\mathbf{v} = (0, 1, 1)$: $\mathbf{u} \cdot \mathbf{v} = 1$ and $\lVert \mathbf{u} \rVert = \lVert \mathbf{v} \rVert = \sqrt{2}$, so $\cos\theta = \tfrac{1}{2}$ and $\theta = 60^\circ$.

Cosine similarity. The quantity $\cos\theta$ is called cosine similarity in data science:

$\cos\theta$	Interpretation
$1$	Same direction (one is a positive multiple of the other)
$0$	Unrelated (perpendicular)
$-1$	Opposite directions

Because it ignores length, a short document and a long document on the same topic score as similar.

import numpy as np

def angle(u, v, degrees=True):
    cos_t = u @ v / (np.linalg.norm(u) * np.linalg.norm(v))
    cos_t = np.clip(cos_t, -1.0, 1.0)    # rounding can push |cos| just above 1, and arccos would return nan
    t = np.arccos(cos_t)
    return np.degrees(t) if degrees else t

print(round(angle(np.array([1, 0, 1]), np.array([0, 1, 1])), 6))   # 60.0
print(angle(np.array([1, 2]), np.array([-2, 1])))                  # 90.0
print(angle(np.array([1, 1]), np.array([-3, -3])))                 # 180.0

Key takeaways

The law of cosines gives $\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \cos\theta$ in 2D and 3D.
In $\mathbb{R}^n$, this formula defines the angle, with $\theta \in [0, \pi]$.
Cosine similarity compares directions and ignores lengths.
Clip the cosine to $[-1, 1]$ before calling arccos in floating-point code.

🧪 Practice

Find the angle between $(2, -1, 1)$ and $(1, 1, 2)$.

Find the angle between the main diagonal $(1, 1, 1)$ of a cube and one of its edges, $(1, 0, 0)$.

Show that the angle between $(1, 1)$ and $(1, \sqrt{3})$ is $15^\circ$ by computing $\cos\theta$ and comparing with $\cos 15^\circ = \tfrac{\sqrt{6} + \sqrt{2}}{4}$.

Interview: Pick two random vectors in $\mathbb{R}^{1000}$ whose entries are independently $+1$ or $-1$. Roughly what angle do you expect between them, and why? Hint: the dot product is a sum of 1000 random $\pm 1$ terms; compare its typical size, about $\sqrt{1000}$, with the product of the two norms.

Orthogonality

Why it matters. Perpendicular directions do not interfere with each other. Moving north does not change how far east you are. That independence makes computations easier: coordinates along perpendicular axes can be computed separately. Least squares, Fourier series, and the Gram-Schmidt process (Chapter 10) all rely on it.

Definition. Vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal, written $\mathbf{u} \perp \mathbf{v}$, if

$$\mathbf{u} \cdot \mathbf{v} = 0.$$

In 2D and 3D this means perpendicular. In $\mathbb{R}^n$ it is the definition of perpendicular. The zero vector is orthogonal to every vector.

Pythagorean theorem in $\mathbb{R}^n$. Expanding $\lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + 2,\mathbf{u} \cdot \mathbf{v} + \lVert \mathbf{v} \rVert^2$ shows

$$\mathbf{u} \perp \mathbf{v} \iff \lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + \lVert \mathbf{v} \rVert^2.$$

          *
          |\
          |  \
        v |    \  u + v
          |      \
          |_       \
          O---------*
               u          u . v = 0   <=>   ||u + v||^2 = ||u||^2 + ||v||^2

Orthogonal and orthonormal sets. A set of vectors is orthogonal if every pair is orthogonal, and orthonormal if, in addition, every vector has norm 1. The standard basis $\mathbf{e}_1, \dots, \mathbf{e}_n$ is orthonormal. Chapter 10 shows that any basis can be converted into an orthonormal one.

Finding perpendicular vectors.

In $\mathbb{R}^2$: $(a, b) \perp (-b, a)$, a $90^\circ$ rotation.
In $\mathbb{R}^3$: the cross product $\mathbf{u} \times \mathbf{v}$ (Section 2.4) is perpendicular to both $\mathbf{u}$ and $\mathbf{v}$.
In $\mathbb{R}^n$: the vectors orthogonal to a fixed $\mathbf{a} \ne \mathbf{0}$ are the solutions of the single linear equation $a_1 x_1 + \cdots + a_n x_n = 0$. They form a hyperplane through the origin (Section 2.5).

import numpy as np

u = np.array([1, 2, 2])
v = np.array([2, 1, -2])
w = np.array([2, -2, 1])
print(u @ v, u @ w, v @ w)              # 0 0 0: pairwise orthogonal

Q = np.vstack([u, v, w]) / 3            # each has length 3; dividing makes them unit vectors
print(np.allclose(Q @ Q.T, np.eye(3)))  # True: the rows form an orthonormal set

# Pythagoras holds for orthogonal vectors
print(np.linalg.norm(u + v)**2, np.linalg.norm(u)**2 + np.linalg.norm(v)**2)   # both about 18

# With floats, test "close to zero", not "== 0"
a = np.array([0.1, 0.2, 0.3])
b = np.array([0.5, -0.7, 0.3])          # exactly orthogonal on paper: 0.05 - 0.14 + 0.09 = 0
print(a @ b, np.isclose(a @ b, 0.0))    # 1.3877787807814457e-17 True: rounding error, not a real angle

Key takeaways

$\mathbf{u} \perp \mathbf{v}$ means $\mathbf{u} \cdot \mathbf{v} = 0$; the zero vector is orthogonal to everything.
The Pythagorean theorem holds in every $\mathbb{R}^n$, and it holds exactly for orthogonal pairs.
Orthonormal sets (pairwise orthogonal unit vectors) are the most convenient coordinate systems.
The vectors orthogonal to a nonzero $\mathbf{a}$ form a hyperplane through the origin.

🧪 Practice

Check which pairs among $(1, -1, 0)$, $(1, 1, 1)$, $(1, 1, -2)$ are orthogonal, then normalize them to get an orthonormal set.

Find all vectors orthogonal to both $(1, 0, 1)$ and $(0, 1, 1)$, and describe the set geometrically.

Prove that if $\mathbf{u} \perp \mathbf{v}$, then $\lVert \mathbf{u} - \mathbf{v} \rVert = \lVert \mathbf{u} + \mathbf{v} \rVert$. What does this say about the diagonals of a rectangle?

Interview: Prove that if $\mathbf{v}$ is orthogonal to every vector in $\mathbb{R}^n$, then $\mathbf{v} = \mathbf{0}$. Hint: you only need to test against one carefully chosen vector.

Cauchy-Schwarz Inequality

Why it matters. Without this inequality, the angle formula $\cos\theta = \mathbf{u} \cdot \mathbf{v} / (\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert)$ could produce a "cosine" larger than 1. Cauchy-Schwarz guarantees angles make sense in every dimension. It also proves the triangle inequality, bounds correlations to $[-1, 1]$, and is one of the most frequently used inequalities in mathematics.

Statement. For all $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$,

$$\lvert \mathbf{u} \cdot \mathbf{v} \rvert \le \lVert \mathbf{u} \rVert , \lVert \mathbf{v} \rVert,$$

with equality exactly when one vector is a scalar multiple of the other. In components:

$$\Big(\sum_{i=1}^n u_i v_i\Big)^2 \le \Big(\sum_{i=1}^n u_i^2\Big)\Big(\sum_{i=1}^n v_i^2\Big).$$

Intuition. The shadow of $\mathbf{u}$ on the line of $\mathbf{v}$ cannot be longer than $\mathbf{u}$ itself, and the shadow has length $\lvert \mathbf{u} \cdot \mathbf{v} \rvert / \lVert \mathbf{v} \rVert$.

            * u
           /|
          / |
         /  |
        O---+-----------> v
        |___|
        shadow: |u . v| / ||v||   <=   ||u||

Proof. If $\mathbf{v} = \mathbf{0}$, both sides are 0. Otherwise, consider the function of a real variable $t$:

$$f(t) = \lVert \mathbf{u} + t\mathbf{v} \rVert^2 = \lVert \mathbf{v} \rVert^2 t^2 + 2(\mathbf{u} \cdot \mathbf{v}), t + \lVert \mathbf{u} \rVert^2.$$

As a squared length, $f(t) \ge 0$ for every $t$. A quadratic with positive leading coefficient that is never negative has at most one real root, so its discriminant is $\le 0$:

$$4(\mathbf{u} \cdot \mathbf{v})^2 - 4\lVert \mathbf{u} \rVert^2 \lVert \mathbf{v} \rVert^2 \le 0.$$

Taking square roots gives the inequality. Equality means the discriminant is 0, so $f$ has a root $t_0$. Then $\lVert \mathbf{u} + t_0 \mathbf{v} \rVert = 0$, so $\mathbf{u} = -t_0 \mathbf{v}$, and the vectors are parallel.

Applications.

Application	Choice of vectors	Consequence
Angles in $\mathbb{R}^n$	Any nonzero $\mathbf{u}, \mathbf{v}$	$-1 \le \cos\theta \le 1$
Triangle inequality	$\mathbf{u}, \mathbf{v}$	$\lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert$
Mean vs. quadratic mean	$\mathbf{v} = (1, \dots, 1)$	$(x_1 + \cdots + x_n)^2 \le n(x_1^2 + \cdots + x_n^2)$
Statistics	Mean-centered data vectors	Correlation $r \in [-1, 1]$
Optimization	Fixed $\mathbf{a}$, unit $\mathbf{x}$	$\max_{\lVert \mathbf{x} \rVert = 1} \mathbf{a} \cdot \mathbf{x} = \lVert \mathbf{a} \rVert$

import numpy as np

rng = np.random.default_rng(42)
for _ in range(10_000):
    u, v = rng.normal(size=(2, 6))
    assert abs(u @ v) <= np.linalg.norm(u) * np.linalg.norm(v) + 1e-12   # never fails

u = np.array([1.0, -2.0, 3.0])
w = -4 * u                                          # parallel: the equality case
print(abs(u @ w), np.linalg.norm(u) * np.linalg.norm(w))   # both 56 (up to rounding)

# Pearson correlation = cosine of the angle between mean-centered data vectors
x = np.array([1.0, 2.0, 3.0, 4.0])
y = np.array([2.0, 4.1, 5.9, 8.2])
xc, yc = x - x.mean(), y - y.mean()
r = xc @ yc / (np.linalg.norm(xc) * np.linalg.norm(yc))
print(np.isclose(r, np.corrcoef(x, y)[0, 1]))       # True, and |r| <= 1 by Cauchy-Schwarz

Key takeaways

$\lvert \mathbf{u} \cdot \mathbf{v} \rvert \le \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert$, with equality exactly for parallel vectors.
It guarantees that the angle formula yields a valid cosine in every dimension.
The standard proof uses the fact that $\lVert \mathbf{u} + t\mathbf{v} \rVert^2 \ge 0$ has a non-positive discriminant.
It implies the triangle inequality, bounds on correlation, and "the maximum of $\mathbf{a} \cdot \mathbf{x}$ over unit vectors is $\lVert \mathbf{a} \rVert$".

🧪 Practice

Verify the inequality for $\mathbf{u} = (1, 2, 3)$ and $\mathbf{v} = (4, 5, 6)$, and compute how close it is to equality.

Apply Cauchy-Schwarz with $\mathbf{v} = (1, \dots, 1)$ to prove $(x_1 + \cdots + x_n)^2 \le n(x_1^2 + \cdots + x_n^2)$. When does equality hold?

Find the maximum of $2x + 3y - z$ subject to $x^2 + y^2 + z^2 = 1$, and the point where it is attained.

Interview: Where does the discriminant proof use $\mathbf{v} \ne \mathbf{0}$, and why does the argument fail without it? Hint: what kind of function is $f(t)$ when $\mathbf{v} = \mathbf{0}$?

Triangle Inequality

Why it matters. "A detour is never shorter than the direct route." This property makes the norm a genuine measure of distance. It underpins error bounds in numerical computation (errors add up at most linearly) and lets search algorithms skip candidates without computing their distances.

Statement. For all $\mathbf{u}, \mathbf{v} \in \mathbb{R}^n$,

$$\lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert,$$

with equality exactly when one vector is a nonnegative multiple of the other (they point the same way).

                      *
                   _/ ^
      ||u + v||  _/   |  ||v||
              _/      |
           _/         |
          O---------->*
             ||u||          the direct side is never longer than the other two combined

Proof. Expand, then apply Cauchy-Schwarz to the middle term:

$$\lVert \mathbf{u} + \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 + 2,\mathbf{u} \cdot \mathbf{v} + \lVert \mathbf{v} \rVert^2 \le \lVert \mathbf{u} \rVert^2 + 2\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert + \lVert \mathbf{v} \rVert^2 = \big(\lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert\big)^2.$$

Both sides are nonnegative, so taking square roots preserves the inequality.

Useful variants.

For distances: $d(\mathbf{x}, \mathbf{z}) \le d(\mathbf{x}, \mathbf{y}) + d(\mathbf{y}, \mathbf{z})$ (substitute $\mathbf{u} = \mathbf{x} - \mathbf{y}$, $\mathbf{v} = \mathbf{y} - \mathbf{z}$).
Reverse triangle inequality: $\big\lvert \lVert \mathbf{u} \rVert - \lVert \mathbf{v} \rVert \big\rvert \le \lVert \mathbf{u} - \mathbf{v} \rVert$. Vectors that are close have similar lengths.
Many terms: $\lVert \mathbf{v}_1 + \cdots + \mathbf{v}_k \rVert \le \lVert \mathbf{v}_1 \rVert + \cdots + \lVert \mathbf{v}_k \rVert$.

Metric properties. With $d(\mathbf{u}, \mathbf{v}) = \lVert \mathbf{u} - \mathbf{v} \rVert$, $\mathbb{R}^n$ is a metric space:

Property	Statement
Nonnegativity	$d(\mathbf{u}, \mathbf{v}) \ge 0$
Identity	$d(\mathbf{u}, \mathbf{v}) = 0 \iff \mathbf{u} = \mathbf{v}$
Symmetry	$d(\mathbf{u}, \mathbf{v}) = d(\mathbf{v}, \mathbf{u})$
Triangle inequality	$d(\mathbf{x}, \mathbf{z}) \le d(\mathbf{x}, \mathbf{y}) + d(\mathbf{y}, \mathbf{z})$

The reverse triangle inequality is used in practice to prune nearest-neighbor searches. If the distances from every point to a fixed pivot are precomputed, many candidates can be ruled out without computing their distance to the query:

import numpy as np

rng = np.random.default_rng(7)
points = rng.uniform(0, 100, size=(1000, 2))
pivot = np.array([0.0, 0.0])
d_pivot = np.linalg.norm(points - pivot, axis=1)    # precomputed once, reused for every query

def nearest(q):
    dq = np.linalg.norm(q - pivot)
    best, best_d, skipped = None, np.inf, 0
    for p, dp in zip(points, d_pivot):
        # reverse triangle inequality: ||q - p|| >= | ||q - pivot|| - ||p - pivot|| |
        if abs(dq - dp) >= best_d:
            skipped += 1                              # p cannot beat the current best: no distance computed
            continue
        d = np.linalg.norm(q - p)
        if d < best_d:
            best, best_d = p, d
    return best, best_d, skipped

best, best_d, skipped = nearest(np.array([50.0, 50.0]))
print(f"nearest at distance {best_d:.3f}; skipped {skipped} of 1000 distance computations")

Key takeaways

$\lVert \mathbf{u} + \mathbf{v} \rVert \le \lVert \mathbf{u} \rVert + \lVert \mathbf{v} \rVert$, with equality only when the vectors point the same way.
It follows from Cauchy-Schwarz in a single expansion.
The reverse form $\lvert \lVert \mathbf{u} \rVert - \lVert \mathbf{v} \rVert \rvert \le \lVert \mathbf{u} - \mathbf{v} \rVert$ gives lower bounds on distances.
Together with positivity and symmetry, it makes Euclidean distance a metric.

🧪 Practice

Check the inequality for $\mathbf{u} = (3, 0)$, $\mathbf{v} = (0, 4)$, and for $\mathbf{u} = (1, 2)$, $\mathbf{v} = (2, 4)$. Which case is an equality, and why?

Prove the reverse triangle inequality. Hint: write $\mathbf{u} = (\mathbf{u} - \mathbf{v}) + \mathbf{v}$, then swap the roles of $\mathbf{u}$ and $\mathbf{v}$.

Prove $\lVert \mathbf{v}_1 + \cdots + \mathbf{v}_k \rVert \le \lVert \mathbf{v}_1 \rVert + \cdots + \lVert \mathbf{v}_k \rVert$ by induction on $k$.

Interview: Is the squared distance $d(\mathbf{u}, \mathbf{v}) = \lVert \mathbf{u} - \mathbf{v} \rVert^2$ a metric? Hint: test the triangle inequality on three equally spaced points on a line, such as $0, 1, 2$ in $\mathbb{R}^1$.

Vector Projection

Why it matters. Projection splits a vector into a part along a given direction and a part perpendicular to it. That split answers "how much of this force acts along the ramp?" and "what is the closest point on this line?". It is also the core step of least squares (Chapter 10) and the Gram-Schmidt process.

Derivation. We want the multiple $c\mathbf{v}$ such that the leftover $\mathbf{u} - c\mathbf{v}$ is perpendicular to $\mathbf{v}$:

$$(\mathbf{u} - c\mathbf{v}) \cdot \mathbf{v} = 0 \implies \mathbf{u} \cdot \mathbf{v} - c,(\mathbf{v} \cdot \mathbf{v}) = 0 \implies c = \frac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{v} \cdot \mathbf{v}}.$$

Definitions. For $\mathbf{v} \ne \mathbf{0}$:

$$\operatorname{proj}{\mathbf{v}} \mathbf{u} = \frac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{v} \cdot \mathbf{v}}, \mathbf{v} \quad \text{(vector projection)}, \qquad \operatorname{comp}{\mathbf{v}} \mathbf{u} = \frac{\mathbf{u} \cdot \mathbf{v}}{\lVert \mathbf{v} \rVert} \quad \text{(scalar projection)}.$$

The scalar projection is the signed length of the shadow; it is negative when the angle is obtuse.

              * u
             /|
            / |
           /  |  u - proj_v u    (perpendicular to v)
          /   |
         /    |
        O-----*-------------------> v
        proj_v u

Orthogonal decomposition. Every $\mathbf{u}$ splits uniquely as

$$\mathbf{u} = \underbrace{\operatorname{proj}{\mathbf{v}} \mathbf{u}}{\text{parallel to } \mathbf{v}} + \underbrace{(\mathbf{u} - \operatorname{proj}{\mathbf{v}} \mathbf{u})}{\text{perpendicular to } \mathbf{v}}.$$

Example. $\mathbf{u} = (3, 4)$, $\mathbf{v} = (1, 1)$: $\mathbf{u} \cdot \mathbf{v} = 7$ and $\mathbf{v} \cdot \mathbf{v} = 2$, so $\operatorname{proj}_{\mathbf{v}} \mathbf{u} = (3.5, 3.5)$ and the perpendicular part is $(-0.5, 0.5)$. Check: $(-0.5, 0.5) \cdot (1, 1) = 0$.

Properties.

Only the direction of $\mathbf{v}$ matters: replacing $\mathbf{v}$ by $k\mathbf{v}$ ($k \ne 0$) gives the same projection, because the $k$s cancel.
Projecting twice changes nothing: $\operatorname{proj}{\mathbf{v}}(\operatorname{proj}{\mathbf{v}} \mathbf{u}) = \operatorname{proj}_{\mathbf{v}} \mathbf{u}$.
Closest point: $\operatorname{proj}_{\mathbf{v}} \mathbf{u}$ is the point on the line through the origin in direction $\mathbf{v}$ that is nearest to $\mathbf{u}$.
Matrix form: $\operatorname{proj}_{\mathbf{v}} \mathbf{u} = P\mathbf{u}$ with $P = \dfrac{\mathbf{v}\mathbf{v}^T}{\mathbf{v}^T\mathbf{v}}$, a projection matrix satisfying $P^2 = P$ (Chapter 10).

Physics: work. A constant force $\mathbf{F}$ moving an object along displacement $\mathbf{d}$ does work $W = \mathbf{F} \cdot \mathbf{d} = (\operatorname{comp}_{\mathbf{d}} \mathbf{F})\lVert \mathbf{d} \rVert$. Only the part of the force along the motion counts.

import numpy as np

def project(u, v):
    """Orthogonal projection of u onto the line spanned by v (v must be nonzero)."""
    return (u @ v) / (v @ v) * v

u = np.array([3.0, 4.0])
v = np.array([1.0, 1.0])
p = project(u, v)
r = u - p                               # the part of u perpendicular to v
print(p, r, r @ v)                      # [3.5 3.5] [-0.5  0.5] 0.0

print(project(u, 10 * v))               # [3.5 3.5]  only the direction of v matters

P = np.outer(v, v) / (v @ v)            # projection matrix onto the line through v
print(P @ u)                            # [3.5 3.5]  same result as a matrix-vector product
print(np.allclose(P @ P, P))            # True: projecting twice is the same as projecting once

Key takeaways

$\operatorname{proj}_{\mathbf{v}} \mathbf{u} = \dfrac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{v} \cdot \mathbf{v}}\mathbf{v}$ is the multiple of $\mathbf{v}$ that leaves a perpendicular remainder.
Every vector splits uniquely into a part parallel to $\mathbf{v}$ and a part perpendicular to it.
The projection is the closest point on the line, and it depends only on the direction of $\mathbf{v}$.
Projection is a linear map with matrix $\mathbf{v}\mathbf{v}^T / (\mathbf{v}^T\mathbf{v})$, and applying it twice changes nothing.

🧪 Practice

Project $\mathbf{u} = (1, 2, 3)$ onto $\mathbf{v} = (1, 0, 1)$, find the perpendicular part, and verify that it is orthogonal to $\mathbf{v}$.

A force $\mathbf{F} = (10, 5)$ N pulls a box through the displacement $\mathbf{d} = (6, 0)$ m. Compute the work done and the component of the force along the motion.

Use projection to find the distance from the point $(3, 4)$ to the line through the origin with direction $(1, 1)$.

Interview: Prove that $c = \dfrac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{v} \cdot \mathbf{v}}$ minimizes $\lVert \mathbf{u} - c\mathbf{v} \rVert$ over all real $c$. Hint: write $\mathbf{u} - c\mathbf{v} = (\mathbf{u} - \operatorname{proj}{\mathbf{v}}\mathbf{u}) + (\operatorname{proj}{\mathbf{v}}\mathbf{u} - c\mathbf{v})$ and apply the Pythagorean theorem.

2.4 Cross Product and Spatial Geometry

Three-dimensional space has a second product that returns a vector instead of a number; this subchapter defines the cross product, interprets it geometrically, and uses it to compute areas and volumes.

Cross Product in R^3

Why it matters. Many 3D problems need a vector perpendicular to two given vectors: the normal to a surface for lighting, the axis of rotation for torque, or the third axis of a camera frame. The cross product produces exactly that vector, and its length measures area.

Definition. For $\mathbf{u}, \mathbf{v} \in \mathbb{R}^3$,

$$\mathbf{u} \times \mathbf{v} = \big(u_2 v_3 - u_3 v_2,\ \ u_3 v_1 - u_1 v_3,\ \ u_1 v_2 - u_2 v_1\big).$$

Determinant mnemonic. Expand a formal $3 \times 3$ determinant (Chapter 8) along its first row:

$$\mathbf{u} \times \mathbf{v} = \det\begin{pmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \ u_1 & u_2 & u_3 \ v_1 & v_2 & v_3 \end{pmatrix} = \mathbf{i}\begin{vmatrix} u_2 & u_3 \ v_2 & v_3 \end{vmatrix} - \mathbf{j}\begin{vmatrix} u_1 & u_3 \ v_1 & v_3 \end{vmatrix} + \mathbf{k}\begin{vmatrix} u_1 & u_2 \ v_1 & v_2 \end{vmatrix}.$$

Note the minus sign on the middle term.

Example. $(1, 2, 3) \times (4, 5, 6) = (2 \cdot 6 - 3 \cdot 5,\ 3 \cdot 4 - 1 \cdot 6,\ 1 \cdot 5 - 2 \cdot 4) = (-3, 6, -3)$. Check: $(1, 2, 3) \cdot (-3, 6, -3) = -3 + 12 - 9 = 0$.

Basis vectors. The products of $\mathbf{i}, \mathbf{j}, \mathbf{k}$ follow a cycle:

          i
        /   \
       ^     v          i x j = k     j x k = i     k x i = j   (along the arrows)
      k <---- j         j x i = -k    k x j = -i    i x k = -j  (against the arrows)
                        i x i = j x j = k x k = 0

Properties.

Property	Statement
Anti-commutative	$\mathbf{u} \times \mathbf{v} = -(\mathbf{v} \times \mathbf{u})$
Self product	$\mathbf{v} \times \mathbf{v} = \mathbf{0}$
Distributive	$\mathbf{u} \times (\mathbf{v} + \mathbf{w}) = \mathbf{u} \times \mathbf{v} + \mathbf{u} \times \mathbf{w}$
Scalars factor out	$(c\mathbf{u}) \times \mathbf{v} = c(\mathbf{u} \times \mathbf{v}) = \mathbf{u} \times (c\mathbf{v})$
Perpendicular	$(\mathbf{u} \times \mathbf{v}) \cdot \mathbf{u} = (\mathbf{u} \times \mathbf{v}) \cdot \mathbf{v} = 0$
Lagrange's identity	$\lVert \mathbf{u} \times \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 \lVert \mathbf{v} \rVert^2 - (\mathbf{u} \cdot \mathbf{v})^2$
Not associative	$(\mathbf{i} \times \mathbf{i}) \times \mathbf{j} = \mathbf{0}$, but $\mathbf{i} \times (\mathbf{i} \times \mathbf{j}) = -\mathbf{j}$

Only in $\mathbb{R}^3$. In $\mathbb{R}^n$ for $n > 3$, two vectors have infinitely many independent perpendicular directions, so there is no single natural choice. (A product with similar properties exists in $\mathbb{R}^7$, but it is a curiosity.) The generalization that works in every dimension is the wedge product of exterior algebra.

import numpy as np

def cross(u, v):
    return np.array([
        u[1] * v[2] - u[2] * v[1],      # cover up column 1: 2x2 determinant of columns 2, 3
        u[2] * v[0] - u[0] * v[2],      # written in this order so the minus sign is built in
        u[0] * v[1] - u[1] * v[0],      # cover up column 3: 2x2 determinant of columns 1, 2
    ])

u = np.array([1, 2, 3])
v = np.array([4, 5, 6])
w = cross(u, v)
print(w, np.cross(u, v))                # [-3  6 -3] [-3  6 -3]
print(w @ u, w @ v)                     # 0 0: perpendicular to both inputs
print(np.cross(v, u))                   # [ 3 -6  3]: swapping the order flips the sign

i, j = np.eye(3)[:2]
print(np.cross(np.cross(i, i), j), np.cross(i, np.cross(i, j)))   # [0. 0. 0.] [ 0. -1.  0.]: not associative

Key takeaways

$\mathbf{u} \times \mathbf{v}$ is a vector in $\mathbb{R}^3$, perpendicular to both $\mathbf{u}$ and $\mathbf{v}$.
Compute it with the $3 \times 3$ determinant mnemonic, remembering the minus sign on the middle term.
It is anti-commutative and not associative, and $\mathbf{v} \times \mathbf{v} = \mathbf{0}$.
Lagrange's identity connects its length to the dot product.

🧪 Practice

Compute $(2, 0, -1) \times (1, 3, 2)$ and verify that the result is orthogonal to both vectors.

Prove $\mathbf{u} \times \mathbf{u} = \mathbf{0}$ from the formula, then deduce anti-commutativity by expanding $(\mathbf{u} + \mathbf{v}) \times (\mathbf{u} + \mathbf{v}) = \mathbf{0}$.

Verify Lagrange's identity for $\mathbf{u} = (1, 2, 3)$ and $\mathbf{v} = (4, 5, 6)$.

Interview: A camera at position $E$ looks toward a target $T$, with an approximate "up" direction $\mathbf{a}$ that is not parallel to $T - E$. How do you build three mutually perpendicular unit vectors (forward, right, up) for the camera? Hint: one cross product gives "right"; a second cross product corrects "up".

Geometric Meaning and Right-Hand Rule

Why it matters. The component formula hides what the cross product means. Geometrically, $\mathbf{u} \times \mathbf{v}$ is completely described by a direction (perpendicular, chosen by the right-hand rule) and a length (the area of a parallelogram). Most applications use one of these two facts.

Direction: the right-hand rule. Two vectors perpendicular to both $\mathbf{u}$ and $\mathbf{v}$ are possible, one on each side of their plane. The cross product picks the one given by your right hand: point your fingers along $\mathbf{u}$ and curl them toward $\mathbf{v}$ through the smaller angle, and your thumb points along $\mathbf{u} \times \mathbf{v}$.

             u x v
               ^
               |
               |
               O---------> v
              /
             /
            v
           u
   fingers along u, curl toward v: thumb points along u x v
   swap u and v and the thumb points down: v x u = -(u x v)

Length: $\lVert \mathbf{u} \times \mathbf{v} \rVert = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \sin\theta$. Substitute $\mathbf{u} \cdot \mathbf{v} = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \cos\theta$ into Lagrange's identity:

$$\lVert \mathbf{u} \times \mathbf{v} \rVert^2 = \lVert \mathbf{u} \rVert^2 \lVert \mathbf{v} \rVert^2 (1 - \cos^2\theta) = \lVert \mathbf{u} \rVert^2 \lVert \mathbf{v} \rVert^2 \sin^2\theta.$$

Since $\theta \in [0, \pi]$, $\sin\theta \ge 0$, and the square root gives the formula. Consequently, $\mathbf{u} \times \mathbf{v} = \mathbf{0}$ exactly when $\mathbf{u}$ and $\mathbf{v}$ are parallel or one of them is zero.

Dot vs. cross product.

	Dot product $\mathbf{u} \cdot \mathbf{v}$	Cross product $\mathbf{u} \times \mathbf{v}$
Output	Scalar	Vector
Defined in	Any $\mathbb{R}^n$	$\mathbb{R}^3$ only
Magnitude	$\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \cos\theta$	$\lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \sin\theta$
Zero when	Perpendicular	Parallel
Swapping inputs	No change	Sign flips
Measures	Alignment, projection	Perpendicular direction, area

Physics. Torque $\boldsymbol{\tau} = \mathbf{r} \times \mathbf{F}$ points along the axis about which a force $\mathbf{F}$, applied at lever arm $\mathbf{r}$, tends to rotate an object. Its magnitude $\lVert \mathbf{r} \rVert \lVert \mathbf{F} \rVert \sin\theta$ explains why you push a wrench at its end and at right angles. Angular momentum ($\mathbf{L} = \mathbf{r} \times \mathbf{p}$) and the magnetic force on a charge ($q\mathbf{v} \times \mathbf{B}$) follow the same pattern.

Orientation in 2D. Embedding 2D vectors as $(x, y, 0)$, the cross product has only a $z$-component, $u_1 v_2 - u_2 v_1$. Its sign says whether turning from $\mathbf{u}$ to $\mathbf{v}$ is counterclockwise (positive) or clockwise (negative). This orientation test underlies convex hull algorithms, polygon triangulation, and point-in-polygon checks.

import numpy as np

def orientation(a, b, c):
    """Turn direction of the path a -> b -> c, from the sign of (b - a) x (c - a)."""
    (x1, y1), (x2, y2) = b - a, c - a
    z = x1 * y2 - y1 * x2             # the only nonzero component when both z-coordinates are 0
    return "left (CCW)" if z > 0 else "right (CW)" if z < 0 else "collinear"

a, b = np.array([0, 0]), np.array([4, 0])
print(orientation(a, b, np.array([2, 3])))     # left (CCW)
print(orientation(a, b, np.array([2, -1])))    # right (CW)
print(orientation(a, b, np.array([8, 0])))     # collinear

# Torque: a 0.3 m wrench along x, with a 50 N force along y applied at its end
r = np.array([0.3, 0.0, 0.0])
F = np.array([0.0, 50.0, 0.0])
print(np.cross(r, F))                          # [ 0.  0. 15.]  15 N*m about +z (counterclockwise from above)

Key takeaways

$\mathbf{u} \times \mathbf{v}$ is perpendicular to both inputs, with its direction set by the right-hand rule.
$\lVert \mathbf{u} \times \mathbf{v} \rVert = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \sin\theta$, which is zero exactly for parallel vectors.
The dot product measures alignment ($\cos\theta$); the cross product measures perpendicularity and area ($\sin\theta$).
In 2D, the sign of $u_1 v_2 - u_2 v_1$ is a fast and exact left-turn/right-turn test.

🧪 Practice

Use the right-hand rule to predict $\mathbf{j} \times \mathbf{i}$ and $\mathbf{k} \times \mathbf{j}$, then confirm with the basis table.

If $\lVert \mathbf{u} \rVert = 3$, $\lVert \mathbf{v} \rVert = 4$, and the angle between them is $30^\circ$, find $\lVert \mathbf{u} \times \mathbf{v} \rVert$ and $\mathbf{u} \cdot \mathbf{v}$.

Prove that for nonzero $\mathbf{u}, \mathbf{v}$, $\mathbf{u} \times \mathbf{v} = \mathbf{0}$ if and only if $\mathbf{u}$ and $\mathbf{v}$ are parallel.

Interview: Give a constant-time test for whether a point $P$ lies strictly inside a triangle $ABC$ in the plane, using no angles or square roots. Hint: walk around the triangle's edges and check on which side of each edge $P$ lies.

Scalar Triple Product

Why it matters. Combining a cross product with a dot product produces a single number that measures the signed volume spanned by three vectors. It answers two practical questions directly: do three vectors lie in a common plane, and are they arranged in right-handed order?

Definition. For $\mathbf{u}, \mathbf{v}, \mathbf{w} \in \mathbb{R}^3$,

$$[\mathbf{u}, \mathbf{v}, \mathbf{w}] = \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) = \det\begin{pmatrix} u_1 & u_2 & u_3 \ v_1 & v_2 & v_3 \ w_1 & w_2 & w_3 \end{pmatrix}.$$

The determinant form follows from the mnemonic in "Cross Product in R^3": dotting with $\mathbf{u}$ replaces the row $\mathbf{i}, \mathbf{j}, \mathbf{k}$ with $u_1, u_2, u_3$.

Example. $\mathbf{u} = (1, 2, 0)$, $\mathbf{v} = (0, 1, 3)$, $\mathbf{w} = (2, 0, 1)$. Then $\mathbf{v} \times \mathbf{w} = (1, 6, -2)$ and $\mathbf{u} \cdot (1, 6, -2) = 1 + 12 + 0 = 13$.

Properties.

Cyclic shifts preserve it: $[\mathbf{u}, \mathbf{v}, \mathbf{w}] = [\mathbf{v}, \mathbf{w}, \mathbf{u}] = [\mathbf{w}, \mathbf{u}, \mathbf{v}]$.
Swapping two vectors flips the sign: $[\mathbf{v}, \mathbf{u}, \mathbf{w}] = -[\mathbf{u}, \mathbf{v}, \mathbf{w}]$.
Dot and cross can trade places: $\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) = (\mathbf{u} \times \mathbf{v}) \cdot \mathbf{w}$.
Coplanarity test: $[\mathbf{u}, \mathbf{v}, \mathbf{w}] = 0$ exactly when the three vectors lie in a common plane through the origin, which means one is a linear combination of the others (linear dependence, Chapter 5).
Orientation: $[\mathbf{u}, \mathbf{v}, \mathbf{w}] > 0$ exactly when $(\mathbf{u}, \mathbf{v}, \mathbf{w})$ is right-handed, like $(\mathbf{i}, \mathbf{j}, \mathbf{k})$.

Vector triple product. A related identity simplifies nested cross products ("BAC minus CAB"):

$$\mathbf{u} \times (\mathbf{v} \times \mathbf{w}) = \mathbf{v}(\mathbf{u} \cdot \mathbf{w}) - \mathbf{w}(\mathbf{u} \cdot \mathbf{v}).$$

import numpy as np

u = np.array([1, 2, 0])
v = np.array([0, 1, 3])
w = np.array([2, 0, 1])

print(u @ np.cross(v, w))                           # 13
print(np.cross(u, v) @ w)                           # 13: dot and cross can trade places
print(round(np.linalg.det(np.array([u, v, w]))))    # 13: equals the 3x3 determinant
print(v @ np.cross(u, w))                           # -13: swapping u and v flips the sign

def coplanar(a, b, c, tol=1e-12):
    return abs(a @ np.cross(b, c)) < tol

print(coplanar(u, v, 3 * u - 2 * v))                # True: the third vector is a combination of the first two
print(coplanar(u, v, w))                            # False

Key takeaways

$[\mathbf{u}, \mathbf{v}, \mathbf{w}] = \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w})$ equals the determinant of the matrix with rows $\mathbf{u}, \mathbf{v}, \mathbf{w}$.
Cyclic permutations keep its value, and swapping two vectors negates it.
It is zero exactly when the three vectors are coplanar (linearly dependent).
Its sign gives the handedness of the ordered triple.

🧪 Practice

Compute $[\mathbf{i}, \mathbf{j}, \mathbf{k}]$ and $[\mathbf{j}, \mathbf{i}, \mathbf{k}]$, and interpret their signs.

Determine whether $(1, 1, 0)$, $(0, 1, 1)$, and $(1, 0, -1)$ are coplanar. If they are, express one as a combination of the other two.

Find all real $t$ for which $(1, t, 0)$, $(0, 1, t)$, and $(t, 0, 1)$ are coplanar.

Interview: Prove that $\mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) = 0$ whenever two of the three vectors are equal, and explain the result geometrically. Hint: handle $\mathbf{v} = \mathbf{w}$ and $\mathbf{u} = \mathbf{v}$ separately, using two different properties of the cross product.

Area and Volume Interpretations

Why it matters. Areas and volumes are needed constantly: the surface area of a 3D model built from triangles, the area of a land parcel from GPS coordinates, the volume of mesh elements in engineering simulations. Cross and triple products compute them directly from coordinates, with no angles or heights to measure.

Parallelogram area. The parallelogram spanned by $\mathbf{u}$ and $\mathbf{v}$ has base $\lVert \mathbf{u} \rVert$ and height $\lVert \mathbf{v} \rVert \sin\theta$, so

$$\text{Area} = \lVert \mathbf{u} \rVert \lVert \mathbf{v} \rVert \sin\theta = \lVert \mathbf{u} \times \mathbf{v} \rVert.$$

              *-------------------*
             /|                  /
          v / | h = ||v|| sin θ /
           /  |                /
          / θ |               /
         O----+--------------*
                   u
     area = base * height = ||u|| * ||v|| sin θ = ||u x v||

A triangle with vertices $A, B, C$ is half of a parallelogram: $\text{Area} = \tfrac{1}{2}\lVert (B - A) \times (C - A) \rVert$. In 2D, embedding with $z = 0$ gives $\tfrac{1}{2}\lvert u_1 v_2 - u_2 v_1 \rvert$.

Parallelepiped volume. A slanted box (parallelepiped) spanned by $\mathbf{u}, \mathbf{v}, \mathbf{w}$ has a base parallelogram of area $\lVert \mathbf{v} \times \mathbf{w} \rVert$. Its height is the length of $\mathbf{u}$'s projection onto the base's normal $\mathbf{v} \times \mathbf{w}$, which is $\lvert \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) \rvert / \lVert \mathbf{v} \times \mathbf{w} \rVert$. Multiplying:

$$\text{Volume} = \lvert \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) \rvert.$$

          v x w  (normal to the base)
            ^
            |        * u
            |       /
          h |      /         h = |u . (v x w)| / ||v x w||
            |     /
            |    /
       +----O---/------------+
      /                     /     base: parallelogram spanned by v and w,
     /    area ||v x w||   /      area ||v x w||
    +---------------------+
          volume = base area * h = |u . (v x w)|

A tetrahedron with vertices $A, B, C, D$ has $\tfrac{1}{6}$ the volume of the parallelepiped on its edges: $V = \tfrac{1}{6}\lvert (B - A) \cdot ((C - A) \times (D - A)) \rvert$.

Summary of formulas.

Shape	Formula
Parallelogram spanned by $\mathbf{u}, \mathbf{v}$ in $\mathbb{R}^3$	$\lVert \mathbf{u} \times \mathbf{v} \rVert$
Parallelogram in $\mathbb{R}^2$	$\lvert u_1 v_2 - u_2 v_1 \rvert$
Triangle $ABC$	$\tfrac{1}{2}\lVert (B - A) \times (C - A) \rVert$
Parallelepiped spanned by $\mathbf{u}, \mathbf{v}, \mathbf{w}$	$\lvert \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) \rvert$
Tetrahedron $ABCD$	$\tfrac{1}{6}\lvert (B - A) \cdot ((C - A) \times (D - A)) \rvert$
Simple polygon $(x_1, y_1), \dots, (x_m, y_m)$	$\tfrac{1}{2}\big\lvert \sum_{i} (x_i y_{i+1} - x_{i+1} y_i) \big\rvert$ (shoelace)

The shoelace formula sums the signed 2D cross products of consecutive vertices, splitting the polygon into triangles fanned out from the origin. Signed areas outside the polygon cancel.

import numpy as np

def triangle_area(A, B, C):
    return 0.5 * np.linalg.norm(np.cross(B - A, C - A))

def tetra_volume(A, B, C, D):
    return abs((B - A) @ np.cross(C - A, D - A)) / 6

A, B, C = np.array([1, 0, 0]), np.array([0, 2, 0]), np.array([0, 0, 3])
print(triangle_area(A, B, C))               # 3.5  ((B - A) x (C - A) = (6, 3, 2), length 7)
print(tetra_volume(np.zeros(3), A, B, C))   # 1.0  (a 1 x 2 x 3 box has volume 6; this corner is 1/6 of it)

def shoelace(pts):
    """Area of a simple polygon whose vertices are listed in order."""
    x, y = np.asarray(pts, dtype=float).T
    # x_i * y_{i+1} - x_{i+1} * y_i, wrapping around from the last vertex to the first
    return 0.5 * abs(np.sum(x * np.roll(y, -1) - np.roll(x, -1) * y))

L_shape = [(0, 0), (4, 0), (4, 1), (1, 1), (1, 3), (0, 3)]
print(shoelace(L_shape))                    # 6.0  (a 4 x 1 strip plus a 1 x 2 strip)

Key takeaways

$\lVert \mathbf{u} \times \mathbf{v} \rVert$ is the area of the parallelogram spanned by $\mathbf{u}$ and $\mathbf{v}$; a triangle is half of it.
$\lvert \mathbf{u} \cdot (\mathbf{v} \times \mathbf{w}) \rvert$ is the volume of the parallelepiped; a tetrahedron is one sixth of it.
The shoelace formula computes polygon areas from vertex coordinates by summing 2D cross products.
These are the first appearances of the determinant as an area and volume scale factor (Chapter 8).

🧪 Practice

Find the area of the parallelogram spanned by $(2, 1, 0)$ and $(1, 3, 0)$ with the cross product, and again with the 2D formula.

Find the area of the triangle with vertices $(1, 1, 1)$, $(2, 3, 1)$, and $(1, 2, 4)$.

Find the volume of the parallelepiped spanned by $(1, 0, 1)$, $(0, 2, 0)$, and $(1, 1, 3)$.

Interview: You have a closed, watertight triangle mesh whose triangles are consistently oriented (normals pointing outward). How can you compute the enclosed volume from the vertex coordinates alone? Hint: form a tetrahedron from each triangle and the origin, and keep the signs of the triple products.

2.5 Lines and Planes

With vectors, lines and planes become sets described by a point plus directions, or by a point plus a normal vector; this subchapter develops both descriptions and uses them to compute distances and intersections.

Parametric Equations of Lines

Why it matters. The slope-intercept form $y = mx + b$ cannot describe vertical lines and has no counterpart in 3D. The vector form of a line works in every dimension and has a physical reading as constant-velocity motion. Ray tracing, robot path planning, and collision detection all use it.

Vector equation. The line through a point $P$ (position vector $\mathbf{p}$) in the direction $\mathbf{d} \ne \mathbf{0}$ is

$$\mathbf{x}(t) = \mathbf{p} + t\mathbf{d}, \qquad t \in \mathbb{R}.$$

Think of a particle that starts at $P$ at time $t = 0$ and moves with constant velocity $\mathbf{d}$. Negative $t$ describes where it was before.

       t = -1        t = 0         t = 1         t = 2
   --------*-------------*-------------*-------------*-------->
         p - d           P           p + d         p + 2d
                          \_____ d _____/

Component forms. In $\mathbb{R}^3$, with $\mathbf{p} = (p_1, p_2, p_3)$ and $\mathbf{d} = (d_1, d_2, d_3)$:

Form	Equation	Note
Vector	$\mathbf{x} = \mathbf{p} + t\mathbf{d}$	Works in any $\mathbb{R}^n$
Parametric	$x = p_1 + t d_1,\ y = p_2 + t d_2,\ z = p_3 + t d_3$	One equation per coordinate
Symmetric	$\dfrac{x - p_1}{d_1} = \dfrac{y - p_2}{d_2} = \dfrac{z - p_3}{d_3}$	Needs every $d_i \ne 0$

Line through two points. Take $\mathbf{d} = Q - P$: $\mathbf{x}(t) = P + t(Q - P)$. Restricting $t$ gives pieces of the line:

Parameter range	Set
$t \in \mathbb{R}$	The whole line
$t \ge 0$	The ray from $P$ through $Q$
$0 \le t \le 1$	The segment from $P$ to $Q$

Representations are not unique. Any point on the line can serve as $\mathbf{p}$, and any nonzero multiple of $\mathbf{d}$ can serve as the direction. To check whether two descriptions give the same line, check that the directions are parallel and that one base point lies on the other line.

Example. Through $P = (1, 2, 3)$ and $Q = (4, 0, 5)$: $\mathbf{d} = (3, -2, 2)$, so $x = 1 + 3t$, $y = 2 - 2t$, $z = 3 + 2t$. The point $(7, -2, 7)$ is on the line ($t = 2$), but $(7, -2, 8)$ is not.

import numpy as np

P = np.array([1.0, 2.0, 3.0])
Q = np.array([4.0, 0.0, 5.0])
d = Q - P                                     # direction (3, -2, 2)

line = lambda t: P + t * d
print(line(0), line(1), line(0.5))            # P, Q, and the midpoint of the segment

def on_line(X, P, d, tol=1e-9):
    """X is on the line exactly when X - P is parallel to d."""
    return np.linalg.norm(np.cross(X - P, d)) < tol   # the cross product vanishes only for parallel vectors

print(on_line(np.array([7.0, -2.0, 7.0]), P, d))   # True  (t = 2)
print(on_line(np.array([7.0, -2.0, 8.0]), P, d))   # False

Key takeaways

A line is $\mathbf{x} = \mathbf{p} + t\mathbf{d}$: a point plus all multiples of a direction vector.
The vector form works in every dimension, including the vertical lines that slope-intercept form cannot describe.
$P + t(Q - P)$ gives the line through two points; restricting $t$ gives rays and segments.
The same line has many representations; compare lines by direction and by a shared point.

🧪 Practice

Write vector and parametric equations for the line through $(2, -1, 0)$ and $(5, 3, -2)$.

Do $\mathbf{x} = (1, 0, 0) + t(1, 2, 3)$ and $\mathbf{x} = (3, 4, 6) + s(-2, -4, -6)$ describe the same line?

A particle starts at $(0, 0, 10)$ and moves with velocity $(2, 1, -1)$ per second. When and where does it reach the plane $z = 0$?

Interview: In ray tracing, a camera ray is $\mathbf{p} + t\mathbf{d}$ with $t \ge 0$. Why does the restriction $t \ge 0$ matter, and what does an intersection with $t < 0$ mean? Hint: think about where the camera is and which way it faces.

Vector Equation of a Plane

Why it matters. Planes are the flat surfaces of 3D geometry: walls, mirrors, triangle faces of a mesh. Describing a plane with vectors extends the line description with one more direction. That pattern continues into higher dimensions and leads to the idea of a subspace.

Vector equation. The plane through $P$ containing the directions $\mathbf{u}$ and $\mathbf{v}$ (not parallel) is

$$\mathbf{x}(s, t) = \mathbf{p} + s\mathbf{u} + t\mathbf{v}, \qquad s, t \in \mathbb{R}.$$

The parameters $(s, t)$ work like coordinates on a slanted grid laid on the plane: walk $s$ steps along $\mathbf{u}$ and $t$ steps along $\mathbf{v}$.

             *-------*-------*-------*
            /       /       /       /
           *-------*-------*-------*        points p + s u + t v
          /       /       /       /         (s, t) are coordinates on the plane
         ^       /       /       /
      v /       /       /       /
       P------>*-------*-------*
           u

Through three points. For non-collinear points $A, B, C$, use $\mathbf{u} = B - A$ and $\mathbf{v} = C - A$. If the points are collinear, $\mathbf{u}$ and $\mathbf{v}$ are parallel and generate only a line, so infinitely many planes contain them.

Planes through the origin. When $\mathbf{p} = \mathbf{0}$, the plane is the set of all linear combinations $s\mathbf{u} + t\mathbf{v}$, the span of ${\mathbf{u}, \mathbf{v}}$ from Section 2.2. A plane through $P$ is that span shifted by $\mathbf{p}$. This "subspace plus a shift" structure returns in Chapter 3 as the shape of every solution set of a linear system.

Object	Vector description	Free parameters
Point	$\mathbf{p}$	0
Line	$\mathbf{p} + t\mathbf{d}$	1
Plane	$\mathbf{p} + s\mathbf{u} + t\mathbf{v}$	2
$k$-flat in $\mathbb{R}^n$	$\mathbf{p} + t_1\mathbf{v}_1 + \cdots + t_k\mathbf{v}_k$	$k$

Example. Through $A = (1, 0, 0)$, $B = (0, 2, 0)$, $C = (0, 0, 3)$: $\mathbf{x} = (1, 0, 0) + s(-1, 2, 0) + t(-1, 0, 3)$. Deciding whether a given point lies on the plane means solving for $s$ and $t$: three equations in two unknowns.

import numpy as np

A = np.array([1.0, 0.0, 0.0])
B = np.array([0.0, 2.0, 0.0])
C = np.array([0.0, 0.0, 3.0])
u, v = B - A, C - A                           # two directions within the plane

plane = lambda s, t: A + s * u + t * v
print(plane(1, 0), plane(0, 1))               # B and C
print(plane(1/3, 1/3))                        # [0.333 0.667 1.] (approx.): the centroid of A, B, C

def plane_coords(X):
    """Find (s, t) with A + s*u + t*v = X, and report whether they exist."""
    M = np.column_stack([u, v])               # 3 equations, 2 unknowns
    st, *_ = np.linalg.lstsq(M, X - A, rcond=None)
    return st, np.allclose(M @ st, X - A)     # the best fit is exact only if X is on the plane

print(plane_coords(np.array([0.5, 1.0, 0.0])))   # s = 0.5, t = 0 (up to rounding), True
print(plane_coords(np.array([1.0, 1.0, 1.0])))   # some best-fit (s, t), False

Key takeaways

A plane is $\mathbf{p} + s\mathbf{u} + t\mathbf{v}$ with non-parallel directions $\mathbf{u}, \mathbf{v}$.
Three non-collinear points $A, B, C$ determine a plane, with directions $B - A$ and $C - A$.
A plane through the origin is the span of its two direction vectors; any other plane is a shifted span.
Checking whether a point lies on the plane means solving a small linear system for $(s, t)$.

🧪 Practice

Write a vector equation of the plane through $(1, 1, 1)$, $(2, 0, 1)$, and $(1, 3, 2)$.

Explain why $(0, 0, 0)$, $(1, 2, 3)$, and $(2, 4, 6)$ do not determine a unique plane.

Is $(2, 2, 2)$ on the plane $\mathbf{x} = (1, 0, 0) + s(1, 1, 0) + t(0, 1, 1)$?

Interview: A line needs one direction vector and a plane in $\mathbb{R}^3$ needs two. How many would you need to describe a "hyperplane" in $\mathbb{R}^n$, and what does this suggest about dimension? Hint: count how many independent parameters you need to reach every point of the set.

Normal Vectors and Scalar Equations

Why it matters. The vector form needs two direction vectors and two parameters. A plane can instead be described by one vector, its normal, and one equation. That equation tests membership in one step and tells you which side of the plane a point is on. Collision detection, back-face culling in graphics, and linear classifiers all rely on this.

Normal vectors. A normal $\mathbf{n}$ to a plane is a nonzero vector perpendicular to every direction in the plane. A point $X$ lies on the plane through $P$ exactly when the arrow $X - P$ lies in the plane, that is, when it is orthogonal to $\mathbf{n}$:

$$\mathbf{n} \cdot (\mathbf{x} - \mathbf{p}) = 0 \iff n_1 x + n_2 y + n_3 z = d, \quad \text{where } d = \mathbf{n} \cdot \mathbf{p}.$$

The coefficients of $x, y, z$ in a plane's equation are the components of a normal vector. You can read a normal directly off the equation.

                   n
                   ^
                   |
                   |        * X
         +---------P-------/-------------+
        /           \_____/             /     n . (X - P) = 0
       /             X - P             /      for every X in the plane
      +-------------------------------+

From three points. The normal is perpendicular to both directions, so $\mathbf{n} = (B - A) \times (C - A)$. For $A = (1, 0, 0)$, $B = (0, 2, 0)$, $C = (0, 0, 3)$: $\mathbf{n} = (-1, 2, 0) \times (-1, 0, 3) = (6, 3, 2)$ and $d = \mathbf{n} \cdot A = 6$, giving

$$6x + 3y + 2z = 6 \iff \frac{x}{1} + \frac{y}{2} + \frac{z}{3} = 1 \quad \text{(intercept form)}.$$

Reading geometry from normals.

Relationship	Condition on normals $\mathbf{n}_1, \mathbf{n}_2$
Planes parallel (or identical)	$\mathbf{n}_1 \times \mathbf{n}_2 = \mathbf{0}$
Planes perpendicular	$\mathbf{n}_1 \cdot \mathbf{n}_2 = 0$
Angle between planes	Angle between the normals (taking the acute one)
Line $\mathbf{p} + t\mathbf{d}$ parallel to the plane	$\mathbf{n} \cdot \mathbf{d} = 0$

Half-spaces. The sign of $\mathbf{n} \cdot \mathbf{x} - d$ tells you which side of the plane $\mathbf{x}$ is on:

        n . x - d > 0        (the side n points toward)
   ===================================   plane: n . x = d
        n . x - d < 0        (the opposite side)

In other dimensions. In $\mathbb{R}^2$, the equation $ax + by = c$ is a line with normal $(a, b)$. In $\mathbb{R}^n$, $\mathbf{a} \cdot \mathbf{x} = d$ is a hyperplane with normal $\mathbf{a}$. So every linear equation is a hyperplane, and solving a linear system (Chapter 3) means intersecting hyperplanes.

import numpy as np

A, B, C = np.array([1, 0, 0]), np.array([0, 2, 0]), np.array([0, 0, 3])
n = np.cross(B - A, C - A)                    # normal (6, 3, 2)
d = n @ A                                     # 6
print(f"{n[0]}x + {n[1]}y + {n[2]}z = {d}")   # 6x + 3y + 2z = 6

def side(X):
    s = n @ X - d                             # signed: positive on the side n points to
    return "on plane" if s == 0 else "same side as n" if s > 0 else "opposite side"

print(side(np.array([0, 0, 0])))              # opposite side   (0 - 6 < 0)
print(side(np.array([1, 1, 1])))              # same side as n  (11 - 6 > 0)
print(side(B))                                # on plane

Key takeaways

A plane is $\mathbf{n} \cdot (\mathbf{x} - \mathbf{p}) = 0$, that is, $ax + by + cz = d$, where $(a, b, c)$ is a normal vector.
A normal for the plane through $A, B, C$ is $(B - A) \times (C - A)$.
Parallelism and perpendicularity of planes reduce to the same questions about their normals.
The sign of $\mathbf{n} \cdot \mathbf{x} - d$ classifies points by side, and every linear equation in $n$ unknowns is a hyperplane.

🧪 Practice

Find the scalar equation of the plane through $(2, -1, 3)$ with normal $(1, 4, -2)$.

Find the scalar equation of the plane through $(1, 1, 1)$, $(2, 0, 1)$, and $(1, 3, 2)$, and check that all three points satisfy it.

Are the planes $2x - y + z = 4$ and $-4x + 2y - 2z = 1$ parallel? Find the angle between the planes $x + y = 1$ and $y + z = 2$.

Interview: A linear classifier predicts the class of $\mathbf{x}$ as $\operatorname{sign}(\mathbf{w} \cdot \mathbf{x} + b)$. Describe the decision boundary geometrically and explain the roles of $\mathbf{w}$ and $b$. Hint: compare $\mathbf{w} \cdot \mathbf{x} + b = 0$ with $\mathbf{n} \cdot \mathbf{x} = d$.

Distance from Point to Line and Plane

Why it matters. "How far is this point from that line or surface?" comes up in collision detection, in fitting lines and planes to data, in snapping a cursor to geometry, and in the margin of a support vector machine. Projection and the cross product answer it with closed-form formulas.

Point to plane. Let the plane be $\mathbf{n} \cdot \mathbf{x} = d$, passing through $P$, and let $X_0$ be any point. The shortest path from $X_0$ to the plane runs along the normal, so the distance is the length of the projection of $X_0 - P$ onto $\mathbf{n}$:

$$D = \lvert \operatorname{comp}_{\mathbf{n}}(X_0 - P) \rvert = \frac{\lvert \mathbf{n} \cdot (X_0 - P) \rvert}{\lVert \mathbf{n} \rVert} = \frac{\lvert \mathbf{n} \cdot X_0 - d \rvert}{\lVert \mathbf{n} \rVert}.$$

                 * X0
                /|
               / |
      X0 - P  /  | D = |n . X0 - d| / ||n||
             /   |
            /    |
   ========P=====F==================   plane n . x = d
                 F: foot of the perpendicular

The foot of the perpendicular, the closest point on the plane, is $F = X_0 - \dfrac{\mathbf{n} \cdot X_0 - d}{\mathbf{n} \cdot \mathbf{n}},\mathbf{n}$.

Example. $X_0 = (1, 1, 1)$ and the plane $6x + 3y + 2z = 6$: $D = \lvert 6 + 3 + 2 - 6 \rvert / \sqrt{36 + 9 + 4} = 5/7$.

Point to line in $\mathbb{R}^3$. For the line $\mathbf{p} + t\mathbf{d}$, the vectors $X_0 - P$ and $\mathbf{d}$ span a parallelogram with area $\lVert (X_0 - P) \times \mathbf{d} \rVert$. Its height above the base $\mathbf{d}$ is the distance:

$$D = \frac{\lVert (X_0 - P) \times \mathbf{d} \rVert}{\lVert \mathbf{d} \rVert}.$$

            * X0
            |
            | D  = (area of the parallelogram on X0 - P and d) / ||d||
            |
   ---------F----------P=====d====>-----   line p + t d

In any $\mathbb{R}^n$, use projection instead: $D = \lVert \mathbf{w} - \operatorname{proj}_{\mathbf{d}} \mathbf{w} \rVert$ with $\mathbf{w} = X_0 - P$.

Formula summary.

Distance	Formula
Point $X_0$ to plane $\mathbf{n} \cdot \mathbf{x} = d$	$\lvert \mathbf{n} \cdot X_0 - d \rvert / \lVert \mathbf{n} \rVert$
Point $(x_0, y_0)$ to line $ax + by = c$ in $\mathbb{R}^2$	$\lvert a x_0 + b y_0 - c \rvert / \sqrt{a^2 + b^2}$
Point $X_0$ to line $\mathbf{p} + t\mathbf{d}$ in $\mathbb{R}^3$	$\lVert (X_0 - P) \times \mathbf{d} \rVert / \lVert \mathbf{d} \rVert$
Parallel planes $\mathbf{n} \cdot \mathbf{x} = d_1$, $\mathbf{n} \cdot \mathbf{x} = d_2$	$\lvert d_1 - d_2 \rvert / \lVert \mathbf{n} \rVert$
Skew lines $\mathbf{p}_1 + t\mathbf{d}_1$, $\mathbf{p}_2 + s\mathbf{d}_2$	$\lvert (\mathbf{p}_2 - \mathbf{p}_1) \cdot (\mathbf{d}_1 \times \mathbf{d}_2) \rvert / \lVert \mathbf{d}_1 \times \mathbf{d}_2 \rVert$

For parallel planes, both equations must use the same normal $\mathbf{n}$; rescale one equation first if needed.

import numpy as np

def dist_point_plane(X0, n, d):
    return abs(n @ X0 - d) / np.linalg.norm(n)

def foot_on_plane(X0, n, d):
    return X0 - (n @ X0 - d) / (n @ n) * n            # step back along n by the signed distance

def dist_point_line(X0, P, u):
    return np.linalg.norm(np.cross(X0 - P, u)) / np.linalg.norm(u)   # parallelogram area / base length

n, d = np.array([6.0, 3.0, 2.0]), 6.0
X0 = np.array([1.0, 1.0, 1.0])
F = foot_on_plane(X0, n, d)
print(dist_point_plane(X0, n, d))                     # 0.7142857142857143 = 5/7
print(n @ F, np.linalg.norm(X0 - F))                  # 6.0 (F is on the plane), distance about 5/7 again

P, u = np.array([1.0, 2.0, 3.0]), np.array([3.0, -2.0, 2.0])
print(dist_point_line(np.zeros(3), P, u))             # 3.5397... = sqrt(213 / 17)

Key takeaways

Point-to-plane distance is $\lvert \mathbf{n} \cdot X_0 - d \rvert / \lVert \mathbf{n} \rVert$, the scalar projection onto the normal.
Point-to-line distance in $\mathbb{R}^3$ is the parallelogram area $\lVert (X_0 - P) \times \mathbf{d} \rVert$ divided by $\lVert \mathbf{d} \rVert$.
The closest point is found by stepping from $X_0$ along the normal (plane) or by projection (line).
Normalize equations consistently before comparing offsets of parallel planes.

🧪 Practice

Find the distance from $(2, -1, 3)$ to the plane $x + 2y - 2z = 4$.

Find the distance between the parallel planes $x + y + z = 1$ and $2x + 2y + 2z = 8$.

Find the distance from $(1, 0, 0)$ to the line through the origin and $(1, 1, 1)$, and the closest point on that line.

Interview: A support vector machine's decision boundary is $\mathbf{w} \cdot \mathbf{x} + b = 0$. Show that a point $\mathbf{x}_0$ lies at distance $\lvert \mathbf{w} \cdot \mathbf{x}_0 + b \rvert / \lVert \mathbf{w} \rVert$ from it. Why does maximizing the margin become minimizing $\lVert \mathbf{w} \rVert$ when the closest points satisfy $\lvert \mathbf{w} \cdot \mathbf{x} + b \rvert = 1$? Hint: apply the point-to-plane formula with $\mathbf{n} = \mathbf{w}$ and $d = -b$.

Intersections of Lines and Planes

Why it matters. Finding where a ray hits a surface is the core operation of ray tracing, and finding where two walls meet is basic CAD geometry. These intersection problems also preview Chapter 3: every intersection below is a small linear system, and each case (one point, infinitely many, none) matches a case of solution sets.

Line and plane. Substitute $\mathbf{x} = \mathbf{p} + t\mathbf{d}$ into $\mathbf{n} \cdot \mathbf{x} = c$:

$$\mathbf{n} \cdot \mathbf{p} + t,(\mathbf{n} \cdot \mathbf{d}) = c.$$

Condition	Result
$\mathbf{n} \cdot \mathbf{d} \ne 0$	One point, at $t = \dfrac{c - \mathbf{n} \cdot \mathbf{p}}{\mathbf{n} \cdot \mathbf{d}}$
$\mathbf{n} \cdot \mathbf{d} = 0$ and $\mathbf{n} \cdot \mathbf{p} = c$	The line lies in the plane (infinitely many points)
$\mathbf{n} \cdot \mathbf{d} = 0$ and $\mathbf{n} \cdot \mathbf{p} \ne c$	Parallel, no intersection

Example. The line $(1, 2, 3) + t(3, -2, 2)$ meets $6x + 3y + 2z = 6$ where $18 + 16t = 6$, so $t = -\tfrac{3}{4}$ and the point is $\left(-\tfrac{5}{4}, \tfrac{7}{2}, \tfrac{3}{2}\right)$.

Two planes. If the normals are parallel, the planes are either identical or disjoint. Otherwise they meet in a line:

Direction: the line lies in both planes, so it is perpendicular to both normals: $\mathbf{d} = \mathbf{n}_1 \times \mathbf{n}_2$.
A point: set one coordinate to 0 (choose one for which the remaining $2 \times 2$ system is solvable) and solve the two equations.

Example. $x + y + z = 6$ and $x - y + z = 2$: $\mathbf{d} = (1, 1, 1) \times (1, -1, 1) = (2, 0, -2)$. Setting $z = 0$ gives $x + y = 6$, $x - y = 2$, so $(4, 2, 0)$ is on both planes, and the line is $(4, 2, 0) + t(1, 0, -1)$.

Two lines in $\mathbb{R}^3$. Setting $\mathbf{p}_1 + t\mathbf{d}_1 = \mathbf{p}_2 + s\mathbf{d}_2$ gives three equations in two unknowns. In 2D, non-parallel lines always meet. In 3D there is a fourth case, skew lines, which are neither parallel nor intersecting (like a road and an overpass).

$\mathbf{d}_1 \times \mathbf{d}_2$	Extra test	Configuration
$= \mathbf{0}$	$\mathbf{p}_2 - \mathbf{p}_1$ parallel to $\mathbf{d}_1$	Same line
$= \mathbf{0}$	otherwise	Parallel, distinct
$\ne \mathbf{0}$	$(\mathbf{p}_2 - \mathbf{p}_1) \cdot (\mathbf{d}_1 \times \mathbf{d}_2) = 0$	Intersecting in one point
$\ne \mathbf{0}$	$(\mathbf{p}_2 - \mathbf{p}_1) \cdot (\mathbf{d}_1 \times \mathbf{d}_2) \ne 0$	Skew

The coplanarity test in the last two rows is the scalar triple product from Section 2.4.

Three planes. Three equations in three unknowns describe three planes:

Configuration	Number of common points	Normals
Meet at a single point (like a room's corner)	1	Not coplanar: $[\mathbf{n}_1, \mathbf{n}_2, \mathbf{n}_3] \ne 0$
Share a common line (pages of a book)	Infinitely many	Coplanar
All three coincide	Infinitely many	All parallel
Pairwise lines that never meet (a triangular prism)	0	Coplanar
Two or more parallel and distinct	0	At least two parallel

This table is the 3D picture of a $3 \times 3$ linear system. A unique solution occurs exactly when the normals, which are the rows of the coefficient matrix, are not coplanar, that is, when the determinant is nonzero (Chapter 8).

import numpy as np

def line_plane(p, d, n, c, tol=1e-12):
    """Intersect the line x = p + t d with the plane n . x = c."""
    denom = n @ d
    if abs(denom) < tol:                            # direction lies parallel to the plane
        return "line lies in plane" if abs(n @ p - c) < tol else "no intersection"
    t = (c - n @ p) / denom
    return p + t * d

print(line_plane(np.array([1., 2., 3.]), np.array([3., -2., 2.]),
                 np.array([6., 3., 2.]), 6.0))      # [-1.25  3.5   1.5 ]

def classify_lines(p1, d1, p2, d2, tol=1e-12):
    cr = np.cross(d1, d2)
    w = p2 - p1
    if np.linalg.norm(cr) < tol:                    # parallel directions
        return "same line" if np.linalg.norm(np.cross(w, d1)) < tol else "parallel"
    return "skew" if abs(w @ cr) > tol else "intersecting"   # scalar triple product: coplanar or not

o = np.zeros(3)
print(classify_lines(o, np.array([1., 0, 0]), np.array([0., 1, 0]), np.array([0., 0, 1])))  # skew
print(classify_lines(o, np.array([1., 1, 0]), np.array([2., 0, 0]), np.array([0., 1, 0])))  # intersecting (at (2, 2, 0))

# Two planes: direction from the cross product, a point from a 2x2 solve with z = 0
n1, n2 = np.array([1., 1, 1]), np.array([1., -1, 1])
direction = np.cross(n1, n2)                        # [ 2.  0. -2.]
xy = np.linalg.solve(np.array([[1., 1], [1, -1]]), np.array([6., 2]))
print(direction, np.append(xy, 0.0))                # [ 2.  0. -2.] [4. 2. 0.]

Key takeaways

Line and plane: substitute the line into the plane equation; $\mathbf{n} \cdot \mathbf{d} = 0$ signals the parallel cases.
Two non-parallel planes meet in a line with direction $\mathbf{n}_1 \times \mathbf{n}_2$.
Two lines in $\mathbb{R}^3$ can be identical, parallel, intersecting, or skew; the scalar triple product distinguishes intersecting from skew.
Every intersection problem is a small linear system, and its cases (one, infinitely many, or no points) match the cases studied in Chapter 3.

🧪 Practice

Find where the line $(0, 0, 10) + t(2, 1, -1)$ meets the plane $x + y + z = 4$.

Find a vector equation for the line of intersection of $x + 2y - z = 1$ and $2x + y + z = 5$.

Classify the lines $(1, 0, 0) + t(1, 1, 1)$ and $(0, 1, 0) + s(1, -1, 0)$, and find their intersection point if one exists.

Interview: Three planes in $\mathbb{R}^3$ can meet in a single point, along a line, in a whole plane, or not at all. Describe each configuration in terms of the normal vectors, and say which corresponds to a $3 \times 3$ linear system with exactly one solution. Hint: start by asking whether the three normals are coplanar, and use the scalar triple product to test it.

← Linear Algebra