Vector Spaces, Bases, and Dimension

I. Breaking the Limits of the “Arrow”

So far we have treated a vector as an “arrow in space”. But that view has limits — the real power of linear algebra lies in the fact that a “vector” can be any object that obeys the rules of operation.

Think about it: from beginning to end we have used only two operations — addition and scalar multiplication. As soon as an object can be “added” and “multiplied by a number”, and satisfies those properties, it can be used as a “vector” with every tool of linear algebra!

II. Generalising the Vector: More Things That “Masquerade as Arrows”

1. Functions Are Vectors Too

Functions can be added and multiplied by a number:

$$(f + g)(x) = f(x) + g(x), \quad (cf)(x) = c\,f(x)$$

So functions form a vector space! A “linear combination” of functions is $c_1f_1 + c_2f_2 + \cdots$.

This brings about a startling fact: the derivative operator of calculus is a “linear transformation”!

$$\frac{d}{dx}(f + g) = \frac{df}{dx} + \frac{dg}{dx}, \quad \frac{d}{dx}(cf) = c\frac{df}{dx}$$

Differentiation satisfies the two properties of a linear transformation — so differentiation is a “linear transformation on a space of functions”, and its “eigenvectors” satisfy $\frac{df}{dx} = \lambda f$, which solves to $f(x) = Ce^{\lambda x}$.

2. Other Vector Spaces

Key point: they all obey the same “addition + scalar multiplication + eight axioms” rule, so every tool of linear algebra (bases, linear transformations, eigenvalues…) applies to all of them.

III. The Axiomatic Definition of a Vector Space

A set $V$ is a vector space if it satisfies:

1. Closure under addition: $\mathbf{v} + \mathbf{w} \in V$
2. Closure under scalar multiplication: $c\mathbf{v} \in V$
3. Associativity, commutativity, a zero element and an additive inverse for addition
4. Associativity, distributivity and an identity element for scalar multiplication

Satisfying these is all it takes — whatever the “vectors” inside happen to look like.

IV. The Space of Polynomials and the Space of Matrices

The Space of Polynomials

All polynomials of degree ≤ 2 form a three-dimensional vector space, with basis

$$1, \quad x, \quad x^2$$

Every quadratic polynomial is a linear combination of them:

$$a_0 + a_1x + a_2x^2 = a_0\cdot 1 + a_1\cdot x + a_2\cdot x^2$$

Dimension = 3 (the number of coefficients).

The Space of Matrices

All $2\times2$ matrices form a four-dimensional vector space, with basis

$$\begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}$$

Every $2\times2$ matrix is a linear combination of them, so dimension = 4.

V. Why This Matters

1. Unity: differential equations, Fourier series and machine learning can all be described in the language of linear algebra
2. Reusable tools: a method learned in one vector space transfers to every vector space
3. The power of abstraction: ignore the appearance and grasp the essence, which is the “linear structure”

The Fourier viewpoint: the Fourier transform = “changing the basis” of a function to the sine/cosine basis (the frequency basis) — this is exactly the change-of-basis idea. The sine functions are the eigenfunctions of the “translation operator”.

The final section of the course, Applications of Linear Algebra, surveys the application landscape of the whole discipline.

VI. One Vector, Different Coordinates

The vector $(3, 2)$ means "$3\hat{\imath} + 2\hat{\jmath}$" in the standard basis $\{\hat{\imath}, \hat{\jmath}\}$.

But if we change to another basis $\{\mathbf{b}_1, \mathbf{b}_2\}$ (say $\mathbf{b}_1 = (1, 1)$ and $\mathbf{b}_2 = (1, -1)$), the same vector has different coordinates — it is now expressed as a combination of $\mathbf{b}_1, \mathbf{b}_2$.

The same point:
in the standard basis: v = (3, 2)
in the new basis:      v = 2.5·b₁ + 0.5·b₂   (because 2.5(1,1) + 0.5(1,-1) = (3, 2))

Coordinates are not an intrinsic property of a vector; they are a description “relative to some basis”.

VII. The Change-of-Basis Matrix

From New-Basis Coordinates to Standard-Basis Coordinates

Let the new basis be $\{\mathbf{b}_1, \mathbf{b}_2\}$ (written in standard-basis coordinates). If the coordinates of a vector in the new basis are $[\mathbf{v}]_\mathcal{B} = (x', y')^T$, then its standard-basis coordinates are

$$\mathbf{v} = x'\mathbf{b}_1 + y'\mathbf{b}_2 = \underbrace{\begin{pmatrix} | & | \\ \mathbf{b}_1 & \mathbf{b}_2 \\ | & | \end{pmatrix}}_{B} \begin{pmatrix} x' \\ y' \end{pmatrix}$$

The columns of the change-of-basis matrix $B$ are the new basis vectors (in standard-basis coordinates).

From Standard-Basis Coordinates to New-Basis Coordinates

Conversely, use $B^{-1}$:

$$[\mathbf{v}]_\mathcal{B} = B^{-1}\mathbf{v}$$

Example: B = [[1, 1], [1, -1]] (b₁ = (1,1), b₂ = (1,-1))
B⁻¹ = (1/-2)[[-1, -1], [-1, 1]] = [[0.5, 0.5], [0.5, -0.5]]
Check: B⁻¹·(3, 2) = (0.5×3+0.5×2, 0.5×3−0.5×2) = (2.5, 0.5) ✓
(consistent with 2.5b₁ + 0.5b₂ above)

VIII. A Linear Transformation Under a Change of Coordinates: Similar Matrices

The Question

The same linear transformation $T$ is represented by a matrix $A$ in the standard basis and by a matrix $A'$ in the new basis $\mathcal{B}$. What is the relationship between $A'$ and $A$?

The Derivation

1. First convert the vector from new-basis coordinates back to standard-basis coordinates: $B$
2. Apply the transformation: $A$
3. Convert back to new-basis coordinates: $B^{-1}$

$$A' = B^{-1} A B$$

$A'$ and $A$ are called similar.

IX. Why Change of Basis Matters

The benefit of diagonalisation: computations with a diagonal matrix are extremely simple (entry by entry), and powers are easy as well:

$$A^k = B \begin{pmatrix} \lambda_1^k & 0 \\ 0 & \lambda_2^k \end{pmatrix} B^{-1}$$

X. An Intuitive Summary

Summary

Symbols Used in This Article