Kuzee
Vector Algebra and Analytic Geometry of Space

Inner Products and Orthogonalisation

Inner Product Spaces

Inner product space

Let VV be a real vector space. Suppose that for any a⃗,b⃗∈V\vec{a}, \vec{b} \in V a real number (a⃗,b⃗)(\vec{a}, \vec{b}) is defined, called the inner product of a⃗\vec{a} and b⃗\vec{b}, satisfying:

  1. Symmetry: (a⃗,b⃗)=(b⃗,a⃗)(\vec{a}, \vec{b}) = (\vec{b}, \vec{a})
  2. Linearity: (ka⃗+b⃗,c⃗)=k(a⃗,c⃗)+(b⃗,c⃗)(k\vec{a} + \vec{b}, \vec{c}) = k(\vec{a}, \vec{c}) + (\vec{b}, \vec{c})
  3. Positive definiteness: (a⃗,a⃗)≥0(\vec{a}, \vec{a}) \geq 0, with equality if and only if a⃗=0⃗\vec{a} = \vec{0}

then VV is called an inner product space.

Euclidean Space

The Euclidean inner product

In Rn\mathbb{R}^n, the standard inner product is:

(a⃗,b⃗)=a1b1+a2b2+⋯+anbn=a⃗⋅b⃗(\vec{a}, \vec{b}) = a_1b_1 + a_2b_2 + \dots + a_nb_n = \vec{a} \cdot \vec{b}

The Length and the Angle of Vectors

The Length (Norm) of a Vector

The length of a vector

∥a⃗∥=(a⃗,a⃗)=a⃗⋅a⃗\|\vec{a}\| = \sqrt{(\vec{a}, \vec{a})} = \sqrt{\vec{a} \cdot \vec{a}}

The Angle Between Two Vectors

The angle between two vectors

cos⁡θ=(a⃗,b⃗)∥a⃗∥∥b⃗∥\cos\theta = \frac{(\vec{a}, \vec{b})}{\|\vec{a}\| \|\vec{b}\|}

Orthogonal Vectors

Orthogonal vectors

If (a⃗,b⃗)=0(\vec{a}, \vec{b}) = 0, the vectors a⃗\vec{a} and b⃗\vec{b} are called orthogonal, written a⃗⊥b⃗\vec{a} \perp \vec{b}.

A Set of Orthogonal Vectors

A set of orthogonal vectors

If any two distinct vectors in a set of vectors are orthogonal, the set is called a set of orthogonal vectors.

A Set of Orthonormal Vectors

A set of orthonormal vectors

If a set of vectors is a set of orthogonal vectors and the length of every vector in it is 1, the set is called a set of orthonormal vectors (or a normalised orthogonal set).

The Gram-Schmidt Orthogonalisation Method

Theorem1

Any linearly independent set of vectors can be turned into a set of orthonormal vectors by the Gram-Schmidt orthogonalisation method.

The Gram-Schmidt Procedure

Let a⃗1,a⃗2,…,a⃗n\vec{a}_1, \vec{a}_2, \dots, \vec{a}_n be a linearly independent set of vectors; construct the orthogonal set b⃗1,b⃗2,…,b⃗n\vec{b}_1, \vec{b}_2, \dots, \vec{b}_n:

Gram-Schmidt orthogonalisation
b⃗1=a⃗1b⃗2=a⃗2−(a⃗2,b⃗1)(b⃗1,b⃗1)b⃗1b⃗3=a⃗3−(a⃗3,b⃗1)(b⃗1,b⃗1)b⃗1−(a⃗3,b⃗2)(b⃗2,b⃗2)b⃗2⋮b⃗k=a⃗k−∑j=1k−1(a⃗k,b⃗j)(b⃗j,b⃗j)b⃗j\begin{aligned} \vec{b}_1 &= \vec{a}_1 \\ \vec{b}_2 &= \vec{a}_2 - \frac{(\vec{a}_2, \vec{b}_1)}{(\vec{b}_1, \vec{b}_1)} \vec{b}_1 \\ \vec{b}_3 &= \vec{a}_3 - \frac{(\vec{a}_3, \vec{b}_1)}{(\vec{b}_1, \vec{b}_1)} \vec{b}_1 - \frac{(\vec{a}_3, \vec{b}_2)}{(\vec{b}_2, \vec{b}_2)} \vec{b}_2 \\ &\vdots \\ \vec{b}_k &= \vec{a}_k - \sum_{j=1}^{k-1} \frac{(\vec{a}_k, \vec{b}_j)}{(\vec{b}_j, \vec{b}_j)} \vec{b}_j \end{aligned}

Normalising

Turn the orthogonal set into an orthonormal one:

Normalisation

e⃗i=b⃗i∥b⃗i∥\vec{e}_i = \frac{\vec{b}_i}{\|\vec{b}_i\|}

Orthogonal Matrices

Orthogonal matrix

If a real matrix QQ satisfies QTQ=IQ^T Q = I (equivalently, QT=Q−1Q^T = Q^{-1}), then QQ is called an orthogonal matrix.

Properties of an Orthogonal Matrix

Theorem2

An orthogonal matrix has the following properties:

  1. QT=Q−1Q^T = Q^{-1}
  2. ∣Q∣=±1|Q| = \pm 1
  3. The columns (rows) of QQ form a set of orthonormal vectors
  4. An orthogonal transformation preserves the lengths and the angles of vectors

Exercises

Exercise 1

Use the Gram-Schmidt orthogonalisation method to orthogonalise the set of vectors {(1,1,0),(1,0,1)}\{(1,1,0), (1,0,1)\}.

Gram-Schmidt orthogonalisationorthogonal vectorsthe computation

Approach:

Work through the Gram-Schmidt formula step by step.

Steps:

  1. b⃗1=a⃗1=(1,1,0)\vec{b}_1 = \vec{a}_1 = (1,1,0)

  2. Compute b⃗2\vec{b}_2: b⃗2=a⃗2−(a⃗2,b⃗1)(b⃗1,b⃗1)b⃗1\vec{b}_2 = \vec{a}_2 - \frac{(\vec{a}_2, \vec{b}_1)}{(\vec{b}_1, \vec{b}_1)} \vec{b}_1 (a⃗2,b⃗1)=(1,0,1)⋅(1,1,0)=1+0+0=1(\vec{a}_2, \vec{b}_1) = (1,0,1) \cdot (1,1,0) = 1 + 0 + 0 = 1 (b⃗1,b⃗1)=(1,1,0)⋅(1,1,0)=1+1+0=2(\vec{b}_1, \vec{b}_1) = (1,1,0) \cdot (1,1,0) = 1 + 1 + 0 = 2 b⃗2=(1,0,1)−12(1,1,0)=(1,0,1)−(0.5,0.5,0)=(0.5,−0.5,1)\vec{b}_2 = (1,0,1) - \frac{1}{2}(1,1,0) = (1,0,1) - (0.5, 0.5, 0) = (0.5, -0.5, 1)

  3. The orthogonal set: {(1,1,0),(0.5,−0.5,1)}\{(1,1,0), (0.5,-0.5,1)\}

Answer: the orthogonal set is {(1,1,0),(0.5,−0.5,1)}\{(1,1,0), (0.5,-0.5,1)\}

Exercise 2

Decide whether the matrix Q=(121212−12)Q = \begin{pmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{pmatrix} is an orthogonal matrix.

an orthogonal matrixproperties of a matrixthe determinant

Approach:

Verify that QTQ=IQ^T Q = I.

Steps:

  1. Compute QTQ^T: QT=(121212−12)Q^T = \begin{pmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{pmatrix}

  2. Compute QTQQ^T Q: QTQ=(121212−12)(121212−12)Q^T Q = \begin{pmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{pmatrix} \begin{pmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \end{pmatrix}

    The result: (12+12,12−12)=(1,0)( \frac{1}{2} + \frac{1}{2}, \frac{1}{2} - \frac{1}{2} ) = (1, 0) (12−12,12+12)=(0,1)( \frac{1}{2} - \frac{1}{2}, \frac{1}{2} + \frac{1}{2} ) = (0, 1)

    So QTQ=IQ^T Q = I

  3. The determinant: det⁡Q=12(−12)−12⋅12=−12−12=−1\det Q = \frac{1}{\sqrt{2}}(-\frac{1}{\sqrt{2}}) - \frac{1}{\sqrt{2}}\cdot\frac{1}{\sqrt{2}} = -\frac{1}{2} - \frac{1}{2} = -1

Answer: QQ is an orthogonal matrix

Summary

Symbols Used in This Article

SymbolTypePronunciation/ExplanationMeaning in This Article
(a⃗,b⃗)(\vec{a}, \vec{b})Inner productinner productThe inner product of the vectors a\mathbf{a} and b\mathbf{b}
∥a⃗∥\|\vec{a}\|Vector normvector normThe length of the vector
a⃗⊥b⃗\vec{a} \perp \vec{b}OrthogonalorthogonalThe relation of orthogonality between vectors
QQOrthogonal matrixorthogonal matrixA matrix satisfying the orthogonality property
Previous
Formulas for the Vector Products
Next
Analytic Geometry of Space

Found an issue with this content?

Tell us what could be improved.