Quadratic Forms

A quadratic form is an expression of the shape $f(x)=x^TAx$. On the surface it looks like a special case of a polynomial, but structurally it is one of the most important objects in the subject: a single formula gathers together whole families of problems that otherwise look unrelated — ellipses against hyperbolas, positive against indefinite forms, and the energy functions that turn up in physics, optimisation and statistics alike.

This chapter builds that structure from nothing. It opens with the definition of a quadratic form and the symmetric matrix that represents it, and explains why restricting attention to symmetric matrices loses nothing at all: every matrix splits into a symmetric part and a skew-symmetric part, and $x^TAx$ only ever sees the symmetric half. From there the chapter studies congruence — how the matrix of a quadratic form changes when the basis changes — and asks the question that makes congruence interesting: why the inertia of a form, that is, how many positive, negative and zero coefficients survive, cannot change no matter which basis you pick. That invariance sits neatly beside the similarity relation introduced earlier. Similarity preserves eigenvalues, congruence preserves inertia, and the two together explain exactly what a change of coordinates can and cannot reveal.

The last idea is positive definiteness. It can be read in several equivalent ways — every eigenvalue positive, every leading principal minor positive, the form congruent to the identity — and the ability to switch between those readings is the single most useful habit the chapter tries to build. Positive definiteness is what turns an abstract quadratic form into a geometric object such as an ellipsoid, an optimisation object such as a convex bowl with a unique minimiser, and a numerical object such as a matrix that admits a Cholesky factorisation.

By the end of this chapter you should be able to write the matrix of a quadratic form, reduce it to a standard form by congruence, read the inertia off that standard form, and decide whether the form is positive definite — and, just as importantly, you should be able to say why none of those answers depends on the basis you happened to start from. Quadratic forms tie together everything that came before: diagonalisation produces a standard form, similarity guarantees that the eigenvalues survive, and congruence reveals the invariants that no coordinate system is able to touch.