Systems of Linear Equations

Systems of linear equations look like something you already solved in middle school, but the three questions — “when does a system have a solution, and how many solutions does it have when it does?” — can only be answered completely once the language of linear algebra is available. The answer is: compare the rank of the coefficient matrix with the rank of the augmented matrix.

This chapter begins with the basic concepts of systems of equations and distinguishes the essential difference between the homogeneous and the non-homogeneous case. It then offers three routes to a solution: Gaussian elimination is the general-purpose algorithmic tool; Cramer’s rule produces an explicit solution when the number of equations equals the number of unknowns (at the cost of a great deal of computation); and the structural analysis of the solution set — fundamental systems of solutions and the general solution — is the real key to answering “how many solutions are there”.

Pay particular attention to the structural difference between the homogeneous and the non-homogeneous case: the solution set of a homogeneous system is a null space, which is itself a linear space; the solution set of a non-homogeneous system is “one particular solution plus the null space”, which is an affine space and does not contain the origin. This distinction is used again and again in later topics such as duality and least squares.

This chapter and the previous one are two sides of the same coin: the column space decides “whether the system has a solution”, and the null space decides “how many degrees of freedom the solutions have”.