ii Bruce N. Cooperstein Elementary Linear Algebra: An eTextbook Email: coop@ucsc.edu c © 2010, All Rights Reserved. Version 2.0 February 24, 2016 Contents 1 Linear Equations 3 1.1 Linear Equations and Their Solution . . . . . . . . . . . . . . . . . . 3 1.2 Matrices and Echelon Forms . . . . . . . . . . . . . . . . . . . . . . 32 1.3 How to Use it: Applications of Linear Systems . . . . . . . . . . . . 62 2 The Vector Space R n 91 2.1 Introduction to Vectors: Linear Geometry . . . . . . . . . . . . . . . 91 2.2 Vectors and the Space R n . . . . . . . . . . . . . . . . . . . . . . . . 116 2.3 The Span of a Sequence of Vectors . . . . . . . . . . . . . . . . . . . 140 2.4 Linear independence in R n . . . . . . . . . . . . . . . . . . . . . . . 165 2.5 Subspaces and Bases of R n . . . . . . . . . . . . . . . . . . . . . . . 194 2.6 The Dot Product in R n . . . . . . . . . . . . . . . . . . . . . . . . . 223 3 Matrix Algebra 251 3.1 Introduction to Linear Transformations and Matrix Multiplication . . 251 3.2 The Product of a Matrix and a Vector . . . . . . . . . . . . . . . . . . 273 3.3 Matrix Addition and Multiplication . . . . . . . . . . . . . . . . . . . 294 3.4 Invertible Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 3.5 Elementary Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 340 3.6 The LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . 361 3.7 How to Use It: Applications of Matrix Multiplication . . . . . . . . . 379 4 Determinants 403 4.1 Introduction to Determinants . . . . . . . . . . . . . . . . . . . . . . 403 4.2 Properties of Determinants . . . . . . . . . . . . . . . . . . . . . . . 421 4.3 The Adjoint of a Matrix and Cramer’s Rule . . . . . . . . . . . . . . 447 5 Abstract Vector Spaces 461 5.1 Introduction to Abstract Vector Spaces . . . . . . . . . . . . . . . . . 461 5.2 Span and Independence in Vector Spaces . . . . . . . . . . . . . . . . 478 5.3 Dimension of a finite generated vector space . . . . . . . . . . . . . . 510 5.4 Coordinate vectors and change of basis . . . . . . . . . . . . . . . . . 529 5.5 Rank and Nullity of a Matrix . . . . . . . . . . . . . . . . . . . . . . 551 iv CONTENTS 5.6 Complex Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . 573 5.7 Vector Spaces Over Finite Fields . . . . . . . . . . . . . . . . . . . . 600 5.8 How to Use it: Error Correcting Codes . . . . . . . . . . . . . . . . . 615 6 Linear Transformations 635 6.1 Introduction to Linear Transformations on Abstract Vector Spaces . . 635 6.2 Range and Kernel of a Linear Transformation . . . . . . . . . . . . . 656 6.3 Matrix of a Linear Transformation . . . . . . . . . . . . . . . . . . . 680 7 Eigenvalues and Eigenvectors 703 7.1 Introduction to Eigenvalues and Eigenvectors . . . . . . . . . . . . . 703 7.2 Diagonalization of Matrices . . . . . . . . . . . . . . . . . . . . . . 727 7.3 Complex Eigenvalues of Real Matrices . . . . . . . . . . . . . . . . . 753 7.4 How to Use It: Applications of Eigenvalues and Eigenvectors . . . . . 780 8 Orthogonality in R n 803 8.1 Orthogonal and Orthonormal Sets in R n . . . . . . . . . . . . . . . . 803 8.2 The Gram-Schmidt Process and QR-Factorization . . . . . . . . . . 824 8.3 Orthogonal Complements and Projections . . . . . . . . . . . . . . . 843 8.4 Diagonalization of Real Symmetric Matrices . . . . . . . . . . . . . . 869 8.5 Quadratic Forms, Conic Sections and Quadratic Surfaces . . . . . . . 895 8.6 How to Use It: Least Squares Approximation . . . . . . . . . . . . . 932 Introduction: How to Use This Book This book will probably be unlike any textbook you have ever used and it has been specifically written to facilitate your learning in elementary linear algebra. It has been my experience over more than forty five years of teaching linear algebra that student success is highly correlated with mastery of the definitions and concepts of which there are certainly more then fifty and very likely as many as one hundred. In a typical book if there is a term or concept that you are not completely familiar with you would go to the index and find the page where it is defined. In a math class such as this, often when you do that you will find that the definition makes use of other concepts which you have not mastered which would send you back to the index or, more likely, give up. This digital textbook makes it easy to study and master the definitions in two ways. First, every instance of a fundamental concept is linked to its definition so you can look it up just by clicking and then you can return to your original place. Second, at the very beginning of every section is a subsection, “ Am I Ready for This Material ” you will find listed all the concepts that have been previously introduced and used in the section. You will also find a short quiz which tests whether you have mastered the methods from previous sections which will be used in the current one. The quiz is linked to solutions and these, in turn, are linked to explanations of how you do that kind of exercise. Subsequent to the “readiness subsection” is a subsection referred to as “ New Con- cepts ”. Here you are alerted to the definitions which will be introduced in the section and they are linked to the place where they are introduced. Following is the “heart” of the section, a subsection referred to as “ Theory: Why It Works. ” Here the new concepts are introduced and theorems about them are proved. The theorems are then used throughout the text to prove other theorems. A particular feature of the book is that any citation of a theorem to prove another theorem is linked back to its original statement and its proof. Next comes a feature that is missing from most elementary linear algebra textbooks to the frustration of many students. First, a subsection entitled “ What You Can Now Do .” This will alert you to the new types of exercises that you can solve based on the theorems proved in the Theory subsection. Then there is a subsection, “ Method: How 2 CONTENTS to do it ” which provides explicit algorithms, essentially recipes, for solving these ex- ercises. In each instance, these are illustrated with multiple examples. It is the absence of these algorithms from most expensive hard copy textbooks that sends students to Amazon.com to spend an addition $15-$20 to purchase Schaum’s Outline of Linear Algebra in order to figure out how to get the homework done. Note : If you are only reading the Theory subsection you may get the impression that the book is bereft of examples. You have to go to the Method subsection and there you will find multiple examples, far more than in any other text. Finally, the section ends with “ Exercises ” that are done using the methods introduced (and are generally linked to the methods in order to prime you about what to use and how to proceed as well as “ Challenge Exercises (CE) ” or problems that require you to do some real mathematical reasoning making use of the theorems that were proved with each CE linked to the theorem or theorems that you need to cite. Hopefully this text eases you way through elementary linear algebra. Typically text- books are written for professors, not students, because professors choose the book and students then have to buy it and what professors want in a book is often very different then what a student wants in a book, especially when the majority of students are not in the course in order to become mathematics major but rather are studying engineer- ing, computer science, physics, economics or some other subject. If there are features missing from the book let me know at coop@ucsc.edu. Likewise, if there are portions that are unclear or confusing, also let me know since I see this as a continuous project of refinement and improvement. Bruce Cooperstein Professor, Department of Mathematics University of California, Santa Cruz Chapter 1 Linear Equations 1.1. Linear Equations and Their Solution In this section we review the concepts of a linear equation and linear system. We develop systematic methods for determining when a linear system has a solution and for finding the general solution. Below is a guide to what you find in this section. Am I Ready for This Material Readiness Quiz New Concepts Theory (Why It Works) What You Can Now Do Method (How To Do It) Exercises Challenge Exercises (Problems) 4 Chapter 1. Linear Equations Am I Ready for This Material To be prepared for linear algebra you must have a complete mastery of pre-calculus, some mathematical sophistication, some experience with proofs and the willingness to work hard. More specifically, at the very least you need to be comfortable with algebraic notation, in particular, the use of literals in equations to represent arbitrary numbers as well as the application of subscripts. You should also be able to quickly solve a single linear equation in one variable as well as a system of two linear equations in two variables. You will also need some mathematical sophistication in order to follow the arguments of the proofs and even attempt some mathematical reasoning of your own. The most important requirement is that you are willing to perspire, that is, put in the necessary effort. Each section will introduce many new concepts and these have to be understood and mastered before moving on. “Kind of” understanding will not do. Nor will an “intuitive” understanding be sufficient - you will be often asked to fol- low deductive arguments and even do some proofs yourself. These depend on a deep and genuine understanding of the concepts, and there are lots of them, certainly over fifty major definitions. So, its important to keep up and to periodically return to the definitions and make sure that you truly understand them. With that in mind let me share with you my three “axioms” of time management developed over forty years of teaching (and putting two sons through university): ◦ However much time you think you have, you always have less. ◦ However much time you think it will take, it will always take more. ◦ Something unexpected will come up. Linear algebra is a beautiful subject and at the same time accessible to first and second year mathematics students. I hope this book conveys the elegance of the subject and is useful to your learning. Good luck. Before going on to the material of this section you should take the quiz here to see if you have the minimum mathematical skills to succeed in linear algebra. Quiz 1. Solve the equation 3( x − 2) + 7 − 2( x + 4) = x − 11 2. Solve the linear equation: 1.1. Linear Equations and Their Solution 5 4( x − 2) − 2( x − 4) = 5( x − 4) − 4( x − 5) 3. Solve the linear equation: 3(2 − x ) − 2(3 − x ) = 4(3 − x ) − 3(4 − x ) 4. Solve system of two equations in two unknowns: 2 x + 3 y = 4 − 3 x + y = 5 Quiz Solutions New Concepts linear equation solution of linear equation solution set of a linear equation equivalent linear equations standard form of a linear equation homogeneous linear equation inhomogeneous linear equation leading term linear equation in standard form leading variable of a linear equation in standard form free variables of a linear equation in standard form linear system constants of a linear system coefficients of a linear system homogenous linear system inhomogenous linear system solution of a linear system solution set of a linear system 6 Chapter 1. Linear Equations consistent linear system inconsistent linear system the trivial solution to a homogeneous linear system non-trivial solution to a homogeneous linear system equivalent linear systems echelon form for a linear system leading variable free variable elementary equation operation Theory (Why It Works) Before getting started a brief word about an important convention. This book will be dealing with particular types of equations, linear equations, which we define imme- diately below. Equations involve “variables”. Usually these are real variables which means that we can substitute real numbers for them and the result will be a state- ment about the equality of two numbers. When there are just a few variables, two or three or possibly four, we will typically use letters at the end of the alphabet, for example x, y for two, x, y, z when there are three variables, and sometimes w, x, y, z when there are four variables. When there are more than a few we will use a sin- gle letter but “subscript” the letter with positive whole numbers starting at one, for example, x 1 , x 2 , x 3 , x 4 , x 5 , . . . The subscript of such a variable is called its “index.” We will also consider our variables to be ordered. The ordering is the natural one for x 1 , x 2 , x 3 , x 4 , . . . while for smaller sets of variables we will use the alphabetical ordering: x, y for two variables, x, y, z for three and w, x, y, z for four. We now begin our discussion with a definition that is probably familiar from previous mathematics you have studied: Definition 1.1. A linear equation is an equation of the form b 1 x 1 + b 2 x 2 + · · · + b n x n + c = d 1 x 1 + d 2 x 2 + · · · + d n x n + e (1.1) In Equation (1.1) the variables are x 1 , x 2 , . . . , x n The b i , d i stand for (arbitrary but fixed) real numbers and are the coefficients of the variables and c, e are (arbitrary but fixed) real numbers which are the constants of the equation. 1.1. Linear Equations and Their Solution 7 An example of the simplest type of linear equation , in its most general form, is the following: ax + b = cx + d Example 1.1.1. More concretely we have the equation 5 x − 11 = 2 x + 7 (1.2) When we solve Equation (1.2), that is, find the real numbers we can substitute for x to make both sides the same number, we do the following things: 1. Add − 2 x to both sides to get 3 x − 11 = 7 (1.3) Implicit in this operation is that we choose − 2 x because 2 x + ( − 2 x ) = 0 Secondly, we actually add − 2 x to 5 x − 11 and 2 x + 7 , which respectively yields ( − 2 x ) + (5 x − 11) and ( − 2 x ) + (2 x + 7) We are using the fact that ( − 2 x )+(5 x − 11) = ( − 2 x +5 x ) − 11 and ( − 2 x )+(2 x +7) = ( − 2 x + 2 x ) + 7 and also asserting that when equals are added to equals the results are equal. Now the second expression becomes ( − 2 + 2) x + 7 It then becomes 0 x + 7 and we are using the fact that 0 x = 0 and 0 + 7 = 7 Let’s return to the equation which has been transformed into 3 x − 11 = 7 2. We now add 11 to both sides. We choose 11 because − 11 + 11 = 0 This gives (3 x − 11) + 11 = 7 + 11 = 18 , 3 x + ( − 11 + 11) = 18 , 3 x + 0 = 18 and finally 3 x = 18 3. Now we divide by 3 or what amounts to the same thing we multiple by 1 3 to obtain 1 3 (3 x ) = 1 3 × 18 = 6 , [ 1 3 × 3] x = 6 , and 1 x = 6 whence x = 6 The choice of the 1 3 was made since it is the number which, when multiplied by 3, yields 1. Note that the last equation has a transparent “solution” (actually, we have not defined this yet), namely, the only value that can be substituted for x in this equation to make it a true statement is the number 6 and this is the solution of the original equation (check this). 8 Chapter 1. Linear Equations To summarize we used the following facts about our system of numbers in Example (1.1.1): (A1) We have an operation called addition which takes two numbers a, b and combines them to obtain the sum a + b which is another number. (A2) For numbers a, b, c addition satisfies the associative law a + ( b + c ) = ( a + b ) + c (A3) There is the existence of a neutral element for addition also called an additive identity. This means that there is a special number 0 which satisfies 0 + a = a for every number a. (A4) For every number there is an additive inverse , that is, every number a has an opposite or negative − a such that a + ( − a ) = 0 (A5) We did not use it, but also addition satisfies the commutative law a + b = b + a. (M1) There is also an operation, multiplication, which takes two numbers a, b and com- bines them to obtain the product ab which is also a number. This operation additionally satisifies: (M2) Multiplication is commutative that is, for any two numbers a, b ab = ba. (M3) Also, multiplication satisfies the associative law : for numbers a, b and c we have a ( bc ) = ( ab ) c. (M4) There is a neutral element for multiplication also called a multiplicative identity, namely the number 1 satisfies 1 a = a for every number a. (M5) We also used the fact that every number (excepting 0) has a multiplicative oppo- site or inverse which we denote by 1 a or a − 1 which satisfies a × a − 1 = a − 1 × a = 1 (M6) Finally, the distributive law holds: For any numbers a, b, c we have a ( b + c ) = ab + ac. The first person to extract these properties and study systems of numbers which satisfy them was Lagrange . Such a system was first referred to as a system of rationality but today we usually refer to it as a field Examples are: the fractions or rational numbers which we denote by Q , the real numbers, R , and the complex numbers, C Apart from defining a field, these properties are exceedingly important to us because they are similar to the axioms for a vector space which is the central concept in linear algebra. To remind you the rational field or simply rationals consists of all numbers m n where m, n are integers, n 6 = 0 1.1. Linear Equations and Their Solution 9 The real field or reals consists of all the possible decimals. The complex field consists of all expressions of the form a + bi where a and b are real numbers and i = √− 1 , that is, satisfies i 2 = − 1 Complex numbers are added and multiplied in the following way: ( a + bi ) + ( c + di ) = [ a + c ] + [ b + d ] i and ( a + bi )( c + di ) = [ ac − bd ] + [ ad + bc ] i. The next simplest type of linear equation (and why they are so-called) has the form ax + by + c = dx + ey + f (1.4) By using the operations like those above we can reduce this to one of the forms Ax + By = C, y = A ′ x + C ′ (1.5) This should be recognizable from high school mathematics. Because the graph of Equation (1.5) is a line we refer to the equation as a linear equation and we extend this definition to any equation in one or more variables in which no power of any variable exceeds 1, that is, there are no x 2 , x 3 , . . . terms for any variable x and there are no products of distinct variables either (e.g. no terms like xy or x 2 y 3 ) As we previously defined ( Definition (1.1)) the most general form of a linear equation is b 1 x 1 + b 2 x 2 + · · · + b n x n + c = d 1 x 1 + d 2 x 2 + · · · + d n x n + e (1.6) Definition 1.2. By a solution to an equation b 1 x 1 + b 2 x 2 + · · · + b n x n + c = d 1 x 1 + d 2 x 2 + · · · + d n x n + e we mean an assignment of actual numbers to the variables x 1 = γ 1 , x 2 = γ 2 , . . . , x n = γ n which makes the two sides equal, that is, makes the proposition a true statement. The solution set of a linear equation is the collection consisting of all the solutions to the equation. 10 Chapter 1. Linear Equations We will make extensive use of equations which have the same solution set. Because of how frequently we will refer to this relationship we give it a name: Definition 1.3. Two linear equations are said to be equivalent if they have exactly the same solution sets. The general linear equation is always equivalent to an equation of the form: a 1 x 1 + a 2 x 2 + · · · + a n x n = b (1.7) This gives rise to the following definition: Definition 1.4. A linear equation of the form a 1 x 1 + a 2 x 2 + · · · + a n x n = b is said to be in standard form Example 1.1.2. The equation 3 x − 2 y + z − 5 = 2 x − 3 y − 2 z − 4 is equivalent to the following equation which in standard form x + y + 3 z = 1 Definition 1.5. A linear equation like that in Equation (1.7) in standard form is said to be homogeneous if b = 0 and inhomogeneous otherwise. Example 1.1.3. The equation 3 x 1 + 4 x 2 − x 3 + 7 x 4 = 0 is homogeneous The equation x 1 − 3 x 2 + 5 x 3 − 7 x 4 = 12 is inhomogeneous Definition 1.6. When a linear equation is in standard form as in Equation (1.7) with the variables in their natural order the term with the first nonzero coefficient is called the leading term and the variable of this term is called the leading variable of the equation . Other variables are non-leading variables. 1.1. Linear Equations and Their Solution 11 Example 1.1.4. In the inhomogeneous linear equation x + y + 3 z = 1 x is the leading term. Moreover, the triples ( x, y, z ) = (1 , 6 , − 2) , (0 , 1 , 0) , (3 , − 5 , 1) are solutions , while (1,1,1) is not a solution. Example 1.1.5. The equation 0 x 1 +2 x 2 − x 3 +2 x 4 = 5 is an inhomogeneous linear equation . The leading term is 2 x 2 Definition 1.7. In an equation such as those appearing in Example (1.1.4) or Ex- ample (1.1.5) the non-leading variables are referred to as free variables . We use this term because these variables can be chosen in any way we wish and once we have done so we can solve for the leading variable and get a unique solution . Typ- ically, we assign parameters to the free variables and then get an expression for each variable in terms of these parameters. This expression is the general solution Example 1.1.6. The leading variable of the ( standard form ) linear equation x + y + 3 z = 1 is x. We may set y = s, z = t and then x = 1 − s − 3 t. Consequently, the general solution is ( x, y, z ) = (1 − s − 3 t, s, t ) where s and t are free to take on any value. Example 1.1.7. In the ( standard form ) linear equation 0 x 1 + 2 x 2 − x 3 + 2 x 4 = 5 the leading variable is x 2 We can set x 1 = r, x 3 = s, x 4 = t and then x 2 = 5+ s − 2 t 2 and we have the general solution ( x 1 , x 2 , x 3 , x 4 ) = ( r, 5+ s − 2 t 2 , s, t ) where r, s and t can take on any value. 12 Chapter 1. Linear Equations Definition 1.8. A system of linear equations or simply a linear system is just a collection of equations like the standard one : a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = b 2 a m 1 x 1 + a m 2 x 2 + · · · + a mn x n = b m This system consists of m equations in n variables. The numbers b i are called the constants of the system and the a ij are called the coefficients of the system. If all the b i = 0 then the system is called homogeneous . Otherwise, if at least one constant is non-zero, it is called inhomogeneous Example 1.1.8. The system 2 x 1 − x 2 − 2 x 3 = 3 x 1 + 2 x 2 + 4 x 3 = 7 is an inhomogeneous system of two equations in three variables. Example 1.1.9. The system x 1 − 3 x 2 + 5 x 3 − 7 x 4 = 0 3 x 1 + 5 x 2 + 7 x 3 + 9 x 4 = 0 5 x 1 − 7 x 2 + 9 x 3 = 0 is a homogeneous linear system of three equations in four variables. Definition 1.9. By a solution to a system of linear equations a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = b 2 a m 1 x 1 + a m 2 x 2 + · · · + a mn x n = b m we mean a sequence of numbers γ 1 , γ 2 , . . . , γ n such that when they are substituted for x 1 , x 2 , . . . , x n , all the equations are satisfied, that is, we get equalities. The set (collection) of all possible solutions is called the solution set . A generic or representative element of the solution set is called a general solution 1.1. Linear Equations and Their Solution 13 Example 1.1.10. Consider the linear system x 1 + 2 x 2 + x 3 = 1 3 x 1 + 5 x 2 + 2 x 3 = − 1 Then ( x 1 , x 2 , x 3 ) = ( − 7 , 4 , 0) is a solution, as is ( x 1 , x 2 , x 3 ) = ( − 4 , 1 , 3) However, ( x 1 , x 2 , x 3 ) = (0 , − 2 , 7) is not a solution. Definition 1.10. If a linear system has a solution then it is consistent otherwise it is inconsistent Example 1.1.11. The linear system of Example (1.1.10) is clearly consistent The system 2 x + y = 2 2 x + y = 1 is obviously inconsistent It is less obvious that the following linear system is inconsistent . We shall shortly develop methods for determining whether a linear system is consistent or not. x 1 + 2 x 2 + x 3 = 4 2 x 1 + 3 x 2 + x 3 = 6 x 1 + 3 x 2 + 2 x 3 = 5 To see that the above linear system is inconsistent , note that x 1 + 3 x 2 + 2 x 3 = 3( x 1 + 2 x 2 + x 3 ) − (2 x 1 + 3 x 2 + x 3 ) If the first two equations are satisfied then we must have x 1 + 3 x 2 + 2 x 3 = 3 × 4 − 6 = 6 6 = 5 14 Chapter 1. Linear Equations Definition 1.11. A homogeneous system of linear equations a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = 0 a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = 0 a m 1 x 1 + a m 2 x 2 + · · · + a mn x n = 0 always has a solution . Namely, set all the variables equal to zero, that is, set x 1 = 0 , x 2 = 0 , . . . , x n = 0 The all zero solution is called the trivial solution A solution to a homogeneous system in which some variable is non zero is said to be nontrivial Example 1.1.12. The following homogeneous linear system x 1 + x 2 + x 3 − 3 x 4 = 0 x 1 + x 2 − 3 x 3 + x 4 = 0 x 1 − 3 x 2 + x 3 + x 4 = 0 − 3 x 1 + x 2 + x 3 + x 4 = 0 has the nontrivial solution ( x 1 , x 2 , x 3 , x 4 ) = (1 , 1 , 1 , 1) Definition 1.12. Two systems of linear equations are said to be equivalent if their solution sets are identical. Solving a linear system The way in which we solve a linear system such as a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = b 2 a m 1 x 1 + a m 2 x 2 + · · · + a mn x n = b m is to tranform it into an equivalent linear system whose solution set can be (more) easily determined. The next definition encapsulates just such a type of linear system 1.1. Linear Equations and Their Solution 15 Definition 1.13. Consider the following system of linear equations a 11 x 1 + a 12 x 2 + · · · + a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + · · · + a 2 n x n = b 2 a m 1 x 1 + a m 2 x 2 + · · · + a mn x n = b m Let l i be the index of the leading variable of the i th equation. The system is said to have echelon form if l 1 < l 2 < · · · < l m , that is, the indices of the leading variables are increasing. When a linear system is in echelon form, the sequence consisting of those variables which are the leading variable in some equation of the system are the leading vari- ables of the linear system. The complementary variables (those which are not a lead- ing variable of any of the equations of the linear system) are referred to as the free variables of the system. Example 1.1.13. The linear system 2 x 1 + 3 x 2 + x 3 = 13 − 2 x 2 + 3 x 3 = 5 2 x 3 = 6 is in echelon form Example 1.1.14. The linear system x 1 + 2 x 2 − 2 x 3 + x 4 − x 5 = − 3 x 2 − 2 x 3 + 4 x 4 − 3 x 5 = 8 2 x 4 + x 5 = 6 is in echelon form Back Substitution When a linear system is in echelon form it can be solved by the method of back substitution . In this method each free variable of the linear system is set equal to a parameter (that is, it is free to take on any value). Then, beginning with the last equation the leading variable of this equation is solved in terms of the parameters and a constant (possibly zero). This expression is then substituted into the preceding equations and the leading variable in this equation is solved in terms of the parameters. We continue 16 Chapter 1. Linear Equations in this way until every leading variable is expressed in terms of the parameters assigned to the free variables (and a constant, possibly zero). The next theorem establishes the theoretic basis for this method. Theorem 1.1.1. Let a 11 x 1 + a 12 x 2 + . . . + a 1 n x n = b 1 a 22 x 2 + . . . + a 2 n x n = b 2 a nn x n = b n be a linear system of n equations in n variables in echelon form Assume that a 11 , a 22 , . . . , a nn 6 = 0 Then the system has a unique solution Proof . If n = 1 then the system simply has the form ax = b with a 6 = 0 and the only solution is x = b a If n = 2 then the system looks like a 11 x + a 12 y = b 1 a 22 y = b 2 From the second equation we get unique solution for y, namely, y = b 2 a 22 When this is substituted into the first equation we then get a unique solution for x. The general proof is obtained by applying the principle of mathematical induction to the number of variables (which is equal to the number of equations). The principle of mathematical induction says that some proposition (assertion) about natural numbers (the counting numbers 1 , 2 , 3 , . . . ) is true for every natural number if we can show it is true for 1 and anytime it is true for a natural number n then we can show that it holds for the next number, n + 1 We have shown that the theorem is true for a single linear equation in one variable and a linear system of two equations in two variables. Instead of doing the actual proof showing that it holds for n + 1 whenever it holds for n , for purposes of illustration we demonstrate how we can go from three variables (and equations) to four variables and equations. Assume that we know if we have a linear system of three equations in three variables in echelon form then there is a unique solution . Now suppose we are given a system of four equations in four variables which is in echelon form: a 11 x 1 + a 12 x 2 + a 13 x 3 + a 14 x 4 = b 1 a 22 x 2 + a 23 x 3 + a 24 x 4 = b 2 a 33 x 3 + a 34 x 4 = b 3 a 44 x 4 = b 4