Matrix equations and inverses

A system of linear equations \begin{align*} a_{11}x_1+\dotsb+a_{1n}x_n=&b_1\\ \vdots&\\ a_{m1}x_1+\dotsb+a_{mn}x_n=&b_m. \end{align*}

gives a coefficient matrix and a constant vector: $A\mathbf x=\begin{pmatrix} a_{11}x_1+\dotsb+a_{1n}x_n\\ \vdots\\ a_{m1}x_1+\dotsb+a_{mn}x_n \end{pmatrix},\,\mathbf b=\begin{pmatrix} b_1\\\vdots\\b_m \end{pmatrix}.$ That is, the SLE can also be written as the vector equation $A\mathbf x=\mathbf b,$ where $$\mathbf x=(x_1,\dotsc,x_n)^T$$. In this section, we will start by considering how solving SLE's with the same coefficient matrix $$A$$ can be made easier, which will lead us to inverse matrices.

Example. Consider the SLE \begin{align*} 3x_1-5x_2&=0\\ -2x_1+3x_2&=1. \end{align*} To solve it, we can reduce the corresponding augmented matrix: $\left(\begin{array}{cc|c} 3 & -5 & 0 \\ -2 & 3 & 1 \end{array}\right)\sim \left(\begin{array}{cc|c} 0 & 1 & -3 \\ 1 & 0 & -5 \end{array}\right).$ This show that the solution is $$x_1=-5,\,x_2=-3$$, that is $A\mathbf x=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix}-5 \\ -3\end{pmatrix}=\begin{pmatrix}0 \\ 1 \end{pmatrix}.$ Let's now solve the SLE \begin{align*} 3x_1-5x_2&=0\\ -2x_1+3x_2&=2. \end{align*} In matrix form, it is $A\mathbf x=\begin{pmatrix}0 \\ 2\end{pmatrix}.$ What you can notice is that this constant vector is twice the original one: $$(0,2)^T=2(0,1)^T$$. Therefore, multiplying the solution to $$A\mathbf x=(0,1)^T$$ gives us the answer: $A2\begin{pmatrix}-5\\-3\end{pmatrix}=2A\begin{pmatrix}-5\\-3\end{pmatrix}=2\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}0\\2\end{pmatrix}.$ That is, the solution to this SLE is $$x_1=-10,\,x_2=-6$$. Let's now solve the SLE \begin{align*} 3x_1-5x_2&=1\\ -2x_1+3x_2&=0. \end{align*} Since $$(1,0)^T$$ is not parallel to $$(0,1)^T$$, we need to calculate a solution. But in the reduction, we can use the exact same reduction steps. We get $\left(\begin{array}{cc|c} 3 & -5 & 1 \\ -2 & 3 & 0 \end{array}\right)\sim \left(\begin{array}{cc|c} 0 & 1 & -2 \\ 1 & 0 & -3 \end{array}\right).$ That is, the solution is $$x_1=-3,\,x_2=-2$$: $A\mathbf x=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix} -3 \\ -2 \end{pmatrix}=\begin{pmatrix}1\\0\end{pmatrix}$ Let's now consider the SLE \begin{align*} 3x_1-5x_2&=1\\ -2x_1+3x_2&=-1. \end{align*}

In matrix form, it is $A\mathbf x=\begin{pmatrix}1\\-1\end{pmatrix}.$ Note that we have $$(1,-1)^T=(1,0)^T-(0,1)^T$$. Therefore, we can use our solutions to $$A\mathbf x=(1,0)^T$$ and $$A\mathbf x=(0,1)^T$$ to get a solution to this one: $A\left(\begin{pmatrix}-3\\-2\end{pmatrix}-\begin{pmatrix}-5\\-3\end{pmatrix}\right)=A\begin{pmatrix}-3\\-2\end{pmatrix}-A\begin{pmatrix}-5\\-3\end{pmatrix}=\begin{pmatrix}1\\0\end{pmatrix}-\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}1\\-1\end{pmatrix}.$ Note that similarly we can now solve any equation with the same $$A$$ like this. If the constant vector is $$\mathbf b=(b_1,b_2)^T$$, then the solution is $$b_1(-3,-2)^T+b_2(-5,-3)^T$$.

Let's form the matrix $$A'$$ with columns the two solutions $$(-3,-2)^T$$ and $$(-5,-3)^T$$. Then by construction, we get $AA'=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix}-3 & -5 \\ -2 & -3\end{pmatrix}=\begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix}.$ One can check that we also have $A'A=\begin{pmatrix}-3 & -5 \\ -2 & -3 \end{pmatrix}\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}=\begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix}.$ Definition. Let $$A$$ be an $$m\times m$$ matrix. Then its inverse $$A^{-1}$$ is the matrix with $$AA^{-1}=A^{-1}A=I_m$$. If $$A$$ has an inverse, then we say that $$A$$ is invertible, or nonsingular. If $$A$$ does not have an inverse, then we say it is singular.

Proposition. Let $$A$$ be an $$m\times m$$ invertible matrix. Then its inverse is unique: if $$B$$ is another matrix such that $$AB=BA=I$$, then $$B=A^{-1}$$.

Proof. We have $A^{-1}=IA^{-1}=(BA)A^{-1}=B(AA^{-1})=BI=B.$

Method. Let $$A$$ be an $$m\times m$$ matrix. To see if it has an inverse, reduce the $$m\times(2m)$$-matrix $$(A|I_m)$$. If the RREF matrix you get is $$(I_m|B)$$, then $$B=A^{-1}$$.

Example. Let $$A=\left(\begin{smallmatrix}1 & 0 & 2 \\ -1 & 1 & 0 \\ 2 & 0 & 1\end{smallmatrix}\right)$$. Let's see if it has an inverse. We have $\left(\begin{array}{ccc|ccc}1 & 0 & 2 & 1 & 0 & 0 \\ -1 & 1 & 0 & 0 & 1 & 0 \\ 2 & 0 & 1 & 0 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 0 & 2 & 1 & 0 & 0 \\ 0 & 1 & 2 & 1 & 1 & 0 \\ 0 & 0 & -3 & -2 & 0 & 1\end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 0 & 0 & -1/3 & 0 & 2/3 \\ 0 & 1 & 0 & -1/3 & 1 & 2/3 \\ 0 & 0 & 1 & 2/3 & 0 & -1/3\end{array}\right).$ This shows that $$A$$ is invertible, and its inverse is $$A^{-1}=\left(\begin{smallmatrix}-1/3 & 0 & 2/3 \\ -1/3 & 1 & 2/3 \\ 2/3 & 0 & -1/3\end{smallmatrix}\right)$$.

Example. Let $$A=\left(\begin{smallmatrix}2 & 1 & -1 \\ 0 & -2 & 1 \\ 2 & 5 & -3\end{smallmatrix}\right)$$. Let's see if it has an inverse. We have $\left(\begin{array}{ccc|ccc}2 & 1 & -1 & 1 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 2 & 5 & -3 & 0 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 1/2 & -1/2 & 1/2 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 0 & 4 & -2 & -1 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 1/2 & -1/2 & 1/2 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & -1 & 2 & 1\end{array}\right).$ We can see that in the last row, the leading 1 is not in the coefficient matrix. Therefore, it impossible to put this in the form $$(I_m|B)$$. This shows that $$A$$ is not invertible.