Matrix equations and inverses

A system of linear equations \[\begin{align*} a_{11}x_1+\dotsb+a_{1n}x_n=&b_1\\ \vdots&\\ a_{m1}x_1+\dotsb+a_{mn}x_n=&b_m. \end{align*}\]

gives a coefficient matrix and a constant vector: \[ A\mathbf x=\begin{pmatrix} a_{11}x_1+\dotsb+a_{1n}x_n\\ \vdots\\ a_{m1}x_1+\dotsb+a_{mn}x_n \end{pmatrix},\,\mathbf b=\begin{pmatrix} b_1\\\vdots\\b_m \end{pmatrix}. \] That is, the SLE can also be written as the vector equation \[ A\mathbf x=\mathbf b, \] where \(\mathbf x=(x_1,\dotsc,x_n)^T\). In this section, we will start by considering how solving SLE's with the same coefficient matrix \(A\) can be made easier, which will lead us to inverse matrices.

Example. Consider the SLE \[\begin{align*} 3x_1-5x_2&=0\\ -2x_1+3x_2&=1. \end{align*}\] To solve it, we can reduce the corresponding augmented matrix: \[ \left(\begin{array}{cc|c} 3 & -5 & 0 \\ -2 & 3 & 1 \end{array}\right)\sim \left(\begin{array}{cc|c} 0 & 1 & -3 \\ 1 & 0 & -5 \end{array}\right). \] This show that the solution is \(x_1=-5,\,x_2=-3\), that is \[ A\mathbf x=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix}-5 \\ -3\end{pmatrix}=\begin{pmatrix}0 \\ 1 \end{pmatrix}. \] Let's now solve the SLE \[\begin{align*} 3x_1-5x_2&=0\\ -2x_1+3x_2&=2. \end{align*}\] In matrix form, it is \[ A\mathbf x=\begin{pmatrix}0 \\ 2\end{pmatrix}. \] What you can notice is that this constant vector is twice the original one: \((0,2)^T=2(0,1)^T\). Therefore, multiplying the solution to \(A\mathbf x=(0,1)^T\) gives us the answer: \[ A2\begin{pmatrix}-5\\-3\end{pmatrix}=2A\begin{pmatrix}-5\\-3\end{pmatrix}=2\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}0\\2\end{pmatrix}. \] That is, the solution to this SLE is \(x_1=-10,\,x_2=-6\). Let's now solve the SLE \[\begin{align*} 3x_1-5x_2&=1\\ -2x_1+3x_2&=0. \end{align*}\] Since \((1,0)^T\) is not parallel to \((0,1)^T\), we need to calculate a solution. But in the reduction, we can use the exact same reduction steps. We get \[ \left(\begin{array}{cc|c} 3 & -5 & 1 \\ -2 & 3 & 0 \end{array}\right)\sim \left(\begin{array}{cc|c} 0 & 1 & -2 \\ 1 & 0 & -3 \end{array}\right). \] That is, the solution is \(x_1=-3,\,x_2=-2\): \[ A\mathbf x=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix} -3 \\ -2 \end{pmatrix}=\begin{pmatrix}1\\0\end{pmatrix} \] Let's now consider the SLE \[\begin{align*} 3x_1-5x_2&=1\\ -2x_1+3x_2&=-1. \end{align*}\]

In matrix form, it is \[ A\mathbf x=\begin{pmatrix}1\\-1\end{pmatrix}. \] Note that we have \((1,-1)^T=(1,0)^T-(0,1)^T\). Therefore, we can use our solutions to \(A\mathbf x=(1,0)^T\) and \(A\mathbf x=(0,1)^T\) to get a solution to this one: \[ A\left(\begin{pmatrix}-3\\-2\end{pmatrix}-\begin{pmatrix}-5\\-3\end{pmatrix}\right)=A\begin{pmatrix}-3\\-2\end{pmatrix}-A\begin{pmatrix}-5\\-3\end{pmatrix}=\begin{pmatrix}1\\0\end{pmatrix}-\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}1\\-1\end{pmatrix}. \] Note that similarly we can now solve any equation with the same \(A\) like this. If the constant vector is \(\mathbf b=(b_1,b_2)^T\), then the solution is \(b_1(-3,-2)^T+b_2(-5,-3)^T\).

Let's form the matrix \(A'\) with columns the two solutions \((-3,-2)^T\) and \((-5,-3)^T\). Then by construction, we get \[ AA'=\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}\begin{pmatrix}-3 & -5 \\ -2 & -3\end{pmatrix}=\begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix}. \] One can check that we also have \[ A'A=\begin{pmatrix}-3 & -5 \\ -2 & -3 \end{pmatrix}\begin{pmatrix}3 & -5 \\ -2 & 3 \end{pmatrix}=\begin{pmatrix}1 & 0 \\ 0 & 1 \end{pmatrix}. \] Definition. Let \(A\) be an \(m\times m\) matrix. Then its inverse \(A^{-1}\) is the matrix with \(AA^{-1}=A^{-1}A=I_m\). If \(A\) has an inverse, then we say that \(A\) is invertible, or nonsingular. If \(A\) does not have an inverse, then we say it is singular.

Proposition. Let \(A\) be an \(m\times m\) invertible matrix. Then its inverse is unique: if \(B\) is another matrix such that \(AB=BA=I\), then \(B=A^{-1}\).

Proof. We have \[ A^{-1}=IA^{-1}=(BA)A^{-1}=B(AA^{-1})=BI=B. \]

Method. Let \(A\) be an \(m\times m\) matrix. To see if it has an inverse, reduce the \(m\times(2m)\)-matrix \((A|I_m)\). If the RREF matrix you get is \((I_m|B)\), then \(B=A^{-1}\).

Example. Let \(A=\left(\begin{smallmatrix}1 & 0 & 2 \\ -1 & 1 & 0 \\ 2 & 0 & 1\end{smallmatrix}\right)\). Let's see if it has an inverse. We have \[ \left(\begin{array}{ccc|ccc}1 & 0 & 2 & 1 & 0 & 0 \\ -1 & 1 & 0 & 0 & 1 & 0 \\ 2 & 0 & 1 & 0 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 0 & 2 & 1 & 0 & 0 \\ 0 & 1 & 2 & 1 & 1 & 0 \\ 0 & 0 & -3 & -2 & 0 & 1\end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 0 & 0 & -1/3 & 0 & 2/3 \\ 0 & 1 & 0 & -1/3 & 1 & 2/3 \\ 0 & 0 & 1 & 2/3 & 0 & -1/3\end{array}\right). \] This shows that \(A\) is invertible, and its inverse is \(A^{-1}=\left(\begin{smallmatrix}-1/3 & 0 & 2/3 \\ -1/3 & 1 & 2/3 \\ 2/3 & 0 & -1/3\end{smallmatrix}\right)\).

Example. Let \(A=\left(\begin{smallmatrix}2 & 1 & -1 \\ 0 & -2 & 1 \\ 2 & 5 & -3\end{smallmatrix}\right)\). Let's see if it has an inverse. We have \[ \left(\begin{array}{ccc|ccc}2 & 1 & -1 & 1 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 2 & 5 & -3 & 0 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 1/2 & -1/2 & 1/2 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 0 & 4 & -2 & -1 & 0 & 1 \end{array}\right)\sim \left(\begin{array}{ccc|ccc}1 & 1/2 & -1/2 & 1/2 & 0 & 0 \\ 0 & -2 & 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & -1 & 2 & 1\end{array}\right). \] We can see that in the last row, the leading 1 is not in the coefficient matrix. Therefore, it impossible to put this in the form \((I_m|B)\). This shows that \(A\) is not invertible.