Operations on matrices

Addition, scalar multiplication and difference

Definition. Let's recall the basic definitions. For positive integers \(m,n\), an $mn matrix* is a table of scalars with \(m\) rows and \(n\) columns. We also say that the dimensions of \(A\) are \(m\times n\). Matrices are usually denoted by uppercase letters.

Let \(A\) be a matrix. Then its entry in the \(i\)-th row and the \(j\)-th column is denoted by \(a_{ij}\) or \(A_{ij}\). The row vector that is the \(i\)-th row of \(A\) is denoted by \(\mathbf a_i\) or \(A_i\). The column vector that is the \(j\)-th column of \(A\) is denoted by \(\mathbf a^j\) or \(A^j\).

Let \(A\) and \(B\) be two matrices. Then we have \(A=B\), if they have the same number of rows and columns, let's say \(m\) and \(n\), and for all \(1\le i\le m\) and \(1\le j\le n\), we have \(a_{ij}=b_{ij}\).

The \(m\times n\) zero matrix \(O\) is the matrix with all its entires zeroes: for all \(1\le i\le m\) and \(1\le j\le n\), we have \(O_{ij}=0\).

Let \(A\) and \(B\) be matrices. Then their sum \(A+B\) and difference \(A-B\) are defined only when their dimensions are the same, let's say \(m\times n\). Just as for vectors, addition and difference are calculated componentwise: \[ (A+B)_{ij}=a_{ij}+b_{ij},\,(A-B)_{ij}=a_{ij}-b_{ij}. \] Let \(c\) be a scalar. Then just as for vectors, the scalar multiple \(cA\) is calculated componentwise: \[ (cA)_{ij}=ca_{ij}. \]

Example. We have \[ 2\begin{pmatrix}2 & 3 & -1 \\ 0 & 1 & 2 \end{pmatrix}-\begin{pmatrix}-2 & 0 & 1 \\ 0 & 3 & -2\end{pmatrix}=\begin{pmatrix}6 & 6 & -3 \\ 0 & -1 & 6\end{pmatrix}. \]

Matrix multiplication

Special case 1. To remember matrix multiplication, I find it very useful to remember that taking a dot product of two \(n\)-vectors \(u\) and \(v\) is a special case of a matrix product, provided that \(u\) is written as a row vector, that is as a \(1\times n\)-matrix, and \(v\) is written as a column vector, that is as an \(n\times 1\)-matrix, and you think of the resulting scalar as a \(1\times1\)-matrix: \[ \begin{pmatrix}u_1 & \dotsb & u_n\end{pmatrix}\cdot\begin{pmatrix}v_1 \\ \vdots \\ v_n\end{pmatrix}=\begin{pmatrix}u_1v_1+\dotsb+u_nv_n\end{pmatrix} \]

Special case 2. Using matrix multiplication, we can write systems of linear equations as vector equations. Consider the SLE \[\begin{align*} a_{11}x_1+\dotsb+a_{1n}x_n=&b_1\\ \vdots&\\ a_{m1}x_1+\dotsb+a_{mn}x_n=&b_m. \end{align*}\]

In chapter 2, we have been concerned with the corresponding augmented matrix. Now let's separate the coefficient matrix \(A\), and the constant vector \(\mathbf b\). That is, let \(A\) be the \(m\times n\) matrix with \(A_{ij}=a_{ij}\), and let \(\mathbf b\) be the column \(m\)-vector with the entries \(b_i\). Then the augmented matrix can be gotten by affixing \(\mathbf b\) to the right of \(A\): \[ (A|\mathbf b)=\left(\begin{array}{ccc|c} a_{11} & \dotsb & a_{1n} & b_1 \\ \vdots & \ddots & \vdots & \vdots \\ a_{m1} & \dotsb & a_{mn} & b_m \end{array}\right). \] Let \(\mathbf x\) be the variable column \(n\)-vector, that is the one with entries the \(x_i\). Then the matrix product \(A\mathbf x\) is the column \(m\)-vector with components the left sides of the SLE: \[ A\mathbf x=\begin{pmatrix} a_{11}x_1+\dotsb+a_{1n}x_n\\ \vdots\\ a_{m1}x_1+\dotsb+a_{mn}x_n \end{pmatrix}. \] That is, the SLE can also be written as the vector equation \[ A\mathbf x=\mathbf b. \]

General formula. Let \(A\) and \(B\) be matrices. Then the matrix product \(AB\) can be formed only when the number of rows of \(A\) is the same as the number of columns of \(B\). Suppose that this is the case, let's say \(A\) is \(m\times n\), and \(B\) is \(n\times\ell\). Then the product \(AB\) is the \(m\times\ell\)-matrix where for \(1\le i\le m\) and \(1\le j\le\ell\), the \(ij\)-entry is the dot product of the \(i\)-th row vector of \(A\), and the \(j\)-th column vector of \(B\): \[ (AB)_{ij}=\mathbf a_i\mathbf b^j=a_{i1}b_{1j}+\dotsb+a_{in}b_{nj}. \]

Warning. Matrix multiplication is not commutative, that is we don't necessarily have \(AB=BA\). For example, if \(m\ne\ell\), then the product \(BA\) can't even be formed.

Example. Let \(A=\left(\begin{smallmatrix}a & b \\ c & d \end{smallmatrix}\right)\) and \(e_{12}=\left(\begin{smallmatrix}0 & 1 \\ 0 & 0 \end{smallmatrix}\right)\). Then we have \[ Ae_{12}=\begin{pmatrix}0 & a \\ 0 & c \end{pmatrix}\text{, and }e_{12}A=\begin{pmatrix}c & d \\ 0 & 0 \end{pmatrix}. \] That is, we can only have \(Ae_{12}=e_{12}A\) if the components match up: \(0=c,\,a=d\). For example, for \(A=\left(\begin{smallmatrix}1 & 0 \\ 0 & -1 \end{smallmatrix}\right)\), we have \[ Ae_{12}=\begin{pmatrix}0 & 1 \\ 0 & 0 \end{pmatrix}\text{, and }e_{12}A=\begin{pmatrix}0 & -1 \\ 0 & 0 \end{pmatrix}. \] That is, even though both products are defined, we have \(Ae_{12}\ne e_{12}A\).

On the other hand, for \(A=\left(\begin{smallmatrix}1 & 0 \\ 0 & 1 \end{smallmatrix}\right)\), we have \[ Ae_{12}=e_{12}=e_{12}A. \]

Definition. Let \(A\) be an \(m\times n\) matrix. Then it is a square matrix, if \(m=n\). Suppose that this is the case. Then the main diagonal is the collection of elements \(a_{ii}\). \(A\) is a diagonal matrix, if it can only have nonzero entries in the main diagonal: if \(i\ne j\), then \(a_{ij}=0\). The \(n\times n\) identity matrix \(I(=I_n)\) is the \(n\times n\) matrix, which is a diagonal matrix, and all the entries in the main diagonal are 1's: \[ I_{ij}=\begin{cases} 1 & i=j\\ 0 & i\ne j. \end{cases} \] \(A\) is a scalar matrix, if it is a diagonal matrix, with all entries in the main diagonal being the same. That is, there exists a scalar \(c\) such that \(A=cI_n\).

Theorem. 1. Let \(A\) be any \(m\times n\) matrix, and \(B\) any \(n\times\ell\) matrix. Then we have \[ AI_n=A\text{, and }I_nA=A. \]

  1. Let \(c\) be any scalar. Then we have \[ A(cI_n)=cA\text{, and }(cI_n)A=cA. \]

  2. Let \(A\) be any \(n\times n\) matrix. If for all \(n\times n\)-matrices \(B\) we have \(AB=BA\), then \(A\) is a scalar matrix.

Definition. Let \(A\) be an \(m\times n\) matrix. Then its transpose \(A^T\) is the \(n\times m\)-matrix such that \((A^T)_{ij}=a_{ij}\).

Example 1. We have \[ \begin{pmatrix} 5 & 1 & -3 \\ -1 & 0 & 2 \end{pmatrix}^T\begin{pmatrix} 0 & -2 \\ 1 & 3 \end{pmatrix}=\begin{pmatrix}5 & -1 \\ 1 & 0 \\ -3 & 2 \end{pmatrix}\begin{pmatrix}0 & -2 \\ 1 & 3 \end{pmatrix}=\begin{pmatrix}-1 & 2 \\ 0 & -2 \\ 2 & 12 \end{pmatrix}. \]

Definition. Let \(A\) be an \(m\times m\) matrix. Then its powers are defined just as for numbers: \[ A^0=I_m,\,A^1=A,\,A^2=AA,\,A^3=AAA,\dotsc \]

Example. Let \(A=\left(\begin{smallmatrix}1&1&0\\1&0&0\\0&0&-1\end{smallmatrix}\right)\). Then we have \[ A^2=\begin{pmatrix}1&2&0\\0&1&0\\0&0&1\end{pmatrix},\,A^3=\begin{pmatrix}1&3&0\\0&1&0\\0&0&-1\end{pmatrix}. \] From this, we might guess that \(A^n=\left(\begin{smallmatrix}1&n&0\\0&1&0\\0&0&(-1)^n\end{smallmatrix}\right)\). To prove this, we use induction. The statement is already proven for \(n=1,2,3\), so what's left to do is the induction step. Assuming that the statement holds for some \(n\), we prove it for \(n+1\): \[ \begin{pmatrix}1&n&0\\0&1&0\\0&0&(-1)^n\end{pmatrix}\begin{pmatrix}1&1&0\\0&1&0\\0&0&-1\end{pmatrix}=\begin{pmatrix}1&n+1&0\\0&1&0\\0&0&(-1)^{n+1}\end{pmatrix}. \]

Example. Let \(A=\left(\begin{smallmatrix}2&2\\2&2\end{smallmatrix}\right)\). Then we have \[ A^2=\begin{pmatrix}8&8\\8&8\end{pmatrix}=\begin{pmatrix}2^3&2^3\\2^3&2^3\end{pmatrix},\,A^3=\begin{pmatrix}32&32\\32&32\end{pmatrix}=\begin{pmatrix}2^5&2^5\\2^5&2^5\end{pmatrix}. \] From this, we might guess that \(A^n=\left(\begin{smallmatrix}2^{2n-1}&2^{2n-1}\\2^{2n-1}&2^{2n-1}\end{smallmatrix}\right)\). The statement is already proven for \(n=1,2,3\), and the induction step is proven as follows. \[ \begin{pmatrix}2^{2n-1}&2^{2n-1}\\2^{2n-1}&2^{2n-1}\end{pmatrix}\begin{pmatrix}2&2\\2&2\end{pmatrix}=\begin{pmatrix}2^{2n+1}&2^{2n+1}\\2^{2n+1}&2^{2n+1}\end{pmatrix}. \]