# Operations on matrices

## Addition, scalar multiplication and difference

Definition. Let's recall the basic definitions. For positive integers $$m,n$$, an \$mn matrix* is a table of scalars with $$m$$ rows and $$n$$ columns. We also say that the dimensions of $$A$$ are $$m\times n$$. Matrices are usually denoted by uppercase letters.

Let $$A$$ be a matrix. Then its entry in the $$i$$-th row and the $$j$$-th column is denoted by $$a_{ij}$$ or $$A_{ij}$$. The row vector that is the $$i$$-th row of $$A$$ is denoted by $$\mathbf a_i$$ or $$A_i$$. The column vector that is the $$j$$-th column of $$A$$ is denoted by $$\mathbf a^j$$ or $$A^j$$.

Let $$A$$ and $$B$$ be two matrices. Then we have $$A=B$$, if they have the same number of rows and columns, let's say $$m$$ and $$n$$, and for all $$1\le i\le m$$ and $$1\le j\le n$$, we have $$a_{ij}=b_{ij}$$.

The $$m\times n$$ zero matrix $$O$$ is the matrix with all its entires zeroes: for all $$1\le i\le m$$ and $$1\le j\le n$$, we have $$O_{ij}=0$$.

Let $$A$$ and $$B$$ be matrices. Then their sum $$A+B$$ and difference $$A-B$$ are defined only when their dimensions are the same, let's say $$m\times n$$. Just as for vectors, addition and difference are calculated componentwise: $(A+B)_{ij}=a_{ij}+b_{ij},\,(A-B)_{ij}=a_{ij}-b_{ij}.$ Let $$c$$ be a scalar. Then just as for vectors, the scalar multiple $$cA$$ is calculated componentwise: $(cA)_{ij}=ca_{ij}.$

Example. We have $2\begin{pmatrix}2 & 3 & -1 \\ 0 & 1 & 2 \end{pmatrix}-\begin{pmatrix}-2 & 0 & 1 \\ 0 & 3 & -2\end{pmatrix}=\begin{pmatrix}6 & 6 & -3 \\ 0 & -1 & 6\end{pmatrix}.$

## Matrix multiplication

Special case 1. To remember matrix multiplication, I find it very useful to remember that taking a dot product of two $$n$$-vectors $$u$$ and $$v$$ is a special case of a matrix product, provided that $$u$$ is written as a row vector, that is as a $$1\times n$$-matrix, and $$v$$ is written as a column vector, that is as an $$n\times 1$$-matrix, and you think of the resulting scalar as a $$1\times1$$-matrix: $\begin{pmatrix}u_1 & \dotsb & u_n\end{pmatrix}\cdot\begin{pmatrix}v_1 \\ \vdots \\ v_n\end{pmatrix}=\begin{pmatrix}u_1v_1+\dotsb+u_nv_n\end{pmatrix}$

Special case 2. Using matrix multiplication, we can write systems of linear equations as vector equations. Consider the SLE \begin{align*} a_{11}x_1+\dotsb+a_{1n}x_n=&b_1\\ \vdots&\\ a_{m1}x_1+\dotsb+a_{mn}x_n=&b_m. \end{align*}

In chapter 2, we have been concerned with the corresponding augmented matrix. Now let's separate the coefficient matrix $$A$$, and the constant vector $$\mathbf b$$. That is, let $$A$$ be the $$m\times n$$ matrix with $$A_{ij}=a_{ij}$$, and let $$\mathbf b$$ be the column $$m$$-vector with the entries $$b_i$$. Then the augmented matrix can be gotten by affixing $$\mathbf b$$ to the right of $$A$$: $(A|\mathbf b)=\left(\begin{array}{ccc|c} a_{11} & \dotsb & a_{1n} & b_1 \\ \vdots & \ddots & \vdots & \vdots \\ a_{m1} & \dotsb & a_{mn} & b_m \end{array}\right).$ Let $$\mathbf x$$ be the variable column $$n$$-vector, that is the one with entries the $$x_i$$. Then the matrix product $$A\mathbf x$$ is the column $$m$$-vector with components the left sides of the SLE: $A\mathbf x=\begin{pmatrix} a_{11}x_1+\dotsb+a_{1n}x_n\\ \vdots\\ a_{m1}x_1+\dotsb+a_{mn}x_n \end{pmatrix}.$ That is, the SLE can also be written as the vector equation $A\mathbf x=\mathbf b.$

General formula. Let $$A$$ and $$B$$ be matrices. Then the matrix product $$AB$$ can be formed only when the number of rows of $$A$$ is the same as the number of columns of $$B$$. Suppose that this is the case, let's say $$A$$ is $$m\times n$$, and $$B$$ is $$n\times\ell$$. Then the product $$AB$$ is the $$m\times\ell$$-matrix where for $$1\le i\le m$$ and $$1\le j\le\ell$$, the $$ij$$-entry is the dot product of the $$i$$-th row vector of $$A$$, and the $$j$$-th column vector of $$B$$: $(AB)_{ij}=\mathbf a_i\mathbf b^j=a_{i1}b_{1j}+\dotsb+a_{in}b_{nj}.$

Warning. Matrix multiplication is not commutative, that is we don't necessarily have $$AB=BA$$. For example, if $$m\ne\ell$$, then the product $$BA$$ can't even be formed.

Example. Let $$A=\left(\begin{smallmatrix}a & b \\ c & d \end{smallmatrix}\right)$$ and $$e_{12}=\left(\begin{smallmatrix}0 & 1 \\ 0 & 0 \end{smallmatrix}\right)$$. Then we have $Ae_{12}=\begin{pmatrix}0 & a \\ 0 & c \end{pmatrix}\text{, and }e_{12}A=\begin{pmatrix}c & d \\ 0 & 0 \end{pmatrix}.$ That is, we can only have $$Ae_{12}=e_{12}A$$ if the components match up: $$0=c,\,a=d$$. For example, for $$A=\left(\begin{smallmatrix}1 & 0 \\ 0 & -1 \end{smallmatrix}\right)$$, we have $Ae_{12}=\begin{pmatrix}0 & 1 \\ 0 & 0 \end{pmatrix}\text{, and }e_{12}A=\begin{pmatrix}0 & -1 \\ 0 & 0 \end{pmatrix}.$ That is, even though both products are defined, we have $$Ae_{12}\ne e_{12}A$$.

On the other hand, for $$A=\left(\begin{smallmatrix}1 & 0 \\ 0 & 1 \end{smallmatrix}\right)$$, we have $Ae_{12}=e_{12}=e_{12}A.$

Definition. Let $$A$$ be an $$m\times n$$ matrix. Then it is a square matrix, if $$m=n$$. Suppose that this is the case. Then the main diagonal is the collection of elements $$a_{ii}$$. $$A$$ is a diagonal matrix, if it can only have nonzero entries in the main diagonal: if $$i\ne j$$, then $$a_{ij}=0$$. The $$n\times n$$ identity matrix $$I(=I_n)$$ is the $$n\times n$$ matrix, which is a diagonal matrix, and all the entries in the main diagonal are 1's: $I_{ij}=\begin{cases} 1 & i=j\\ 0 & i\ne j. \end{cases}$ $$A$$ is a scalar matrix, if it is a diagonal matrix, with all entries in the main diagonal being the same. That is, there exists a scalar $$c$$ such that $$A=cI_n$$.

Theorem. 1. Let $$A$$ be any $$m\times n$$ matrix, and $$B$$ any $$n\times\ell$$ matrix. Then we have $AI_n=A\text{, and }I_nA=A.$

1. Let $$c$$ be any scalar. Then we have $A(cI_n)=cA\text{, and }(cI_n)A=cA.$

2. Let $$A$$ be any $$n\times n$$ matrix. If for all $$n\times n$$-matrices $$B$$ we have $$AB=BA$$, then $$A$$ is a scalar matrix.

Definition. Let $$A$$ be an $$m\times n$$ matrix. Then its transpose $$A^T$$ is the $$n\times m$$-matrix such that $$(A^T)_{ij}=a_{ij}$$.

Example 1. We have $\begin{pmatrix} 5 & 1 & -3 \\ -1 & 0 & 2 \end{pmatrix}^T\begin{pmatrix} 0 & -2 \\ 1 & 3 \end{pmatrix}=\begin{pmatrix}5 & -1 \\ 1 & 0 \\ -3 & 2 \end{pmatrix}\begin{pmatrix}0 & -2 \\ 1 & 3 \end{pmatrix}=\begin{pmatrix}-1 & 2 \\ 0 & -2 \\ 2 & 12 \end{pmatrix}.$

Definition. Let $$A$$ be an $$m\times m$$ matrix. Then its powers are defined just as for numbers: $A^0=I_m,\,A^1=A,\,A^2=AA,\,A^3=AAA,\dotsc$

Example. Let $$A=\left(\begin{smallmatrix}1&1&0\\1&0&0\\0&0&-1\end{smallmatrix}\right)$$. Then we have $A^2=\begin{pmatrix}1&2&0\\0&1&0\\0&0&1\end{pmatrix},\,A^3=\begin{pmatrix}1&3&0\\0&1&0\\0&0&-1\end{pmatrix}.$ From this, we might guess that $$A^n=\left(\begin{smallmatrix}1&n&0\\0&1&0\\0&0&(-1)^n\end{smallmatrix}\right)$$. To prove this, we use induction. The statement is already proven for $$n=1,2,3$$, so what's left to do is the induction step. Assuming that the statement holds for some $$n$$, we prove it for $$n+1$$: $\begin{pmatrix}1&n&0\\0&1&0\\0&0&(-1)^n\end{pmatrix}\begin{pmatrix}1&1&0\\0&1&0\\0&0&-1\end{pmatrix}=\begin{pmatrix}1&n+1&0\\0&1&0\\0&0&(-1)^{n+1}\end{pmatrix}.$

Example. Let $$A=\left(\begin{smallmatrix}2&2\\2&2\end{smallmatrix}\right)$$. Then we have $A^2=\begin{pmatrix}8&8\\8&8\end{pmatrix}=\begin{pmatrix}2^3&2^3\\2^3&2^3\end{pmatrix},\,A^3=\begin{pmatrix}32&32\\32&32\end{pmatrix}=\begin{pmatrix}2^5&2^5\\2^5&2^5\end{pmatrix}.$ From this, we might guess that $$A^n=\left(\begin{smallmatrix}2^{2n-1}&2^{2n-1}\\2^{2n-1}&2^{2n-1}\end{smallmatrix}\right)$$. The statement is already proven for $$n=1,2,3$$, and the induction step is proven as follows. $\begin{pmatrix}2^{2n-1}&2^{2n-1}\\2^{2n-1}&2^{2n-1}\end{pmatrix}\begin{pmatrix}2&2\\2&2\end{pmatrix}=\begin{pmatrix}2^{2n+1}&2^{2n+1}\\2^{2n+1}&2^{2n+1}\end{pmatrix}.$