In this post, we discuss some of the unary and binary matrix operations. The unary matrix operations include additive inversion, multiplicative inversion, matrix transpose, and matrix exponential. The binary matrix operations are like addition, subtraction, and multiplication of matrices.

Let us recall that a matrix is a rectangular table of objects arranged in rows and columns. When we say that \(M\) is a \(p \times q\) matrix or its dimension is \(p \times q\), we mean that the matrix \(M\) has \(p\) rows and \(q\) columns. Each object of a matrix is called an array of the matrix. We denote an object \(m\) of a matrix \(M\) by \(a_{ij}\) if it is located on the \(i\)th row and the \(j\)th column. In this case, we denote the matrix \(M\) as follows: $$M = (a_{ij}),$$ where we know that $$1 \leq i \leq p, \text{ and } 1 \leq j \leq q.$$

Note that the term “real matrix” refers to a matrix whose objects (arrays) are all real numbers. The set of all \(p \times q\) real matrices is denoted by \(M_{p \times q} (\mathbb{R})\), in some references.

## Binary matrix operations

First we define matrix addition and subtraction. Then, we proceed to define matrix multiplication.

### Matrix addition and subtraction

Let \(A = (a_{ij})\) and \(B = (a_{ij})\) be two \(p \times q\) real matrices. The addition of the matrices \(A\) and \(B\), denoted by \(A+B\), is defined as follows: $$A+B = (a_{ij} + b_{ij}).$$ The subtraction of the matrix \(B\) from the matrix \(A\), denoted by \(A-B\), is defined as follows: $$A-B = (a_{ij} – b_{ij}).$$

**Example**. The addition of the following \(3 \times 4\) matrices $$ A = \begin{pmatrix} 1 & 2 & 3 & 4\\ 5 & 6 & 7 & 8\\ 9 & 10 & 11 & 12 \end{pmatrix}$$ and $$ B = \begin{pmatrix} 0.1 & 0.2 & 0.3 & 0.4\\ 5.1 & 6.2 & 7.3 & 8.4\\ 9.11 & 10.15 & 11.19 & 12 \end{pmatrix}$$ is the following \(3 \times 4\) matrix: $$ A+B = $$ $$ \begin{pmatrix} 1.1 & 2.2 & 3.3 & 4.4\\ 10.1 & 12.2 & 14.3 & 16.4\\ 18.11 & 20.15 & 22.19 & 24 \end{pmatrix}.$$

**Remark**. If two matrices \(A\) and \(B\) do not have the same dimension, their addition and subtraction are undefined.

**Exercise**. Compute \(A-B\) where \(A\) and \(B\) are the matrices given above.

**Technical comment**. The set \(M_{p \times q}(\mathbb{R})\) of all \(p \times q\) real matrices together with the addition of matrices is an Abelian group.

### Matrix multiplication

The multiplication of matrices is a bit tricky. Let \(A\) and \(B\) be two real matrices. The multiplication of the matrix \(A\) by the matrix \(B\), denoted by \(A \cdot B\), can be defined (is definable) if the number of columns in the matrix \(A\) is the same as the number of rows in the matrix \(B\).

For example, let \(A\) and \(B\) be of dimension \(p \times q\) and \(q \times r\), respectively. Then, each array \(m_{ij}\) of \(M = AB\) is calculated by the following formula: $$m_{ij} = \sum_{k=1}^{q} a_{ik} b_{kj}.$$ Note that \(m_{ij}\) is the dot product of the \(i\)th row vector $$R_i = (a_{i1}, \dots, a_{iq})$$ of the matrix \(A\) and \(j\)th column vector $$C_j = (b_{1j}, \dots, b_{qj})$$ of the matrix \(B\).

**Example**. Consider the following real matrices: $$ A = \begin{pmatrix} 1 & 2 & 3\\ 5 & 6 & 7\end{pmatrix}$$ and $$ B = \begin{pmatrix} 0.1 & 0.2 & 0.3 & 0.4\\ 5.1 & 6.2 & 7.3 & 8.4\\ 9.11 & 10.15 & 11.19 & 12 \end{pmatrix}.$$ Since the dimension of \(A\) is \(2 \times 3\) and the dimension of \(B\) is \(3 \times 4\), the multiplication of \(A\) by \(B\) is definable. Set \(M = AB \). Then, for example, \(m_{22}\) is $$5 \times 0.2 + 6 \times 6.2 + 7 \times 10.15,$$ and so, \(m_{22} = 109.25\)

The reader may compute \(AB\) and the answer must be the following \(2 \times 4\) matrix: $$\begin{pmatrix} 37.63 & 43.05 & 48.47 & 53.2\\ 94.87 & 109.25 & 123.63 & 136.4 \end{pmatrix}.$$

**Remark**. Later, we will use the definition of “matrix multiplication” to convert a system of linear equations into a linear matrix equation.

#### Matrix multiplication properties

Matrix multiplication has the following properties:

- If \(A\), \(B\), and \(C\) are real matrices and the multiplication of \(A\) by \(B\) and the multiplication of \(B\) by \(C\) are definable, their the following associativity law holds: $$A(BC) = (AB)C.$$
- If \(A\), \(B\), and \(C\) are real matrices and \(B\) and \(C\) have the same dimension and \(AB\) is definable, then \(AC\) and \(A(B+C)\) are also definable and the following left distributive law holds: $$A(B+C) = AB + AC.$$ On the other hand, if \(BA\) is definable, then \(CA\) and \((B+C)A\) are also definable and the following right distributive law holds: $$(B+C)A = BA + CA.$$

**Remark**. Matrix addition and matrix multiplication are induced by the addition and composition of linear maps (linear functions). For more see the post on the linear maps in vector spaces.

#### The identity matrix

The identity matrix \(I_n\) of dimension \(n \times n\) is, by definition, a matrix that its arrays \(\delta_{ij}\)s are defined as follows:

- \(\delta_{ii} = 1,\)
- \(\delta_{ij} = 0,\) if \(i \neq j \).

The reader can easily verify that if \(A\) is a \(p \times q\) real matrix, then we have the following: $$I_p A = A I_q = A.$$

**Technical remark**. Let \(A\) and \(B\) be \(n \times n\) square matrices, where \(n \geq 2\). Then, it may happen that \(AB \neq BA.\) However, if we denote the set of all \(n \times n\) real matrices by \(M_{n \times n} \mathbb{R}\), then \(M_{n \times n} \mathbb{R}\) together with matrix addition and matrix multiplication is a unitary ring though not commutative.

**Exercise**. Let \(A\) and \(B\) be \(n \times n\) real matrices. Prove that $$(A+B)^2 = A^2 + 2AB + B^2$$ if and only if \(AB = BA\).

**Exercise**. A square real matrix \(D = (d_{ij})\) is, by definition, a diagonal matrix if \(d_{ij} = 0\), for all \(i \neq j\). Compute \(D^n\), for each positive integer \(n\), if \(D\) is a diagonal matrix.

## Unary matrix operations

The additive inverse of a matrix is the first unary matrix operation that we introduce. Observe that the additive inverse of a \(p \times q\) matrix \((a_{ij})\) is the \(p \times q\) matrix \((-a_{ij})\). Therefore, the dimension of the matrix \(A\) is the same as the dimension of its additive inversion \(-A\). However, this is sometimes not the case when computing the transpose of a matrix.

The multiple of a real matrix \((a_{ij})\) by a real number \(r\) is the matrix \((ra_{ij})\). Note that multiple of a matrix by a number is a function of the form $$\mathbb{R} \times M_{p \times q} (\mathbb{R}) \rightarrow M_{p \times q} (\mathbb{R}),$$ defined by $$ (r, (a_{ij})) \mapsto (ra_{ij}).$$ Now, if we fix a real number \(\alpha\), then $$ (a_{ij}) \mapsto (\alpha a_{ij})$$ is a unary matrix operation.

- \(r(A+B) = rA + rB\), if \(A\) and \(B\) are real matrices with the same dimension and \(r\) is a real number.
- \((r+s) A = rA + sA\), if \(A\) is a matrix, and, \(r\) and \(s\) are real numbers.
- \(r(sA) = (rs)A\), if \(A\) is a real matrix, and \(r\) and \(s\) are real numbers.
- \(1A = A\).

**Technical comment**. The set \(M_{p \times q}(\mathbb{R})\) of all \(p \times q\) real matrices together with the addition of matrices and the scalar multiplication is a real vector space.

In addition, since the multiplication of real numbers is commutative, the following property holds:

If \(A\) and \(B\) are real matrices and \(r\) is a real number, then $$rAB = ArB = ABr.$$

### Transpose of a matrix

Note that by definition the transpose of a \(p \times q\) matrix \(A=(a_{ij})\) is the \(q \times p\) matrix \(A^T = (a_{ji})\). It is, now, clear that the dimension of \(A^T\) is the same as the dimension of \(A\) if and only if \(p = q\), i.e. \(A\) is a square matrix.

Let \(A\) and \(B\) be real matrices. The transpose has the following properties:

- \((A+B)^T = A^T + B^T\), if \(A\) and \(B\) have the same dimension.
- \((cA)^T = cA^T\), if \(c\) is a real number.
- \((AB)^T = B^T A^T\), if the multiplication \(AB\) is definable.

**Remark**. A matrix \(A\) is symmetric if \(A^T = A\). Note that all symmetric matrices need to be square matrices.

### Multiplicative inverse of a matrix

Let \(A\) be a real matrix. By definition, \(A\) is invertible, if there are two real matrices \(B\) and \(C\) such that $$A B = C A = I,$$ where \(I_n\) is the identity \(n \times n\) matrix. From the definition of a multiplicative inverse of a matrix, we easily obtain that the matrices \(A\), \(B\) and \(C\) are \(n \times n\) matrices and \(B = C\). We add that if a square matrix \(A\) is invertible, then its inverse is denoted by \(A^{-1}\).

Although deciding whether a matrix is invertible or not, and if it is invertible, how to compute its multiplicative inverse is a bit tricky. These are the things we will discuss later. However, since a \(2 \times 2\) real matrix makes things easier to handle, we will discuss them here.

#### The multiplicative inverse of 2 by 2 matrices

Let \(A\) be the following \(2 \times 2\) real matrix: $$ A = \begin{pmatrix} a & b\\ c & d \end{pmatrix}.$$ Set $$ B = \begin{pmatrix} d & -b\\ -c & a \end{pmatrix}.$$ It is easy to see that $$AB = BA = $$ $$\begin{pmatrix} ad-bc & 0\\ 0 & ad-bc \end{pmatrix}.$$ From this easy calculation, one can deduce that the matrix \(A\) is invertible if and only if \(ad-bc\) is nonzero. Observe that if \(ad-bc\) is nonzero, the multiplicative inverse of \(A\) is the following matrix: $$A^{-1} = \frac{1}{ad-bc}\begin{pmatrix} d & -b\\ -c & a \end{pmatrix}.$$

The number \(ad-bc\) is called to be the determinant of the matrix \(A\) and denoted by \(\det(A)\), i.e. in this easy case, $$\det(A) = ad-bc.$$

**Example**. Find the multiplicative inverse of the following matrices:

- \(A = \begin{pmatrix} 5 & 3\\ 3 & 2 \end{pmatrix}\)
- \(S = \begin{pmatrix} 2 & 4\\ 1 & 2 \end{pmatrix}\)

Since \(\det(A) = 1\), the matrix \(A\) is invertible, and its multiplicative inverse is $$ A^{-1} = \begin{pmatrix} 2 & -3\\ -3 & 5 \end{pmatrix}.$$

Since \(\det(S) = 0\), the matrix \(S\) is not invertible.

**Exercise**. Let \(A\) be an invertible matrix. Prove that the transpose of the inverse of the matrix \(A\) is the same as the inversion of the inverse of the matrix \(A\).

**Solution**. Observe that $$A^T (A^{-1})^T = (A^{-1} A)^T = I^T = I.$$ This means that \((A^{-1})^T\) is the same as \((A^T)^{-1}.\)

**Exercise**. Find the multiplicative inverse of the following matrix: $$ R = \begin{pmatrix} \cos\theta & -\sin\theta\\ \sin\theta & \cos\theta \end{pmatrix}.$$

**Exercise**. Let \(r\) be a nonzero real number and \(A\) be an invertible real matrix. Prove that $$(rA)^{-1} = r^{-1} A^{-1}.$$

### Matrix exponential

Let \(X\) be a square matrix. Then, similar to single-variable calculus, the exponential of the matrix \(X\) is defined as follows: $$ e^X = \sum_{n=0}^{+\infty} \frac{1}{n!} X^n.$$ Note that \(e^X\) is convergent, for any square matrix \(X\).

Observe that if \(X\) is nilpotent, i.e., there is a positive integer \(p\) such that \(X^p = 0\), then $$ e^X =$$ $$ I + X + \dots + \frac{1}{(p-1)!} X^{p-1}.$$

**Exercise**. Prove that if \(D\) is a diagonal matrix and \(E = e^D\), then any array \(e_{ij}\) of \(E\) is zero if \( i\neq j\) and \(e_{ii} = e^{d_{ii}}\).

**Exercise**. Let \(X\) be a square matrix. Prove that $$e^{(X^T)} = {(e^X)}^T.$$