There are certain kinds of matrices and vectors that are particularly useful in data science. With examples, we explain special matrices and vectors.

### Special matrices and vectors

#### Special matrices (diagonal and elementary)

In this section, we discuss special matrices. Some special matrices like orthogonal matrices need more detailed discussion. For this reason, we argue them in separate posts.

**Diagonal matrices**. A matrix \(D = (d_{ij})\) is diagonal if \(d_{ij} = 0\) for all \(i\) and \(j\) with \(i \neq j\). For example, the identity matrix \(I\) is a diagonal matrix. Another example of a diagonal matrix is a square zero matrix. Recall that a square matrix \(A\) is an identity matrix if \(a_{ii} = 1\), for each \(i\) and \(a_{ij} = 0\) if \(i \neq j\).

It is clear that a diagonal matrix \(D = (d_{ij})\) is invertible if and only if \(d_{ii} \neq 0\) for each \(i\). The inverse of a diagonal matrix \(D = (d_{ij})\) is \(D^{-1} = (d^{-1}_{ij})\).

The diagonal matrix $$D = \begin{pmatrix} 2 & 0 & 0 \\ 0 & – 3 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$ is invertible.

**Elementary matrices**. By definition, a square matrix derived by applying an elementary row operation to the identity matrix is an elementary matrix. Elementary matrices have applications in finding the inverse of an invertible matrix, solving systems of linear equations, and the rank of a matrix.

#### Special matrices (idempotent and orthogonal)

**Idempotent matrices**. In linear algebra, an idempotent matrix is a matrix which, when multiplied by itself, yields itself. In other words, A square matrix \(A\) is idempotent if \(A^2 = A\).

**Exercise**. Show that if \(A\) is an idempotent matrix, then so is the matrix \(I – A\).

Also, note that if \(k \geq 2\) is a positive integer, then a square matrix \(A\) is \(k\)-potent if \(A^k = A\).

**Orthogonal matrices**. A matrix \(A\) is orthogonal if $$A A^T = A^T A = I.$$ In other words, a square matrix \(A\) is orthogonal if its inverse equals to its transpose. For examples of orthogonal matrices, see orthogonal matrices.

#### Special matrices (symmetric and triangular)

**Symmetric matrices**. A matrix \(A\) is symmetric if it equals to its own transpose, i.e., \(A = A^T\). In other words, a matrix \(A = (a_{ij})\) is symmetric if and only if \(a_{ij} = a_{ji}\), for all indices \(i\) and \(j\).

Consider \(n\) vectors, say \(u_i\)s, in \(\mathbb{R}^m\). The distance matrix of \(u_i\)s is defined as follows: $$D = ( d(u_i, u_j) ).$$ Since \(d(u_i, u_j) = d(u_j, u_i)\), for all indices, \(D\) is an example of a symmetric matrix.

A matrix \(A = (a_{ij})\) is, by definition, a Lehmer matrix if for all indices \(i\) and \(j\), $$a_{ij} = \frac{\min\{i,j\}}{\max\{i,j\}}.$$ It is evident that any Lehmer matrix is symmetric.

A Lehmer matrix of order 3 is: $$L_3 = \begin{pmatrix} 1 & 1/2 & 1/3 \\ 1/2 & 1 & 2/3 \\ 1/3 & 2/3 & 1\end{pmatrix}.$$

**Remark**. The trace of a Lehmer matrix of order \(n\) is \(n\).

**Triangular matrices**. Triangular matrices are special kinds of square matrices. A square matrix is called lower-triangular if all the entries above the main diagonal are zero. Similarly, a square matrix is called upper-triangular if all the entries below the main diagonal are zero.

The matrix $$\begin{pmatrix} 1 & -3 & 1 \\ 0 & 6 & -9 \\ 0 & 0 & 1 \end{pmatrix}$$ is upper-triangular.

**Remark**. A square matrix is diagonal if it is both lower-triangular and upper-triangular.

### Special vectors

An \(n\)-dimensional real vector is a unit vector if its Euclidean norm is 1. In other words, a vector $$ u = (u_1, \dots, u_n)$$ is unit if $$ \sqrt{u^2_1 + \dots + u^2_n} = 1.$$ For example, $$ u = (1/\sqrt{3}, -1/\sqrt{3}, 1/\sqrt{3})$$ is a unit vector in the space of real three-dimensional vectors.

Two \(n\)-dimensional nonzero real vectors \(u\) and \(v\) are orthogonal if their dot product equals zero. The vectors \(u\) and \(v\) are orthonormal if they are unit vectors and orthogonal.

An \(n\)-dimensional real vector is a zero vector if and only if each of its components is zero, or equivalently, its Euclidean norm is \(0\).