The main purpose of this post is to introduce different matrix norms such as Frobenius norm and \(p\)-norms. In the following, first we give a general definition of matrix norms.

### General definition of matrix norms

In the post on matrix operations, it is explained that \(M_{n\times n}(\mathbb{R})\) is a real vector space. A matrix norm on \(M_{n\times n}(\mathbb{R})\) is a vector space norm \(\Vert \cdot \Vert\) on \(M_{n\times n}(\mathbb{R})\) with the so-called submultiplicative property:

- \(\Vert AB \Vert \leq \Vert A \Vert \Vert B \Vert,\) for all matrices \(A\) and \(B\) in \(M_{n\times n}(\mathbb{R})\).

It is clear that if \(k\) is an arbitrary positive integer, then $$\Vert A^k \Vert \leq \Vert A \Vert^k.$$

By definition, a real square matrix \(A\) is idempotent if \(A^2 = A.\) More generally, if \(k > 1\) is a positive integer, a real square matrix \(A\) is defined to be \(k\)-potent if \(A^k = A.\)

**Exercise**. Let \(A\) be a nonzero \(k\)-potent matrix. Prove the matrix norm of \(A\) is not less than 1.

**Solution**. Observe that $$\Vert A \Vert = \Vert A^k \Vert \leq \Vert A \Vert^k.$$ Since \(A\) is nonzero, its matrix norm is also a positive real number. This implies that $$ 1 \leq \Vert A \Vert^{k-1}.$$ Obviously, this causes the matrix norm of \(A\) to be not less than 1 and the proof is complete.

#### Examples of matrix norms

**Frobenius norm**. The first matrix norm that may come to one’s mind is the Frobenius norm, denoted by \(\Vert \cdot \Vert_{F}\) and defined as follows: Let \(A\) be a real square matrix. Then, $$\Vert A \Vert_{F} = \sqrt{\sum_{i,j = 1}^{n} a^2_{ij}}.$$

**Exercise**. Find the Frobenius norm of the following matrix: $$A = \begin{pmatrix} 3 & 4\\ 5 & 6 \end{pmatrix}.$$

**Solution**. Observe that $$\Vert A \Vert_{F} = $$ $$\sqrt{3^2 + 4^2 + 5^2 + 6^2} \approx$$ $$ 9.2736.$$

**Remark**. Frobenius norm is also called the Hilbert-Schmidt norm or the Schur norm, in some references.

We skip the proof of the following result:

**Result**. Fix a vector norm \(\Vert \cdot \Vert\) on \(\mathbb{R}^n\) and assume that \(A\) is a square \(n \times n\) real matrix. Then, for any vector \(u\) in \(\mathbb{R}^n\), we have the following: $$\Vert Au \Vert \leq m \Vert u \Vert,$$ where \(m\) is a fix non-negative real number depending only on the norm \(\Vert \cdot \Vert\) and the matrix \(A\).

A corollary to this result is that if \(\Vert \cdot \Vert\) is a vector norm on \(\mathbb{R}^n\) and \(A\) is a square \(n \times n\) real matrix, then there is a non-negative real number \(m\) such that for all nonzero vector \(u\) in \(\mathbb{R}^n\), we have $$\frac{\Vert Au \Vert}{\Vert u \Vert} \leq m.$$ This implies that $$\sup_{u \neq \vec{0}} \frac{\Vert Au \Vert}{\Vert u \Vert} \leq m.$$ On the other hand, if \(u\) is a nonzero vector, then $$\frac{\Vert Au \Vert}{\Vert u \Vert} = \Vert A(u/\Vert u \Vert) \Vert.$$ Hence, we have the following: $$ \sup_{u \neq \vec{0}} \frac{\Vert Au \Vert}{\Vert u \Vert} = \sup_{\Vert u \Vert = 1} \Vert Au \Vert.$$

#### Operator norm

By definition, if \(\Vert \cdot\Vert\) is a vector norm, then \(\Vert A\Vert_{op}\) is defined as follows: $$ \Vert A\Vert_{op} = \sup_{\Vert u \Vert = 1} \Vert Au \Vert.$$ One can check that \(\Vert \cdot \Vert_{op}\) is indeed a vector norm on \(M_{n \times n}(\mathbb{R})\) satisfying the following property for all vectors \(u\): $$\Vert Au \Vert \leq \Vert A\Vert_{op} \Vert u \Vert.$$

Using this, we obtain that $$\Vert ABu \Vert \leq \Vert A\Vert_{op} \Vert Bu \Vert \leq $$ $$\Vert A\Vert_{op} \Vert B\Vert_{op} \Vert u \Vert.$$ Definitely, this implies that $$\Vert AB \Vert_{op} \leq \Vert A\Vert_{op} \Vert B\Vert_{op}$$ and this means that \(\Vert \cdot \Vert_{op}\) is, in fact, a matrix norm on \(M_{n \times n}(\mathbb{R})\) known as the “operator norm” induced by the given norm \(\Vert \cdot \Vert\).

#### How to compute some operator norms

The operator norm induced by \(\Vert \cdot \Vert_1\) is computed as follows: $$\Vert A \Vert_1 = \max_{j} \sum_{i=1}^{n} \vert a_{ij} \vert.$$

**Exercise**. Compute \(\Vert A \Vert_1\) of the following matrix: $$A = \begin{pmatrix} 3 & 4\\ 5 & 6 \end{pmatrix}.$$

**Solution**. First you sum the absolute value of each array that you see on each column separately, then, you choose the maximum of such numbers. $$\Vert A \Vert_1 = \max\{3+5, 4+6\} = 10.$$

The operator norm induced by \(\Vert \cdot \Vert_{\infty}\) is computed as follows: $$\Vert A \Vert_{\infty} = \max_{i} \sum_{j=1}^{n} \vert a_{ij} \vert.$$

**Exercise**. Compute \(\Vert A \Vert_{\infty}\) of the following matrix: $$A = \begin{pmatrix} 3 & 4\\ 5 & 6 \end{pmatrix}.$$ The solution is left to the reader. The answer is 11.

The computation of the operator norm induced by \(\Vert \cdot \Vert_{2}\) needs more advanced tools in linear algebra and is usually difficult and challenging. However, there are some software packages which can help with the computation of \(\Vert A \Vert_2.\)