Vector orthogonal projection is an essential operation in the Gram-Schmidt orthonormalizing process and the least square problem. In this post, we discuss vector orthogonal projection, give some suitable examples, and solve some exercises.

#### A motivational example of a vector orthogonal projection in analytic geometry

Consider the point \(P=(1,7)\) and the line \(L\) passing through the point \(N=(-4,2)\) and the origin in the plane \(\mathbb{R}^2\). One good question is to find the nearest point on the line \(L\) to the point \(P\). By finding the nearest point, we can determine the shortest distance between \(P\) and \(L\).

The parametric equation of the line \(L\) is as follows: $$x = -4t, y = 2t.$$

Therefore, the distance of an arbitrary point on \(L\) from the point \(P\) is: $$d(t) = \sqrt{(-4t – 1)^2 + (2t-7)^2},$$ and so, $$d(t) = \sqrt{5(2t-1)^2 + 45}.$$ This implies that \(d(t)\) minimizes if \(t = 1/2\), and so, the nearest point on \(L\) to \(P\) is the point $$Q = (-2,1).$$

Now, we proceed to find the angle between the sides \(\overline{OQ}\) and \(\overline{QP}\) of the triangle \(\triangle{OPQ}\). The vectors parallel to the sides \(\overline{OQ}\) and \(\overline{QP}\) are $$u = (-2,1)$$ and $$v = (1,7) – (-2,1) = (3,6)$$ and since the dot product of \(u\) and \(v\) is zero, we see that the sides \(\overline{OQ}\) and \(\overline{QP}\) are perpendicular to each other. In this case, it is said that the vector \(u = \vec{OQ}\) is the orthogonal projection of the vector \(\vec{OP}\) onto the vector \(\vec{ON}\).

### Vector orthogonal projection in inner product real vector spaces

Let \(V\) be an inner product real vector space and \(y\) be a vector in \(V\). As a generalization to the above observation, we would like to decompose the vector \(y\) into the addition of two vectors such that one is parallel to a given nonzero vector \(u\) and the other orthogonal to \(u\). Therefore, we are searching for two vectors \(\hat{y}\) and \(z\) such that \(y = \hat{y} + z,\) where \(\hat{y} = ru\) for some real number \(r\) and \(u \cdot z = 0\).

Observe that the vector $$z = y – \hat{y} = y – ru$$ is orthogonal to the vector \(u\) if and only if $$0 = (y-ru) \cdot u = (y \cdot u) – r (u \cdot u).$$ This implies that $$r = \frac{y \cdot u}{u \cdot u}.$$ Note that the real number \(r\) is always definable since \(u\) is nonzero, and so, \(u \cdot u\) which is in the denominator of the latter fraction is nonzero.

**Remark**. For the definition of the inner product vector spaces, check the post on norms of vector spaces.

**Exercise**. Find the orthogonal projection of the vector \(y\) onto the vector \(u\) in \(\mathbb{R}^3\), if $$ y = (7,6,2) \text{ and } u = (4,2,1).$$

**Solution**. Observe that $$y \cdot u = 42, u \cdot u = 21.$$ The orthogonal projection of \(y\) onto \(u\) is computed as follows: $$\hat{y} = \frac{y \cdot u}{u \cdot u} u = 2u.$$ So, \(\hat{y} = (8,4,2).\)

#### An advanced example of an inner product real vector space

Let \(a < b\) be two real numbers. Define \(C[a,b]\) to be the set of all continuous functions from the closed interval \([a,b]\) into the set of real numbers \(\mathbb{R}\). By some results on continuity of functions in single-variable calculus, it is easy to see that \(C[a,b]\) is a real vector space.

Now, the reader may try to prove that the function $$C[a,b] \times C[a,b] \rightarrow \mathbb{R}$$ by $$(f,g) \mapsto \frac{1}{b-a} \int_{a}^{b} f(t)g(t) dt$$ defines an inner product on \(C[a,b]\). We recall the reader that the proof of the claimed result needs the following:

**A Result in Mathematical Analysis**. Let \(f: [a,b] \rightarrow \mathbb{R}\) be a non-negative continuous function. If \(\int_{a}^{b} f(t)dt = 0,\) then \(f = 0\).

**Exercise**. Find the orthogonal projection of the vector \(y\) onto the vector \(u\) in \(C[0,1]\), if $$ y(t) = 2t \text{ and } u(t) = 12t^2 .$$

**Solution**. Note that $$y(t) \cdot u(t) = \int_{0}^{1} 24t^3 dt = 6$$ and $$ u(t) \cdot u(t) = \int_{0}^{1} 144t^4 dt = \frac{144}{5}.$$ It is, now, obvious that $$\hat{y}(t) = \frac{6}{\frac{144}{5}} u(t) = 5t^2/2.$$

For Gram-Schmidt process see the post on basis for vector spaces.