### Definition of the dot product in the two-dimensional space

The definition of the dot product of two vectors \(u = (u_1,u_2)\) and \(v = (v_1,v_2)\), denoted by \( u \cdot v \), is as follows: $$ u \cdot v = u_1 v_1 + u_2 v_2. $$

**Example**. Let us compute the dot product of \(u = (3,\pi)\) and \(v = (-\pi,2)\):

$$ u \cdot v = 3 \times (-\pi) + \pi \times 2. $$ So, \(u \cdot v = -\pi.\)

In the following, we give the basic algebraic properties of the dot product:

Let \(u\) and \(v\) be arbitrary two-dimensional real vectors.

- \(u \cdot v = v \cdot u\), (commutativity of the dot product).
- \( u \cdot (v+w) = u \cdot v + u \cdot w \), (the distributive law).
- \( (ru) \cdot (sv) = (rs) (u \cdot v)\), for all real numbers \(r\) and \(s\).
- \( u \cdot \vec{0} = \vec{0} \cdot u = 0\).

**Question**. How do the concept of the dot product and the concept of the norm of a vector relate to each other?

**Answer**. The dot product of a vector by itself is the square of its norm. In other words, if \( u \) is an arbitrary vector, then $$ u \cdot u = \Vert u \Vert^2.$$

**Exercise**. Prove the correctness of the above claim.

#### Definition of linear functions with the help of the dot product

Note that a function \(f\) from \(\mathbb{R}^2 \) into \(\mathbb{R} \) is a linear function if and only if \(f\) is defined as follows: $$f(x,y) = px+qy,$$ for some real numbers \(p\) and \(q\). In the language of the dot product of vectors, we re-state this as follows:

**Example**. If \(k = (3,-2)\) and \(f(u) = k \cdot u \), for all \(u \in \mathbb{R}^2\), then $$f(x,y) = 3x-2y.$$

### Definition of the dot product in the three-dimensional space

Similar to the case of two-dimensional vectors, definition of the dot product of three-dimensional vectors is the sum of the element-wise multiplication of their corresponding components. In other words, if $$ u = (u_1, u_2, u_3)$$ and $$v = (v_1, v_2, v_3),$$ then their dot product is obtained from the following formula:

$$ u \cdot v = u_1 v_1 + u_2 v_2 + u_3 v_3.$$

**Example**. If $$ u = (3, \pi, \sqrt{\pi})$$ and $$v = (-\pi, 2, \sqrt{\pi}),$$ then their dot product is: $$-3\pi + 2\pi + \pi = 0.$$

## The Cauchy-Schwarz inequality

One of the most important inequalities in mathematical optimization is the Cauchy-Schwarz inequality.

Note that the Cauchy-Schwarz inequality is dependent on the algebraic properties of the dot product mentioned above and this point that $$ u \cdot u \geq 0, $$ for each vector \(u.\)

The proof of the above claim is as follows:

Let \(u\) and \(v\) be arbitrary vectors of the same kind, i.e., either they are both two-dimensional or three-dimensional. The dot product of the vector \(u-v\) by itself is always non-negative. Using the commutativity of the dot product, and also, the distributive law explained above, we see that $$(u-v) \cdot (u-v)$$ equals to $$u \cdot u + v \cdot v – 2 u \cdot v.$$

From all we said and in view of the formula, $$ u \cdot u = \Vert u \Vert^2,$$ we obtain that $$u \cdot v \leq (\Vert u \Vert^2 + \Vert v \Vert^2)/2.$$ Without loss of generality, we may assume that \(u\) and \(v\) are nonzero vectors (if one of them is the zero vector, the inequality holds obviously).

In the recent inequality, we replace \(u\) by \(u/\Vert u \Vert \), and, \(v\) by \(v/\Vert v \Vert \). Since \(u/\Vert u \Vert \) and \(v/\Vert v \Vert \) are unit vectors, we have $$(u/\Vert u \Vert) (v/\Vert v \Vert ) \leq (1^2 + 1^2)/2.$$ From this, we obtain that $$ u \cdot v \leq \Vert u \Vert \Vert v \Vert.$$ Now, in the latter inequality, if we replace \(v\) by \(-v\), we obtain that $$-\Vert u \Vert \Vert v \Vert \leq u \cdot v,$$ and finally, the Cauchy-Schwarz inequality is obtained and the proof is complete.

### The angle between two vectors

Using the elementary Euclidean geometry techniques, it is easy to see that the cosine of the angle between two nonzero vectors is equal to the dot product of the two vectors over the multiplication of their norms. In other words, if \(u\) and \(v\) are two nonzero vectors and the angle between them is \(\theta\), then $$\cos \theta = \frac{u \cdot v}{\Vert u \Vert \Vert v \Vert}.$$

**Exercise**. Find the angle between the following vectors:

- \((1,1,0)\) and \((0,0,1)\).
- \((1,1,0)\) and \((0,1,1)\).

Since the dot product of \((1,1,0)\) and \((0,0,1)\) is zero, the angle between them is 90 degrees, i.e., they are perpendicular to each other.

Now, let \(u = (1,1,0)\) and \(v = (0,1,1)\). It is clear that \(\Vert u \Vert\) and \(\Vert v \Vert \) are \(\sqrt{2}\). Also, it is easy to see that $$u \cdot v = 1.$$ Therefore, if the angle between these two vectors is \(\theta\), then we have $$\cos \theta = 1/2.$$ So, the angle between \(u\) and \(v\) is 60 degrees.