0.1 Vectors

We have discussed on vectors more in-depth in the part for Linear Algebra. In this section however, let’s focus more on the geometric intuition than the abstract nature of vectors.

Some example of vectors include force, velocity, and acceleration because they all have a specific direction and also a magnitude. When we graph them, we can think of them as a arrow with starting and ending points.

From now, let’s represent vectors with \(\mathbf {u}, \mathbf {v}, \mathbf {w}\) and scalars, or real numbers, with \(a, b, \alpha , \beta \). From the graph above, we can define another definition.

When we want to add two vectors \(\mathbf {v}\) and \(\mathbf {w}\), we would connect the terminal point of \(\mathbf {v}\) by the starting point of \(\mathbf {w}\) and obtain a new vector by connecting the initial point of \(\mathbf {v}\) and the terminal point of \(\mathbf {w}\) as shown by the figure below.

Multiplying a vector by scalar \(k\) can be though of as lengthening the vector by a factor of \(k\). If \(k < 0\), the vector is pointing the opposite direction.

Also for the purpose of the notes, let’s consider the vectors with same magnitude and direction but different initial vectors as the same vectors. The being said, we can write any vectors in \(n\)-dimension as \(\langle v_1, v_2, v_3, \ldots , v_n \rangle \) where \((v_1, v_2, v_3, \ldots , v_n)\) is the terminal point and \((0, 0, , 0 \ldots , 0)\) is the initial point. Such numbers \(v_1, \ldots , v_n\) are known as the components of \(\mathbf {v}\).

Now that we have defined \(-\mathbf {v}\), we can consider vector subtraction. We can consider vector subtraction as adding two vectors where one is scaled by a negative factor.

Using components, we can write addition, subtraction and scalar multiplication as the following for \(\mathbf {v} = \langle v_1, v_2, v_3 \rangle \) and \(\mathbf {w} = \langle w_1, w_2, w_3 \rangle \). \begin{align*} \mathbf {v} + \mathbf {w} &= \langle v_1 + w_1, v_2 + w_2, v_3 + w_3 \rangle \\ \mathbf {v} - \mathbf {w} &= \langle v_1 - w_1, v_2 - w_2, v_3 - w_3 \rangle \\ k\mathbf {v} &= \langle kv_1, kv_2, kv_3 \rangle \end{align*}

As you can see from the equations above, we can see that vector addition is commutative and associative and scalar multiplication is distributive. I will keep them brief here because we already discussed about these in-depth on my other notes for linear algebra. Now this property leads to the natural question of subtracting a vector by itself.

Intuitively, we can think of a zero vector as vector with the same initial and terminal points. Another way of thinking about a zero vector is a vector with zero as its components. This will naturally leads to \(\mathbf {v} + \mathbf {0} = \mathbf {v}\) for any vector \(\mathbf {v}\). For the purpose of this notes, let’s consider zero vector as the vector that is parallel to every vectors.

Continuing with the definition of components and the magnitude of a vector, we can define an important terminology.

Notice that the magnitude, or length, cannot be negative. Therefore, we can see that the norm of any vector is non-negative. Moreover, \(\| \mathbf {0} \| = 0\). Also note that scalar multiplication is defined element-wise. Using the distance formula, we can also obtain the following very important equation for a vector \(\mathbf {v}\) and scalar \(k\). \[ \| k \mathbf {v} \| = |k| \cdot \| \mathbf {v} \| \] Let’s take a look at a quick example. Consider the vector \(\mathbf {v} = \langle 1, 2, 3 \rangle \). Notice that the following equation holds. \[ \| \mathbf {v} \| = \sqrt {1^2 + 2^2 + 3^2} = \sqrt {14} \] We can also see that the following equation holds with scalar \(2\) and \(2 \langle 1, 2, 3 \rangle = \langle 2, 4, 6 \rangle \). \begin{align*} \| 2 \mathbf {v} \| &= \sqrt {2^2 + 4^2 + 6^2} = \sqrt {2^2 (1^2 + 2^2 + 3^2)} \\ &= 2 \sqrt {1^2 + 2^2 + 3^2} \\ &= 2 \cdot \| \mathbf {v} \| \end{align*}

With this equation in mind, let’s consider the following question.

One way of thinking this is rearranging our equation \(\| k \mathbf {v} \| = |k| \cdot \| \mathbf {v} \|\). Consider the following equations with unit vector \(\mathbf {u}\) and \(\| \mathbf {v} \| = k \neq 0\). \[ \left | \frac {1}{k} \right | \cdot \| \mathbf {v} \| = \frac {1}{k} \cdot k = 1 = \| \mathbf {u} \| \] This solves it! We can find a unit vector with the same direction \(\mathbf {v}\) by dividing it by its norm. Another variation of this equation would be the following. \[ \| \mathbf {v} \| = |k| \cdot \| \mathbf {u} \| \]

Finally, another standard way of writing vectors is by linear combination of the standard basis vectors defined as the following. \[ \mathbf {i} = \langle 1, 0, 0 \rangle , \quad \mathbf {j} = \langle 0, 1, 0 \rangle , \quad \mathbf {k} = \langle 0, 0, 1 \rangle \] With this, we can write \(\mathbf {v} = \langle 1, 2, 3 \rangle \) as \(\mathbf {i} + 2\mathbf {j} + 3\mathbf {k}\).

More detailed and abstract introduction of vectors are included in the linear algebra part of LibreNotebook, and I highly recommend checking that out! From here, we will discuss more foundational and geometric interpretations of vectors that we have not discussed in my notes for linear algebra.

0.1.1 Dot Products and Projections

As always, let’s start with definition.

Writing them with mathematical notations, we can write them as the following. \[ \langle u_1, u_2, \ldots , u_n \rangle \cdot \langle v_1, v_2, \ldots , v_n \rangle = \sum _{i=1}^n u_i v_i \] Here is a quick example. \[ \langle 1, 2, 3 \rangle \cdot \langle -1, 0, 1 \rangle = -1 + 0 + 3 = 2 \] Continuing, below are some properties of dot products. The statements below are quite self-evident as they mirror the properties of addition and subtraction.

One of the reasons why the dot product of two vectors is so important is because it helps us determine the angle between the two vectors. Consider the following theorem.

Now this theorem tells us the relationship between the angle between two vectors and their dot product.

This somewhat simple observation with the cosine function and the fact that \(\| \mathbf {u} \|, \| \mathbf {u} \| > 0\) reveals the relationship between the dot product of two vectors and their angles. Continuing from this observation, we can establish a definition.

Continuing with the relationship and definition, let’s discuss more on the angle of vectors.

To find the direction angles of \(\mathbf {v} = \langle v_1, v_2, v_3 \rangle \), let’s return to Theorem 0.1.7. Consider the following equation for the angle \(\alpha \) between \(\mathbf {v}\) and the standard basis vector \(\mathbf {i}\). \begin{align*} \mathbf {v} \cdot \mathbf {i} &= \| \mathbf {v} \| \| \mathbf {i} \| \cos \alpha \\ v_1 &= \| \mathbf {v} \| \cos \alpha \\ \therefore \alpha &= \cos ^{-1} \left ( \frac {v_1}{\| \mathbf {v} \|} \right ) \end{align*}

Following the same pattern for \(\beta \) and \(\gamma \), the can establish the following equations. \[ \cos \alpha = \frac {v_1}{\| \mathbf {v} \|}, \quad \cos \beta = \frac {v_2}{\| \mathbf {v} \|}, \quad \cos \gamma = \frac {v_3}{\| \mathbf {v} \|} \] Indeed, we have special term for the values above. The values above are known as the direction cosines of \(\mathbf {v}\). For vectors of direction cosines, the can write the following equation. \[ \langle \cos \alpha , \cos \beta , \cos \gamma \rangle = \left \langle \frac {v_1}{\| \mathbf {v} \|}, \frac {v_2}{\| \mathbf {v} \|}, \frac {v_3}{\| \mathbf {v} \|} \right \rangle = \frac {\mathbf {v}}{\| \mathbf {v} \|} \] Continuing, let’s consider the case when \(\| \mathbf {v} \| = 1\). In that case, \[ \langle \cos \alpha , \cos \beta , \cos \gamma \rangle = \left \langle \frac {v_1}{\| \mathbf {v} \|}, \frac {v_2}{\| \mathbf {v} \|}, \frac {v_3}{\| \mathbf {v} \|} \right \rangle = \mathbf {v} \] also hold! Finally, here is another interesting properties of the direction cosines. \[ \cos ^2\alpha + \cos ^2\beta + \cos ^2\gamma = \left ( \frac {v_1}{\| \mathbf {v} \|} \right )^2 + \left ( \frac {v_2}{\| \mathbf {v} \|} \right )^2 + \left ( \frac {v_3}{\| \mathbf {v} \|} \right )^2 = \frac {\| \mathbf {v} \|^2}{\| \mathbf {v} \|^2} = 1 \] We can see that for nonzero \(\mathbf {v}\), we can generalize this for any \(n\). Therefore the sum of the squares of direction cosines are always \(1\).

The last part that I wish to discuss about dot products is projections. First, consider a vector \(\mathbf {v} = \mathbf {w}_1 + \mathbf {w}_2\) in a plane where \(\mathbf {w}_1\) and \(\mathbf {w}_2\) are along the standard basis \(\mathbf {i}\) and \(\mathbf {j}\) respectively. By construction, we can write \(\mathbf {w}_1 = \mathbf {v} \cdot \mathbf {i}\) and \(\mathbf {w}_2 = \mathbf {v} \cdot \mathbf {j}\).

This idea can be generalized for higher dimensions. Moreover, although we used standard basis vectors here, this works for any orthogonal unit vectors.

Another way of expanding this is through geometric interpretation.

As you can see from the diagrams above, we can write the scalar components with respect to the norm of \(\mathbf {v}\) and the angle \(\theta \). Moreover, we can define a new definition.

From the figures above, we can see that \(\proj _{\mathbf {e}_1} = \mathbf {w}_1\) and \(\proj _{\mathbf {e}_2} = \mathbf {w}_2\). Finally, this leads us to the conclusion that for any vector \(\mathbf {v}\) and respective orthogonal unit vectors \(\mathbf {e}_1\) and \(\mathbf {e}_2\), the following equation holds. \[ \mathbf {v} = \proj _{\mathbf {e}_1} \mathbf {v} + \proj _{\mathbf {e}_2} \mathbf {v} \] This is it for dot products and orthogonal projections! Before we conclude our section on vectors, we will discuss about cross products and its geometric interpretations.

0.1.2 The Cross Product and Geometry

Unlike how we defined vector addition and scalar multiplication, we had a special way of defining “multiplication” of vectors with the dot product. However specifically for \(3\)-dimensional space, we can also define cross products.

As you can see from the definition above, unlike the dot product that returns a scalar, the cross product of two vector return a vector. Also, if it is difficult to memorize the formula above, we can also think of cross product as the following. \begin{align*} \mathbf {u} \times \mathbf {v} &= \det \left ( \begin {bmatrix} \mathbf {i} & \mathbf {j} & \mathbf {k} \\ u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \end {bmatrix} \right ) \\ &= (u_2v_3 - u_3v_2) \mathbf {i} - (u_1v_3 - u_3v_1) \mathbf {j} + (u_1v_2 - u_2v_1) \mathbf {k} \end{align*}

If you are unfamiliar with the operation above, I highly recommend reading my notes on determinants, which is in Section ??!

Unlike the dot product, the properties of the cross product are not so nice.

There are a lot to prove, but let’s start with the first one.

Below is the proof for the second one.

Continuing, here is the proof for the third property.

Below is the proof for the fourth property.

Lastly, here is the proof for the last property.

Continuing, it is worth memorizing the cross product between the standard basis vectors. \[ \mathbf {i} \times \mathbf {j} = \mathbf {k}, \quad \mathbf {j} \times \mathbf {k} = \mathbf {i}, \quad \mathbf {k} \times \mathbf {i} = \mathbf {j} \] I personally think and easy way to memorize this is through cycle. For cycle counter clockwise, we get a negative one. \[ \mathbf {j} \times \mathbf {i} = -\mathbf {k}, \quad \mathbf {k} \times \mathbf {j} = -\mathbf {i}, \quad \mathbf {i} \times \mathbf {k} = -\mathbf {j} \] We can also notice that the cross product returns a vector that are orthogonal to those being crossed!

However when we have a dot product and a cross product like \(\mathbf {u} \cdot (\mathbf {v} \times \mathbf {w})\), then parenthesis is not necessary as the cross product is defined only between two three dimensional vectors. Speaking of such forms, consider the following theorem.

From the theorem, we can notice that \(\mathbf {u}\) is orthogonal to \(\mathbf {u} \times \mathbf {v}\), which is orthogonal to \(\mathbf {v}\). These types of operation is known as scalar triple product, and we will be discussing more on that soon! In addition to the orthogonal nature of \(\mathbf {u} \times \mathbf {v}\) and its adherence to the right-hand rule, we can also find the relationship between the norm of \(\mathbf {u} \times \mathbf {v}\) and \(\mathbf {u}\) and \(\mathbf {v}\).

This theorem implies that the norm of the cross product of the two vectors is the product of the norms of the vectors and sine of the angle between them. With this, we can also find another geometric representation of cross products.

This is somewhat simple to notice as if we consider \(\mathbf {u}\) as the base, then \(\| \mathbf {v} \| \sin \theta \) is the height where \(\theta \) is the angle between them. Therefore by Theorem 0.1.14, the area of the parallelogram is \(\| \mathbf {u} \| \| \mathbf {v} \| \sin \theta = \| \mathbf {u} \times \mathbf {v} \|\).

Before we move on to the next section, let’s wrap up our discussion on vectors with the scalar triple product and its geometric representations.

Notice that the output of the scalar triple product of three vectors is a scalar. Also, using the definition of cross products and dot products, we can notice that the scalar triple product is equivalent to the following. \[ \mathbf {u} \cdot \mathbf {v} \times \mathbf {w} = \begin {bmatrix} u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end {bmatrix} \] From the matrix operation above, we can notice that the following equations hold. \begin{align*} \mathbf {u} \cdot \mathbf {v} \times \mathbf {w} &= \begin {bmatrix} u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end {bmatrix} = -\begin {bmatrix} u_1 & u_2 & u_3 \\ w_1 & w_2 & w_3 \\ v_1 & v_2 & v_3 \end {bmatrix} = \begin {bmatrix} w_1 & w_2 & w_3 \\ u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \end {bmatrix} \\ &= \mathbf {w} \cdot \mathbf {u} \times \mathbf {v} \end{align*}

Therefore, we could see that \(\mathbf {u} \cdot \mathbf {v} \times \mathbf {w} = \mathbf {w} \cdot \mathbf {u} \times \mathbf {v}\) and \(\mathbf {w} \cdot \mathbf {u} \times \mathbf {v} = \mathbf {v} \cdot \mathbf {w} \times \mathbf {u}\) both holds by cycling through the vectors.

Lastly before wrapping up, let’s take a look at geometric interpretation of the scalar triple product. First, consider a parallelepiped with edges \(\mathbf {u}, \mathbf {v}, \mathbf {w}\).

From our previous observation, we can see that the area of the base of the parallelepiped can be represented as \(\| \mathbf {v} \times \mathbf {w} \|\). Moreover the height can be represented as \(\| \mathbf {u} \| | \cos \theta |\) where \(\theta \) is the angle between \(\mathbf {u}\) and \(\mathbf {v} \times \mathbf {w}\). Therefore, the volume can be represented as \(\| \mathbf {u} \| | \cos \theta | \| \mathbf {v} \times \mathbf {w} \| = | \mathbf {u} \times \mathbf {v} \times \mathbf {w} |\). Continuing with this idea, we can establish an important theorem.

This is it for the vector section! There were lots of topics and geometric interpretations of vectors that were not included in the notes for linear algebra. In our next section, we will continue our discussion through more extensions of geometric representations of vectors.