LibreNotebook

0.1 Additional Fundamental Topics

This section discusses frequently appearing concepts in matrices that they have their own names!

0.1.1 Transpose

Let’s begin with transpose of a matrix.

Definition 0.1.1

The transpose of a matrix \(A\) is defined by a matrix that has switched rows and columns of \(A\), i.e. each element is defined as following. \[ (A^\intercal )_{ij} = A_{ji} \]

Let’s take a look at an example. You could think of it as holding the left top and bottom right of a matrix and rotating to see its back. For matrix \(A\) defined as the following, \[ A = \begin {bmatrix} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \end {bmatrix} \] its transpose, or \(A\) transpose is defined as the following. \[ A^\intercal = \begin {bmatrix} 1 & 5 \\ 2 & 6 \\ 3 & 7 \\ 4 & 8 \end {bmatrix} \] As you can see, if \(A\) has the order \(m \times n\), its transpose will have the order \(n \times m\). Another observation that we can make is that a transpose of a row matrix is a column matrix and vice versa. Here is a quick example. Assume matrix \(A\) defined as the following. \[ A = \begin {bmatrix} 1 & 2 & 3 \end {bmatrix} \] The transpose of \(A\) can be represented as the following. \[ A^\intercal = \begin {bmatrix} 1 \\ 2 \\ 3 \end {bmatrix} \] Notice that the transpose of transpose of \(A\) is simply \(A\). \[ (A^\intercal )^\intercal = \begin {bmatrix} 1 & 2 & 3 \end {bmatrix} = A \] Before formally stating our observations, here are few definitions to note.

Definition 0.1.2

A positive integral powers of \(A\), or multiplying the matrix \(n\) times can be represented as \(A^n\). \(A^0\) is defined to be the identity matrix \(I_n\) with corresponding order \(n\). A square matrix is nilpotent if there exists \(n \in \mathbb {Z}\) such that \(A^n = 0\).

Other definitions from transpose is symmetry and skew-symmetry.

Definition 0.1.3

For matrix \(A\) and its transpose \(A^\intercal \), matrix \(A\) is symmetric if \(A^\intercal = A\) and skew-symmetric if \(A^\intercal = -A\).

An example of a symmetric matrix is the identity matrix. For more general example, matrix \(A\) and \(B\) below are symmetric and skew-symmetric respectively. \[ A = \begin {bmatrix} a & b & c \\ b & d & e \\ c & e & f \end {bmatrix}, \quad B = \begin {bmatrix} 0 & a & b \\ -a & 0 & c \\ -b & -c & 0 \end {bmatrix} \] Notice that all symmetric and skew-symmetric matrices are square matrices. Moreover, the diagonal elements must be zero for a square matrix to be skew-symmetric.

Now let’s discuss the properties of transpose of a matrix.

Theorem 0.1.4

For matrix \(A\), \(B\), scalar \(\lambda \) and positive integer \(n\), the following equations are true assuming that the addition and multiplication conditions are satisfied.

1.: \((A^\intercal )^\intercal = A\)
2.: \((A + B)^\intercal = A^\intercal + B^\intercal \)
3.: \((\lambda A)^\intercal = \lambda A^\intercal \)
4.: \((AB)^\intercal = B^\intercal A^\intercal \)
5.: \((A^n)^\intercal = (A^\intercal )^n\)

There are lots of properties to prove, but let’s do them one by one. Below is the proof for the first property.

Proof.

Let \(A = [a_{ij}]_{m \times n}\). By definition, its transpose \(A^\intercal \coloneqq [a_{ji}]_{n \times m}\). Continuing, \(\left ( [a_{ji}]_{n \times m} \right )^\intercal = [a_{ij}]_{m \times n} = A\). Therefore, \((A^\intercal )^\intercal = A\).

Below is the proof for the second property.

Proof.

Let \(A = [a_{ij}]_{m \times n}\) and \(B = [b_{ij}]_{m \times n}\). Consider the following equation. \begin{align*} (A + B)^\intercal &= \left ( [a_{ij}]_{m \times n} + [b_{ij}]_{m \times n} \right )^\intercal = \left ( [a_{ij} + b_{ij}]_{m \times n} \right )^\intercal \\ &= [a_{ji} + b_{ji}]_{n \times m} = [a_{ji}]_{n \times m} + [b_{ji}]_{n \times m} \\ &= A^\intercal + B^\intercal \end{align*}

Thus, the property holds.

Let’s continue our proof for \((\lambda A)^\intercal = \lambda A^\intercal \).

Proof.

Let \(A = [a_{ij}]_{m \times n}\). Because \((\lambda A)^\intercal = [\lambda a_{ij}]_{m \times n}^\intercal = [\lambda a_{ji}]_{n \times m} = \lambda [a_{ji}]_{n \times m} = \lambda A^\intercal \), the property holds.

We have two more left! Here is the proof for the fourth property.

Proof.

Let \(A = [a_{ij}]_{m \times n}\) and \(B = [b_{ij}]_{n \times p}\). The following equation is satisfied by definitions of matrix multiplication and transpose. \begin{align*} (AB)^\intercal &= \left ( [a_{ij}]_{m \times n} [b_{ij}]_{n \times p} \right )^\intercal = \left ( \left [ \sum _{k = 1}^{n} a_{ik} b_{kj} \right ]_{m \times p} \right )^\intercal \\ &= \left [ \sum _{k = 1}^{n} a_{jk} b_{ki} \right ]_{p \times m} \end{align*}

Continuing from the right hand side, the following equation is obtained. \begin{align*} B^\intercal A^\intercal &= \left [ b_{ij}^\intercal \right ]_{p \times n} \left [ a_{ij}^\intercal \right ]_{n \times m} = \left [ \sum _{k = 1}^{n} b_{ik}^\intercal a_{kj}^\intercal \right ]_{p \times m} \\ &= \left [ \sum _{k = 1}^{n} b_{ki} a_{jk} \right ]_{p \times m} = \left [ \sum _{k = 1}^{n} a_{jk} b_{ki} \right ]_{p \times m} \end{align*}

Because both the left-hand side and the right-hand side of the equation lead to the same matrix, the property holds.

Finally, below is the proof for the corollary of the fourth property.

Proof.

First, notice that by the fourth property, the following equation holds. \[ \left ( A^2 \right )^\intercal = A^\intercal A^\intercal = \left ( A^\intercal \right )^2 \] Therefore, it suffices to show that \((A^n)^\intercal = (A^\intercal )^n\) holds for \(n = k + 1\) if it is true for \(n = k\). Assume \((A^k)^\intercal = (A^\intercal )^k\). By the fourth property, the following equation is obtained. \[ \left ( A^{k+1} \right )^\intercal = (A^k A)^\intercal = A^\intercal (A^k)^\intercal = A^\intercal (A^\intercal )^k = (A^\intercal )^{k + 1} \] By induction, the property holds for \(k \geq 2\). For \(k = 1\), it is self evident that the property holds true. Therefore, the property holds for all positive integers \(n\).

For the next part of the section, we will discuss partitions.

0.1.2 Partitions

As always, let’s start with key definitions.

Definition 0.1.5

As the name suggests a submatrix is a matrix of \(A\) that is obtained by omitting certain rows or columns.

Consider matrix \(A\) defined as the following. \[ A = \begin {bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end {bmatrix} \] Below are a few example of submatrices of \(A\). \begin{align*} A = \begin {bmatrix} \tikzmarknode {a11}{1} & \tikzmarknode {a12}{2} & \tikzmarknode {a13}{3} \\ \tikzmarknode {a21}{4} & \tikzmarknode {a22}{5} & \tikzmarknode {a23}{6} \\ \tikzmarknode {a31}{7} & \tikzmarknode {a32}{8} & \tikzmarknode {a33}{9} \end {bmatrix} \begin {tikzpicture}[ overlay, remember picture, arrow/.style = {-, very thick, opacity=0.7} ] \draw [arrow, Red] (a21.west) -- (a23.east); \end {tikzpicture} &\rightarrow \begin {bmatrix} 1 & 2 & 3 \\ 7 & 8 & 9 \end {bmatrix} \\ A = \begin {bmatrix} \tikzmarknode {a11}{1} & \tikzmarknode {a12}{2} & \tikzmarknode {a13}{3} \\ \tikzmarknode {a21}{4} & \tikzmarknode {a22}{5} & \tikzmarknode {a23}{6} \\ \tikzmarknode {a31}{7} & \tikzmarknode {a32}{8} & \tikzmarknode {a33}{9} \end {bmatrix} \begin {tikzpicture}[ overlay, remember picture, arrow/.style = {-, very thick, opacity=0.7} ] \draw [arrow, Red] (a11.west) -- (a13.east); \draw [arrow, Red] (a31.west) -- (a33.east); \draw [arrow, Red] (a12.north) -- (a32.south); \end {tikzpicture} &\rightarrow \begin {bmatrix} 4 & 6 \end {bmatrix} \end{align*}

Definition 0.1.6

We say the matrix is partitioned if horizontal and vertical lines are used to divide the matrix into submatrices.

Here are a few examples of different partitions of \(A\). \[ \begin {bNiceArray}{ccc}[margin] 1 & 2 & 3 \\ 4 & 5 & 6 \\ \hline 7 & 8 & 9 \end {bNiceArray}, \quad \begin {bNiceArray}{c|cc}[margin] 1 & \textcolor {Red}{2} & \textcolor {Red}{3} \\ 4 & \textcolor {Red}{5} & \textcolor {Red}{6} \\ \hline 7 & 8 & 9 \end {bNiceArray}, \quad \begin {bNiceArray}{c|c|c}[margin] 1 & 2 & 3 \\ \hline 4 & 5 & 6 \\ 7 & 8 & 9 \end {bNiceArray} \] From different partitions, we can obtain different blocks.

Definition 0.1.7

A block of a partition of \(A\) is a submatrix of \(A\) divided by the horizontal and vertical partition lines.

The highlighted portion of the second partition above is an example of a block. Now, why should we even care about partitions and blocks? Partitions of a matrix are important because it could be a very useful tool in computationally heavy matrix multiplication. Let’s take a look at a generalized example \(AB\). \[ AB = \begin {bmatrix} a_{11} & a_{12} & a_{13} & a_{14} \\ a_{21} & a_{22} & a_{23} & a_{24} \\ a_{31} & a_{32} & a_{33} & a_{34} \\ a_{41} & a_{42} & a_{43} & a_{44} \end {bmatrix} \begin {bmatrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ b_{31} & b_{32} & b_{33} \\ b_{41} & b_{42} & b_{43} \end {bmatrix} \] At least for me, this multiplication looks very painful. Fortunately, using partitions, we can turn this monster into a baby monster. \begin{align*} AB &= \begin {bNiceArray}{cc|cc}[margin] a_{11} & a_{12} & a_{13} & a_{14} \\ a_{21} & a_{22} & a_{23} & a_{24} \\ a_{31} & a_{32} & a_{33} & a_{34} \\ a_{41} & a_{42} & a_{43} & a_{44} \end {bNiceArray} \begin {bNiceArray}{ccc}[margin] b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \\ \hline b_{31} & b_{32} & b_{33} \\ b_{41} & b_{42} & b_{43} \end {bNiceArray} \\ &= \begin {bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \\ a_{41} & a_{42} \end {bmatrix} \begin {bmatrix} b_{11} & b_{12} & b_{13} \\ b_{21} & b_{22} & b_{23} \end {bmatrix} + \begin {bmatrix} a_{13} & a_{14} \\ a_{23} & a_{24} \\ a_{33} & a_{34} \\ a_{43} & a_{44} \end {bmatrix} \begin {bmatrix} b_{31} & b_{32} & b_{33} \\ b_{41} & b_{42} & b_{43} \end {bmatrix} \\ &= \begin {bmatrix} \scriptscriptstyle a_{11}b_{11} + a_{12}b_{21} & \scriptscriptstyle a_{11}b_{12} + a_{12}b_{22} & \scriptscriptstyle a_{11}b_{13} + a_{12}b_{23} \\ \scriptscriptstyle a_{21}b_{11} + a_{22}b_{21} & \scriptscriptstyle a_{21}b_{12} + a_{22}b_{22} & \scriptscriptstyle a_{21}b_{13} + a_{22}b_{23} \\ \scriptscriptstyle a_{31}b_{11} + a_{32}b_{21} & \scriptscriptstyle a_{31}b_{12} + a_{32}b_{22} & \scriptscriptstyle a_{31}b_{13} + a_{32}b_{23} \\ \scriptscriptstyle a_{41}b_{11} + a_{42}b_{21} & \scriptscriptstyle a_{41}b_{12} + a_{42}b_{22} & \scriptscriptstyle a_{41}b_{13} + a_{42}b_{23} \end {bmatrix} \\ &\qquad \qquad \qquad \qquad + \begin {bmatrix} \scriptscriptstyle a_{13}b_{31} + a_{14}b_{41} & \scriptscriptstyle a_{13}b_{32} + a_{14}b_{42} & \scriptscriptstyle a_{13}b_{33} + a_{14}b_{43} \\ \scriptscriptstyle a_{23}b_{31} + a_{24}b_{41} & \scriptscriptstyle a_{23}b_{32} + a_{24}b_{42} & \scriptscriptstyle a_{23}b_{33} + a_{24}b_{43} \\ \scriptscriptstyle a_{33}b_{31} + a_{34}b_{41} & \scriptscriptstyle a_{33}b_{32} + a_{34}b_{42} & \scriptscriptstyle a_{33}b_{33} + a_{34}b_{43} \\ \scriptscriptstyle a_{43}b_{31} + a_{44}b_{41} & \scriptscriptstyle a_{43}b_{32} + a_{44}b_{42} & \scriptscriptstyle a_{43}b_{33} + a_{44}b_{43} \end {bmatrix} \\ &= \begin {bmatrix} \scriptscriptstyle a_{11}b_{11} + a_{12}b_{21} + a_{13}b_{31} + a_{14}b_{41} & \scriptscriptstyle a_{11}b_{12} + a_{12}b_{22} + a_{13}b_{32} + a_{14}b_{42} & \scriptscriptstyle a_{11}b_{13} + a_{12}b_{23} + a_{13}b_{33} + a_{14}b_{43} \\ \scriptscriptstyle a_{21}b_{11} + a_{22}b_{21} + a_{23}b_{31} + a_{24}b_{41} & \scriptscriptstyle a_{21}b_{12} + a_{22}b_{22} + a_{23}b_{32} + a_{24}b_{42} & \scriptscriptstyle a_{21}b_{13} + a_{22}b_{23} + a_{23}b_{33} + a_{24}b_{43} \\ \scriptscriptstyle a_{31}b_{11} + a_{32}b_{21} + a_{33}b_{31} + a_{34}b_{41} & \scriptscriptstyle a_{31}b_{12} + a_{32}b_{22} + a_{33}b_{32} + a_{34}b_{42} & \scriptscriptstyle a_{31}b_{13} + a_{32}b_{23} + a_{33}b_{33} + a_{34}b_{43} \\ \scriptscriptstyle a_{41}b_{11} + a_{42}b_{21} + a_{43}b_{31} + a_{44}b_{41} & \scriptscriptstyle a_{41}b_{12} + a_{42}b_{22} + a_{43}b_{32} + a_{44}b_{42} & \scriptscriptstyle a_{41}b_{13} + a_{42}b_{23} + a_{43}b_{33} + a_{44}b_{43} \end {bmatrix} \end{align*}

Well, this looks good to me! If the matrices being multiplied have higher dimension, partitioning such matrices could help in computation and reducing mistakes. Now that we have seen partitions of matrices, we could derive special forms from the ideas.

0.1.3 Special Forms of Matrix

As always, let’s build from definitions.

Definition 0.1.8

A symmetrically partitioned matrix refers to a partitioned matrix such that its arrangements of the blocks are diagonally symmetric. The diagonal blocks are the blocks that pass through the diagonal symmetry.

We could also define matrices with diagonal blocks that are not necessarily symmetrically partitioned.

Definition 0.1.9

A block diagonal matrix is a matrix with square diagonal blocks and zero on the other entries.

Below is the general form for square matrices \(A_i\). \[ A = \begin {bmatrix} A_1 & 0 & \cdots & 0 \\ 0 & A_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & 0 \\ 0 & 0 & 0 & A_n \\ \end {bmatrix} \] Continuing, we could define rows and columns based on its elements.

Definition 0.1.10

Zero rows and zero columns are rows and columns composed of zeros respectively.

For instance for matrix defined as the following, \[ \begin {bmatrix} 1 & 2 & 0 & 3 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \end {bmatrix} \] the zero row is the second row and the zero column is the third column.

This definition is quite straightforward, and now that we have basic terms in mind, let’s discuss two forms from the definitions above.

Definition 0.1.11

Matrix in row echelon form (REF) satisfies the conditions below.

All nonzero leading entry is \(1\). (Side note here is that some textbooks and lectures do not require the leading entry to be \(1\), but I actually would like to keep it this way as this is how I initially learned it.)
All nonzero leading entries in a nonzero row in a later row are to the right of nonzero leading entries in previous rows.
If there are both nonzero and zero rows, all zero rows must appear after the nonzero rows above.

The leading entry of matrices in REF is known as a pivot. Similarly, matrices in reduced row echelon form (RREF) are matrices that satisfy the following conditions.

All nonzero leading entries in a nonzero row is \(1\).
If there are both nonzero and zero rows, all zero rows must appear after the nonzero rows above.
All leading \(1\)s in the succeeding rows appear in a column after the columns with previous leading \(1\)s.
The leading \(1\) must be the only nonzero element present in each column if there exists such \(1\).

Below are examples of a matrices in REF and RREF respectively. \[ \begin {bmatrix} 1 & 2 & 3 & 4 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end {bmatrix}, \quad \begin {bmatrix} 1 & 0 & 2 & 0 \\ 0 & 1 & 2 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 \end {bmatrix} \] Matrices that follow these forms will be very useful when solving linear systems of equations, but we will save that for the latter notes. Now let’s discuss about triangular matrices.

Triangular matrices are a special type of square matrices that is determined by elements below the main diagonal. If all its elements below the main diagonal are zeros, we call it an upper triangular matrix. Similarly, if all elements above the main diagonal are zeros, we call them lower triangular matrices. For more formal statement, consider the following definition.

Definition 0.1.12

A square matrix \(A = [a_{ij}]\) is upper triangular if \(a_{ij} = 0 \, \forall \, i > j\). Similarly, the matrix is lower triangular if \(a_{ij} = 0 \, \forall \, i < j\).

Let’s take a look at an example. The left and right matrices below are upper triangular and lower triangular matrices respectively. \[ \begin {bmatrix} 1 & 2 & 3 & 4 \\ 0 & 2 & 3 & 4 \\ 0 & 0 & 3 & 4 \\ 0 & 0 & 0 & 4 \end {bmatrix}, \quad \begin {bmatrix} 1 & 0 & 0 & 0 \\ 2 & 1 & 0 & 0 \\ 3 & 2 & 1 & 0 \\ 0 & 3 & 2 & 0 \end {bmatrix} \] Now that we have definitions of triangular matrices in mind, we can discuss a theorem from such matrices.

Theorem 0.1.13

The product of two lower triangular matrices is a lower triangular matrix, and the product of two upper triangular matrices is an upper triangular matrix.

Proof.

Let \(A = [a_{ij}]_{n \times n}\) and \(B = [b_{ij}]_{n \times n}\) be lower triangular matrices. Consider the following equation for their product matrix \(C = [c_{ij}]_{n \times n}\). \[ C = \left [ \sum _{k = 1}^{n} a_{ik} b_{kj} \right ] = \left [ \sum _{k = 1}^{j-1} a_{ik} b_{kj} + \sum _{k = j}^{n} a_{ik} b_{kj} \right ] \] From the expression, it suffices to show that \(\sum _{k = 1}^{j-1} a_{ik} b_{kj} + \sum _{k = j}^{n} a_{ik} b_{kj} = 0\) for \(i < j\). Notice that by definition, \(b_{kj} = 0\) for all \(k = 1\) to \(k = j-1\). Similarly, \(a_{ik} = 0\) for all integer \(k \in [j, n]\). Therefore, \(c_{ij} = 0\) for all \(i < j\), proving the first half of the theorem.

Similarly, let \(A' = [a'_{ij}]_{n \times n}\) and \(B' = [b'_{ij}]_{n \times n}\) be upper triangular matrices. The product matrix \(C'= [c'_{ij}]_{n \times n}\) can be represented as following. \[ C' = \left [ \sum _{k = 1}^{n} a'_{ik} b'_{kj} \right ] = \left [ \sum _{k = 1}^{j-1} a'_{ik} b'_{kj} + \sum _{k = j}^{n} a'_{ik} b'_{kj} \right ] \] By definition, it is evident that \(a'_{ik} = 0\) for integer \(k \in [1, j-1]\) and \(b'_{kj} = 0\) for integer \(k \in [j, n]\). Therefore, for \(i > j\), \(c'_{ij} = 0\) and the theorem is true.

This is it for different topics on fundamentals of matrices! I know this note is quite long, but let’s conclude our notes with the discussion on determinants.