Matrices

Matrices
Matrices are pretty simple. They're just arrays of column vectors. For example, the matrix $$\begin{bmatrix}1 & 2 & 3\\4 & 5 & 6\end{bmatrix}$$ is a $2\times 3$ matrix. The first row is $\begin{bmatrix}1 & 2 & 3\end{bmatrix}$, and the second row is $\begin{bmatrix}4 & 5 & 6\end{bmatrix}$. The first column is $\begin{bmatrix}1 \\ 4\end{bmatrix}$, the second column is $\begin{bmatrix}2 \\ 5\end{bmatrix}$, and the third column is $\begin{bmatrix}3 \\ 6\end{bmatrix}$.

Matrix Equivalency
Two matrices $A$ and $B$ are equivalent if there exist two real numbers $n$ and $p$ such that the following conditions hold:

1. $\text{(Number of rows of }A\text{)}=n=\text{(Number of rows of }B\text{)}$
2. $\text{(Number of columns of }A\text{)}=p=\text{(Number of columns of }B\text{)}$
3. Corresponding indices of $A$ and $B$ are equivalent.

Matrix Addition
Matrix addition is defined elementwise — add corresponding elements. Matrix addition is commutative ($A+B=B+A$) and associative ($(A+B)+C=A+(B+C)$). Undefined if the orders of $A$ and $B$ are not the same.

Scalar Multiplication
For a matrix $A$ and real number $\lambda$, $\lambda A$ is defined by $(\lambda A)_{i, j}=\lambda A_{i,j}$.

1. $\lambda(A+B)=\lambda A+\lambda B$.
2. $(\lambda_1 + \lambda_2)A=\lambda_1A + \lambda_2 A$.
3. $(\lambda_1\lambda_2)A=\lambda_1(\lambda_2A)$.

Matrix Multiplication
Let $p, r$ and $n$ be positive integers, $A$ be a matrix with order $p\times r$, and $B$ be a matrix with order $r\times n$. The product matrix, $AB$, is the $p\times n$ matrix $C[c_{i,j}]$ defined by $$c_{ij}=\sum_{k=1}^ra_{ik}b_{kj},$$ For $i=1,2,\dots,p$ and $j=1,2,\dots n$. This means that $c_{i,I}$ is obtained by multiplying the $i$th row of $A$ with the $j$th row of $B$ and summing. The multiplicative identity matrix is the diagonal matrix with only $1$ as its diagonal elements (and is denoted by $I_n$).

If $B$ and $C$ are matrices such that $BC$ is defined, then $\lambda(BC)=B(\lambda C)$. Here's a gif:

Special Types of Matrices

There are some special types of matrices that we'll need to know about.

Identity Matrix
The identity matrix is a square matrix with $1$'s on the main diagonal and $0$'s elsewhere. It is denoted by $I_n$ (where $n$ is the order of the matrix). The identity matrix is the multiplicative identity for matrices: $AI_n=I_nA=A$.

Zero Matrix
The unique zero matrix of order $p\times n$ is a matrix (of order $p\times n$) whose entries are all zero. It is denoted with $\textbf{0}_{p\times n}$. If $A$ and $\textbf{0}_{p\times n}$ have the same order, then $A+\textbf{0}_{p\times n}=A$.

Triangular Matrices

1. Upper Triangular: A square matrix is upper triangular if $a_{i,j}=0$ whenever $i > j$.
2. Lower Triangular: A square matrix is lower triangular if $a_{i,j}=0$ whenever $i < j$.
   An upper triangular matrix has elements that are all $0$ below the main diagonal, and a lower triangular matrix has zero elements above the main diagonal. The product of a UT or LT matrix with another UT or LT (must be the same) matrix is also UT or LT (must be the same). Here's an example of an upper triangular matrix: $$\begin{bmatrix}1 & 2 & 3\\0 & 4 & 5\\0 & 0 & 6\end{bmatrix}.$$

Diagonal Matrix
A diagonal matrix is a matrix where all the elements are zero except for the diagonal elements. The product of two diagonal matrices is also a diagonal matrix. So, a diagonal matrix is both UT and LT.

Sub-matrix
A sub-matrix of a matrix $A$ is any matrix obtained by removing a number of rows or columns from $A$. We'll need this when we talk about the determinant.

Partitioned Matrix
A matrix is partitioned if it is divided into submatrices by horizontal and vertical lines between rows and columns. The resulting submatrices are often called blocks. Matrices are compatibly partitioned if the number of horizontal blocks in $A=$ the number of vertical blocks in $B$ (i.e., the block dimensions are multiplicable) AND each block $\times$ block operation "makes sense" — i.e., the submatrices can be multiplied. Note that partitioning a matrix doesn't change anything about the matrix — it's just a way to organize the matrix.

Symmetric Partition
A symmetrically partitioned matrix is one where the horizontal and vertical partition lines occur in the same places relative to the sequence of rows and columns.

Diagonal Block
A diagonal block (in a symmetrically partitioned matrix) can be defined in two ways:

1. A block bounded by the $i$-th and $i+1$st horizontal AND vertical partition lines.
2. A block whose diagonal elements are all diagonal elements of $A$.
   Each diagonal block must be square. A block diagonal matrix is one whose nondiagonal blocks are all zero submatrices.

Special Matrix Definitions

Row Equivalence
Two matrices are row equivalent if one can be changed to the other by a sequence of elementary row operations.

Positive Integral Power
$A^n$ (for any positive integer $n$) is called a "positive integral power of $A$."

Transpose
The transpose of a $p\times n$ matrix $A$ is the $n\times p$ matrix $A^T$ such that $$(A^T)_{i,j}=A_{j,i}.$$ Essentially, each row of $A$ becomes a column of $A^T$. Transposes have the following properties: $$\begin{align}(A^T)^T&=A\\(\lambda A)^T&=\lambda(A^T)\\(A+B)^T&=A^T+B^T\\(AB)^T&=B^TA^T\\{\left(A^n\right)}^T&={\left(A^T\right)}^n\end{align}$$

Symmetry
A matrix is symmetric if $A^T=A$. This can be possible if and only if $A$ is square and $A_{ij}=A_{ji}$. So, the main diagonal can be anything, but the other elements must be symmetric.

Skew Symmetry
A matrix $A$ is skew symmetric if $A^T=-A$. This is true if and only if $A$ is square and $A_{ij}=-A_{ji}$. A skew symmetric matrix must have all diagonal elements equal to $0$. It is a good exercise to prove that a skew symmetric matrix $A$ squared is symmetric, and $A^3$ is skew symmetric.