Eigenvectors, Values, Bases, and Spaces

Eigenvectors
Eigenvectors of a transformation are vectors that stay on their span during the transformation. There are two similar ways of defining them: through transformations and matrices. The gist is the same: a nonzero (important!) vector $v\in\mathbb{R}^n$ (matrix) or $v\in\mathbb{V}$ (transformation) is an eigenvector of $A$ (matrix) or $T:\mathbb{V}\to\mathbb{V}$ (linear transformation) if

1. $Av=\lambda v$, or
2. $Tv=\lambda v$. $\lambda$ is the eigenvector's...

Eigenvalue
An eigenvalue describes the scalar that the eigenvector is multiplied by as a result of the transformation. Why are eigenvectors important? Consider a 3d rotation — if you can find an eigenvector, you have found the axis of rotation for that transformation.

Eigenbasis
Whenever a matrix has zeroes everywhere except the diagonal, it's called a diagonal matrix. It means that every single basis vector is an eigenvector! Diagonal matrices allow you to do a lot — computations with them are very easy. But... isn't it unlikely that you'll get a diagonal matrix as your transformation? Well, funny thing — if you can find a set of eigenvectors that span space, you can change basis to those eigenvectors to get a diagonal transformation matrix!

Finding eigenthings
$\lambda$ is an eigenvalue of the $n\times n$ matrix $A$ if and only if $$\det(A-\lambda I)=0.$$ The above equation is called the characteristic equation of the matrix $A$. Solving the characteristic equation yields some number of eigenvalues $\lambda_1,\dots,\lambda_k$. We can find each eigenvalue's eigenspace by solving the equation $Ax=\lambda x$. This is equivalent to solving $$(A-\lambda I_n)x=0.$$

Theorems
There are a ton of results related to eigenvalues and eigenvectors. Here are a few. Try to prove each one, or at least understand why they're true.

Thm. If $\mathbb{V}$ is finite dimensional and $:\mathbb{V}\to\mathbb{V}$ is linear, then there exists a basis $B$ of $\mathbb{V}$ such that the representation of $T$ with respect to $B$ is a diagonal matrix if and only if there exists a basis of $\mathbb{V}$ consisting of only eigenvectors of $T$.

Thm. $u\in \mathbb{V}$ is an eigenvector with eigenvalue $\lambda$ for a given transformation $T:\mathbb{V}\to\mathbb{V}$ if and only if for any basis $C$ of $\mathbb{V}$ and matrix representation $A_C^C$ of $T$, $(u)_C$ is an eigenvector with eigenvalue $\lambda$ for $A_C^C$. So, to find all eigenvectors and eigenvalues of a transformation, it suffices to find all eigenvectors and eigenvalues of any matrix representation of that transformation.

Thm. For a linear transformation $T:\mathbb{V}\to\mathbb{V}$ with eigenvalue $\lambda$, the set $$S_\lambda=\left\{v\in\mathbb{V}\;|\; Tv=\lambda v\right\}=\left\{\textbf{0}\right\}\cup\left\{v\in\mathbb{V}\;|v\text{ is an eigenvector with eigenvalue }\lambda\right\}$$ is a subspace of $\mathbb{V}$. This set is called the eigenspace of $T$ with eigenvalue $\lambda$.

Thm. If $A$ and $B$ are similar matrices, then they have the same characteristic equation, and, as such, the same eigenvalues.

Thm. For an upper or lower triangular matrix, the eigenvalues are the diagonal elements.

Thm. $A$ is singular if and only if it has a zero eigenvalue (obviously: $\det(A-0 I_n)=0$).

Thm. If $x$ is an eigenvector of an invertible $n\times n$ matrix $A$ with eigenvalue $\lambda$, then $x$ is an eigenvector of $A^{-1}$ with eigenvalue $1/\lambda$.

Thm. If $x$ is an eigenvector of $A$ with eigenvalue $\lambda$, then

1. $x$ is an eigenvector of $kA$ with eigenvalue $k\lambda$ for some scalar $k$.
2. $x$ is an eigenvector of $A^n$ with eigenvalue $\lambda^n$ for each positive integer $n$.

Trace
The trace of an $n\times n$ matrix $A=[a_{ij}]$ is $$tr(A)=\sum_{i=1}^n a_{ii},$$ which is just the sum of the diagonal elements.

Thm. Suppose $A$ is an $n\times n$ matrix with eigenvalues $\lambda_1, \dots, \lambda_n$ (possibly complex), listed with multiplicity. Then

1. $tr(A)=\lambda_1 + \lambda_2 + \dots + \lambda_n$.
2. $\det(A)=\lambda_1\lambda_2\dots\lambda_n$.

Quick Trick for Finding $2\times2$ Eigenvalues
There are a few things to know.

1. $\text{tr}\left(\begin{bmatrix}a & b\\c & d\end{bmatrix}\right)=a+d=\lambda_1+\lambda_2$, so the average of these two diagonal entries is the same as the average of the two eigenvalues.
2. $\det\left(\begin{bmatrix}a & b \\ c & d\end{bmatrix}\right)=ad-bc=\lambda_1\lambda_2$.
3. $\lambda_1, \lambda_2 = m\pm \sqrt{m^2-p}$, where $m$ is the mean of the diagonals and $p$ is the determinant of the matrix.

This is from a 3Blue1Brown video, and it's a really cool trick! It's a good way to check your work, and it's a good way to find eigenvalues quickly.