Vector Spaces

What are vectors?
They're arrows in space, right? This is what most people are taught in high school-level mathematics.

But. There are such things as...

Vector-ish Thingies
If you think about it, functions are just like vectors, no? You can add them. You can scale them. In essence, functions are just vectors with infinitely many coordinates, where the base is $x^\infty, \dots, x^2, x, 1$ instead of $\begin{bmatrix}1 & 0 & \dots\end{bmatrix}^T,\begin{bmatrix}0 & 1 & \dots \end{bmatrix}^T$. With this basis (for polynomials of degree $n$ or lower, the basis is called the standard basis of $\mathbb{P}^n$), you can represent any polynomial as a linear combination of the basis vectors. For example, the polynomial $3x^2+2x+1$ can be represented as $\begin{bmatrix}3 & 2 & 1\end{bmatrix}^T$. Back to the question — what are vectors? The answer is, anything that follows the rules. Linear algebra is defined by 10 axioms, and all of the results that can be called "linear algebra" follow as a result of these 10 axioms. So, it doesn't matter what a "vector space" is — so long as it follows the axioms, the findings hold true.

Formal Definition of a Vector Space
A vector space is a set $V$ of objects (vectors) and a set of scalars (either $\mathbb{R}$ or $\mathbb{C}$), together with two binary operations, vector addition $\oplus$ and scalar multiplication $\odot$, that satisfy the following properties:

• Addition:
  • (A1) Closure: For all $u,v\in \mathbb{V},$ $u\oplus v\in \mathbb{V}$.
  • (A2) Commutativity: For all $u,v\in \mathbb{V}$, $u\oplus v=v\oplus u$.
  • (A3) Associativity: For all $u,v,w\in \mathbb{V}$, $(u\oplus v)\oplus w=u\oplus (v\oplus w)$.
  • (A4) Existence of an identity: There exists a zero vector in $\mathbb{V}$ (denoted $\textbf{0}$) such that for every $v\in \mathbb{V}$, $v\oplus \textbf{0}=v$.
  • (A5) Existence of inverse: There exists for every $v\in \mathbb{V}$ a $-v$ such that $v\oplus (-v)=0$.
• Multiplication:
  • (S1) Closure: For all $u\in \mathbb{V}$ and all scalars $\alpha$, $\alpha\odot u\in \mathbb{V}$.
  • (S2) Associativity: For all $u\in \mathbb{V}$ and scalars $\alpha, \beta$, $\alpha \odot(\beta\odot u)=(\alpha\cdot\beta)\odot u$.
  • (S3) Non-scaling property: $1$ is an identity element. For all vectors $u\in \mathbb{V}$, $1\odot u=u$.
  • (S4) Distributivity (1): For all $u\in \mathbb{V}$ and scalars $\alpha$ and $\beta$, $(\alpha + \beta)\odot u=\alpha\odot u \oplus \beta\odot u$. Note that "$+$" refers to standard addition of real and complex numbers.
  • (S5) Distributivity (2): For all $u, v\in \mathbb{V}$ and scalars $\alpha$, $\alpha\odot(u\oplus v)=\alpha\odot u \oplus \alpha\odot v$.

If $\mathbb{R}$ is the set of scalars for a vector space, that vector space is said to be a real vector space. If the set of scalars is $\mathbb{C}$, the vector space is said to be a complex vector space, or a vector space over $\mathbb{C}$. Any vector space defined over $\mathbb{C}$ with dimension $n$ can be viewed as a set over $\mathbb{R}$ with dimension $2n$.

Subspaces
A subspace of $\mathbb{R}^n$ is some subset $V$ of $\mathbb{R}^n$ that satisfies the following properties:

1. Identity: $\vec{\textbf{0}}\in \mathbb{V}$.
2. Closure under multiplication: $\left\{c\cdot\vec{x}\in \mathbb{V}\; \forall\; c\in \mathbb{R},\,\vec{x}\in \mathbb{V}\right\}$.
3. Closure under addition: $\left\{\vec{a}+\vec{b}\in \mathbb{V} \;\forall\; \vec{a}, \vec{b}\in \mathbb{V} \right\}$.
4. Non-emptiness: $\mathbb{V}\neq \emptyset$.

These 4 properties can be simplified to one, closure under linear combination: $$\text{For all } \vec{v}, \vec{w}\in \mathbb{V} \text{ and scalars } c, d, c\vec{v}+d\vec{w}\in \mathbb{V}.$$ As it implies the 4 hold true. We'll see subspaces used more when we talk about eigenvectors and eigenvalues.

Properties of Vector Spaces

• In any vector space, there is a unique zero vector.
• Additive inverses are unique.
• For any $v\in \mathbb{V}$, if $v\oplus v=v$, $v=\textbf{0}$.
• For any $u\in \mathbb{V}$, $u\odot \textbf{0}=\textbf{0}$.
• For any scalar $\alpha$, $\alpha\odot \textbf{0}=\textbf{0}$.
• For any $v\in \mathbb{V}$, $(-1)\odot v=-v$.
• For any $v\in \mathbb{V}$, $v=-(-v)$.
• For any $v\in \mathbb{V}$ and scalar $\alpha$, if $a\odot v=0$, either $a=0$ or $v=\textbf{0}$.

Spanning Set
A set $S$ is a spanning set of $\mathbb{V}$ if $span(S)=\mathbb{V}$. This means that "$S$ spans $\mathbb{V}$." If $S\subset \mathbb{V}$, then:

1. If $span(S)=\mathbb{V}$, then some subset of $S$ is a basis for $\mathbb{V}$.
2. If $S$ is linearly independent, then $S$ is a subset of some basis for $\mathbb{V}$.

Lemmas:

1. If $\mathbb{V}$ is a vector space and $S\subset T\subset \mathbb{V}$, then $span(S)\subset span(T)$.
2. If $\mathbb{V}$ is a vector space and $S\subset \mathbb{V}$, then $span(span(S))=span(S)$.
3. If $\mathbb{V}$ is a vector space and $S,T\subset \mathbb{V}$ such that $S\subset T$, then $span(S)\subset span(T)$.
4. If $v_1,\dots, v_n$ are vectors in vector space $\mathbb{V}$, then
   1. $span\{v_1, \dots, v_k, \dots, v_n\}=span\{v_1,\dots, v_k+\lambda v_j, \dots, v_n\}$
   2. $span\{v_1, \dots, v_k, \dots, v_n\}=span\{v_1,\lambda v_k, \dots, v_n\}$ (where $\lambda\in\mathbb{R}$ and $\lambda \neq 0$).

Basis
A basis for $\mathbb{V}$ is a linearly independent spanning set of $V$. Basically a "minimal" spanning set. A vector space is finitely dimensional if it has a basis with finitely many elements.

Thm. If the set $S=\left\{ v_1,v_2,\dots,v_n \right\}$ spans $\mathbb{V}$, then any set $T$ with more than $n$ elements must be linearly dependent. From this it follows that if $B=\left\{v_1, v_2,\dots, v_n \right\}$ is a basis for $\mathbb{V}$, then every linearly dependent subset of $\mathbb{V}$ must have $n$ or fewer elements. Also, every basis for $\mathbb{V}$ has the same number of elements. The number of elements in a basis for $\mathbb{V}$ is called the dimension of $\mathbb{V}$.

Bases are used to express any vector in the vector space. For any vector $\vec{u}\in \mathbb{V}$, there is only one way to express $\vec{u}$ as a linear combination of the basis vectors. The coefficients used in this linear combination are called the coordinates, and can be written as $n$-tuples or column matrices. We'll use this a lot later when we talk about transformations and change of basis operations.

I wanted to talk a bit more about why we might want to use polynomials as vectors (or anything, for that matter). The truth is, linear algebra is a tool — a very powerful one. Consider taking the derivative of a polynomial. There is no "strictly math" way to implement it, in, say, Python. But if we consider the polynomial as a vector, we can take the derivative of the polynomial by simple matrix multiplication — which is easy-peasy in Python. Then, the derivative could be defined as $$A=\begin{bmatrix}0 & 1 & 0 & 0 & 0&\dots\\0 & 0 & 2 & 0 & 0&\dots\\0 & 0 & 0 & 3 & 0&\dots\\0 & 0 & 0 & 0 & 4 & \dots\\\vdots & \vdots & \vdots & \vdots & \vdots & \ddots\end{bmatrix},$$ with respect to the basis $\left\{1, x, x^2, x^3, \dots\right\}$. So, the derivative of the polynomial $p(x)=1+2x+3x^2$ is just $Ap(x)=\begin{bmatrix}0&2&6&0&\dots\end{bmatrix}^T=2+6x$. This is a very powerful tool, and it's why we use linear algebra. It's not just about vectors and matrices — it's about the power of abstraction and the ability to manipulate things in ways that we couldn't before.

If you're not sure how we came up with $A$, you'll see it later when we talk about transformations. For now, just know that it's a way to represent the derivative of a polynomial as a matrix.