Inner Product Space

A vector space that has an inner product.

Def: Inner Product

If $V$ is a vector space, an inner product on $V$ is a map $⟨ \cdot, \cdot ⟩ : V \times V \to R$ such that:

Symmetry $\forall x, y \in V$ $⟨ x, y ⟩ = ⟨ y, x ⟩$
Linearity $\forall x, y, z \in V, c \in R,$ $⟨ x + c y, z ⟩ = ⟨ x, z ⟩ + c ⟨ y, z ⟩$
Positive Definite $\forall x \in V,$ $⟨ x, x ⟩ \geq 0$ , $⟨ x, x ⟩ = 0 ⟺ x = 0$

symmetry + linearity = bilinear

interesting example $⟨ f, g ⟩ = \int_{0}^{1} f (x) g (x) d x$

Thm: Cauchy-Schwarz

If V is an inner product space then $\forall x, y \in$ V, $⟨ x, y ⟩^{2} \leq ⟨ x, x ⟩ ⟨ y, y ⟩$

pf: let $p = \frac{⟨ x, y ⟩}{⟨ x, x ⟩}$

$0 \leq ⟨ y - p x, y - p x ⟩ = ⟨ y, y - p x ⟩ + ⟨ - p x, y - p x ⟩ = ⟨ y, y ⟩ - 2 p ⟨ x, y ⟩ + p^{2} ⟨ x, x ⟩$

$= ⟨ y, y ⟩ - \frac{⟨ x, y ⟩^{2}}{⟨ x, x ⟩}$

$⟹ \frac{⟨ x, y ⟩^{2}}{⟨ x, x ⟩} \leq ⟨ y, y ⟩$ $⟹ ⟨ x, y ⟩^{2} \leq ⟨ y, y ⟩ ⟨ x, x ⟩$

Def: Orthogonal

If V is an inner-product spave and v, w $\in V$ then v and w are orthogonal if $⟨ v, w ⟩ = 0$

If every element in a set is orthogonal to each other and they are each unit vectors ( $| | v | | = 1$ ) then the set is orthonormal. see: Norm

Orthogonal matrix

A matrix is $A \in M_{n} (R)$ is orthogonal if $A^{T} A = A A^{T} = I_{n}$
if $A^{- 1} = A^{T} ⟹ det(A) = \pm 1$ and the columns of $A$ form an orthonormal basis of $R^{n}$ with the Euclidean inner product

Thm: Fourier Coefficient Theorem

Supposed $(V, ⟨ \cdot, \cdot ⟩)$ is a fdvs with an orthogonal basis $B = {b_{1, \dots,} b_{n}}$ for any $v \in V,$

v = \sum_{i = 1}^{n} \frac{⟨ v, b_{i} ⟩}{⟨ b_{i}, b_{i} ⟩} b_{i}

pf by using $⟨ v, b_{j} ⟩ = ⟨ \sum_{i = 1}^{n} c_{i} b_{i}, b_{i} ⟩$

Since $v$ is written as a unique combination of $b_{i}$ this must be that Linear Independence#Thm Unique Linear Combinations giving us coordinate representation. Each component of the sum is the projection of v on to $b_{i}$ . If $b_{i}$ is orthonormal the denominator in the sum is 1.

Corollary

If $(V, ⟨ \cdot . \cdot ⟩)$ is a fdvs and $B \subseteq V$ is an orthogonal set of non-zero vectors, then $B$ is linearly independent

Def: Perpendicular

If $V$ is an inner product space and $U \subseteq V$ is a subspace, we define:

U^{⊥} = {v \in V : ⟨ u, v ⟩ = 0, \forall u \in U}

Perpendicular Theorems

If $(V, ⟨ \cdot, \cdot ⟩)$ is an inner-product space and $U \subseteq V$ is a subspace then

$U^{⊥}$ is a subspace
If $W$ is another subspace with $U \subseteq W$ then $W^{⊥} \subseteq U^{⊥}$
$U \subseteq (U^{⊥})^{⊥}$ with equality if $V$ is finite dimensional

is trivial
$U \subseteq W$ and $z \in W^{⊥}$ . Thus $⟨ z, w ⟩ = 0$ $\forall w \in W$ and since $U \subseteq W$ then $⟨ z, u ⟩ = 0 \forall u \in U$
trivial

Lemma

if $B = {b_{1}, . ., b_{n}}$ is a basis for $U$ then $U^{⊥} = {v \in V : ⟨ v, b_{i} ⟩ = 0, \forall i = 1, \dots, n}$

Thm: subspace perpendicular decomposition

If $V$ is an finite inner product space and $U \subseteq V$ is a subspace, then $V = U \oplus U^{⊥}$ (direct sums)

pf using this theorem from projections

$v = (v - {proj}_{u} (v)) + {proj}_{u} (v)$ $⟹$ $v = U^{⊥} + U$
Law of excluded middle or $u \in U \cap U^{⊥} ⟹ ⟨ u, u ⟩ = 0 ⟹ U \cap U^{⊥} = {0}$