Editing Outer product (section)

===Connection with the matrix product===
Given a pair of matrices <math>\mathbf{A}</math> of size <math>m\times p</math> and <math>\mathbf{B}</math> of size <math>p\times n</math>, consider the [[matrix multiplication|matrix product]] <math>\mathbf{C} = \mathbf{A}\,\mathbf{B}</math> defined as usual as a matrix of size <math>m\times n</math>.

Now let <math>\mathbf a^\text{col}_k</math> be the <math>k</math>-th column vector of <math>\mathbf A</math> and let <math>\mathbf b^\text{row}_k</math> be the <math>k</math>-th row vector of <math>\mathbf B</math>. Then <math>\mathbf{C}</math> can be expressed as a sum of column-by-row outer products:
:<math display="block">\mathbf{C} = \mathbf{A}\, \mathbf{B} =
\left(
      \sum_{k=1}^p {A}_{ik}\, {B}_{kj}
\right)_{
    \begin{matrix} 1\le i \le m \\[-20pt] 1 \le j\le n \end{matrix}
} =
\begin{bmatrix} & & \\ \mathbf a^\text{col}_{1} & \cdots & \mathbf a^\text{col}_{p} \\ & & \end{bmatrix}
\begin{bmatrix} & \mathbf b^\text{row}_{1} & \\ & \vdots & \\ & \mathbf b^\text{row}_{p} & \end{bmatrix}
= \sum_{k=1}^p \mathbf a^\text{col}_k \mathbf b^\text{row}_k</math>
This expression has duality with the more common one as a matrix built with row-by-column [[inner product space|inner product]] entries (or [[dot product]]): <math>C_{ij} = \langle{\mathbf a^\text{row}_i,\,\mathbf b_j^\text{col}}\rangle</math>

This relation is relevant<ref>{{cite book |last2=Bau III |first2=David |last1=Trefethen |first1=Lloyd N. |author1-link=Lloyd N. Trefethen |title=Numerical linear algebra |publisher=Society for Industrial and Applied Mathematics |location=Philadelphia |isbn=978-0-89871-361-9 |year=1997}}</ref> in the application of the [[singular value decomposition#Separable models|Singular Value Decomposition (SVD)]] (and [[eigendecomposition of a matrix|Spectral Decomposition]] as a special case). In particular, the decomposition can be interpreted as the sum of outer products of each left (<math>\mathbf{u}_k</math>) and right (<math>\mathbf{v}_k</math>) singular vectors, scaled by the corresponding nonzero singular value <math>\sigma_k</math>:
:<math display="block">\mathbf{A} = \mathbf{U \Sigma V^T} = \sum_{k=1}^{\operatorname{rank}(A)}(\mathbf{u}_k \otimes \mathbf{v}_k) \, \sigma_k</math>

This result implies that <math>\mathbf{A}</math> can be expressed as a sum of rank-1 matrices with [[matrix norm#Norms induced by p-norms|spectral norm]] <math>\sigma_k</math> in decreasing order. This explains the fact why, in general, the last terms contribute less, which motivates the use of the [[singular value decomposition#Truncated SVD|truncated SVD]] as an approximation. The first term is the [[least squares]] fit of a matrix to an outer product of vectors.