Editing Singular value decomposition (section)

===  Based on variational characterization ===
{{anchor|vch}}The singular values can also be characterized as the maxima of {{tmath|\mathbf u^\mathrm{T} \mathbf M \mathbf v,}} considered as a function of {{tmath|\mathbf u}} and {{tmath|\mathbf v,}} over particular subspaces. The singular vectors are the values of {{tmath|\mathbf u}} and {{tmath|\mathbf v}} where these maxima are attained.

Let {{tmath|\mathbf M}} denote an {{tmath|m \times n}} matrix with real entries. Let {{tmath|S^{k-1} }} be the unit <math>(k-1)</math>-sphere in <math> \mathbb{R}^k </math>, and define <math>\sigma(\mathbf{u}, \mathbf{v}) = \mathbf{u}^\operatorname{T} \mathbf{M} \mathbf{v},</math> <math>\mathbf{u} \in S^{m-1},</math> <math>\mathbf{v} \in S^{n-1}.</math>

Consider the function {{tmath|\sigma}} restricted to {{tmath|S^{m-1} \times S^{n-1}.}} Since both {{tmath|S^{m-1} }} and {{tmath|S^{n-1} }} are [[compact space|compact]] sets, their [[Product topology|product]] is also compact.  Furthermore, since {{tmath|\sigma}} is continuous, it attains a largest value for at least one pair of vectors {{tmath|\mathbf u}} in {{tmath|S^{m-1} }} and {{tmath|\mathbf v}} in {{tmath|S^{n-1}.}} This largest value is denoted {{tmath|\sigma_1}} and the corresponding vectors are denoted {{tmath|\mathbf u_1}} and {{tmath|\mathbf v_1.}} Since {{tmath|\sigma_1}} is the largest value of {{tmath|\sigma(\mathbf u, \mathbf v)}} it must be non-negative. If it were negative, changing the sign of either {{tmath|\mathbf u_1}} or {{tmath|\mathbf v_1}} would make it positive and therefore larger.

'''Statement.''' {{tmath|\mathbf u_1}} and {{tmath|\mathbf v_1}} are left and right-singular vectors of {{tmath|\mathbf M}} with corresponding singular value {{tmath|\sigma_1.}}

'''Proof.''' Similar to the eigenvalues case, by assumption the two vectors satisfy the Lagrange multiplier equation:

<math display=block>
\nabla \sigma
= \nabla \mathbf{u}^\operatorname{T} \mathbf{M} \mathbf{v}
  - \lambda_1 \cdot \nabla \mathbf{u}^\operatorname{T} \mathbf{u}
  - \lambda_2 \cdot \nabla \mathbf{v}^\operatorname{T} \mathbf{v}
</math>

After some algebra, this becomes

<math display=block> \begin{align}
\mathbf{M} \mathbf{v}_1 &= 2 \lambda_1 \mathbf{u}_1 + 0, \\
\mathbf{M}^\operatorname{T} \mathbf{u}_1 &= 0 + 2 \lambda_2 \mathbf{v}_1.
\end{align}</math>

Multiplying the first equation from left by {{tmath|\mathbf u_1^\textrm{T} }} and the second equation from left by {{tmath|\mathbf v_1^\textrm{T} }} and taking <math> \| \mathbf u \| = \| \mathbf v \| = 1</math> into account gives

<math display=block>
\sigma_1 = 2\lambda_1 = 2\lambda_2.
</math>

Plugging this into the pair of equations above, we have

<math display=block>\begin{align}
\mathbf{M} \mathbf{v}_1 &= \sigma_1 \mathbf{u}_1, \\
\mathbf{M}^\operatorname{T} \mathbf{u}_1 &= \sigma_1 \mathbf{v}_1.
\end{align}</math>

This proves the statement.

More singular vectors and singular values can be found by maximizing {{tmath|\sigma(\mathbf u, \mathbf v)}} over normalized {{tmath|\mathbf u}} and {{tmath|\mathbf v}} which are orthogonal to {{tmath|\mathbf u_1}} and {{tmath|\mathbf v_1,}} respectively.

The passage from real to complex is similar to the eigenvalue case.