Editing Covariance (section)

=== Auto-covariance matrix of real random vectors ===
{{main|Auto-covariance matrix}}

For a vector <math>\mathbf{X} = \begin{bmatrix} X_1 & X_2 & \dots & X_m \end{bmatrix}^\mathrm{T}</math> of <math>m</math> jointly distributed random variables with finite second moments, its auto-covariance matrix (also known as the '''variance–covariance matrix''' or simply the '''covariance matrix''') <math>\operatorname{K}_{\mathbf{X}\mathbf{X}}</math> (also denoted by <math>\Sigma(\mathbf{X})</math> or <math>\operatorname{cov}(\mathbf{X}, \mathbf{X})</math>) is defined as<ref name=Gubner>{{cite book |first=John A. |last=Gubner |year=2006 |title=Probability and Random Processes for Electrical and Computer Engineers |publisher=Cambridge University Press |isbn=978-0-521-86470-1}}</ref>{{rp|p=335}}
<math display="block">\begin{align}
  \operatorname{K}_\mathbf{XX} = \operatorname{cov}(\mathbf{X}, \mathbf{X})
    &= \operatorname{E}\left[(\mathbf{X} - \operatorname{E}[\mathbf{X}]) (\mathbf{X} - \operatorname{E}[\mathbf{X}])^\mathrm{T}\right] \\
    &= \operatorname{E}\left[\mathbf{XX}^\mathrm{T}\right] - \operatorname{E}[\mathbf{X}]\operatorname{E}[\mathbf{X}]^\mathrm{T}.
\end{align}</math>

Let <math>\mathbf{X}</math> be a [[random vector]] with covariance matrix {{math|Σ}}, and let {{math|'''A'''}} be a matrix that can act on <math>\mathbf{X}</math> on the left. The covariance matrix of the matrix-vector product {{math|'''A X'''}} is:
<math display="block">\begin{align}
  \operatorname{cov}(\mathbf{AX},\mathbf{AX}) &=
  \operatorname{E}\left[\mathbf{AX(A}\mathbf{X)}^\mathrm{T}\right] - \operatorname{E}[\mathbf{AX}] \operatorname{E}\left[(\mathbf{A}\mathbf{X})^\mathrm{T}\right] \\
  &= \operatorname{E}\left[\mathbf{AXX}^\mathrm{T}\mathbf{A}^\mathrm{T}\right] - \operatorname{E}[\mathbf{AX}] \operatorname{E}\left[\mathbf{X}^\mathrm{T}\mathbf{A}^\mathrm{T}\right] \\
  &= \mathbf{A}\operatorname{E}\left[\mathbf{XX}^\mathrm{T}\right]\mathbf{A}^\mathrm{T} - \mathbf{A}\operatorname{E}[\mathbf{X}] \operatorname{E}\left[\mathbf{X}^\mathrm{T}\right]\mathbf{A}^\mathrm{T} \\
  &= \mathbf{A}\left(\operatorname{E}\left[\mathbf{XX}^\mathrm{T}\right] - \operatorname{E}[\mathbf{X}] \operatorname{E}\left[\mathbf{X}^\mathrm{T}\right]\right)\mathbf{A}^\mathrm{T} \\
  &= \mathbf{A}\Sigma\mathbf{A}^\mathrm{T}.
\end{align}</math>

This is a direct result of the linearity of [[expected value|expectation]] and is useful
when applying a [[linear transformation]], such as a [[whitening transformation]], to a vector.