Editing Covariance (section)

== Definition ==
For two [[Joint distribution|jointly distributed]] [[real number|real]]-valued [[random variable]]s <math>X</math> and <math>Y</math> with finite [[second moment]]s, the covariance is defined as the [[expected value]] (or mean) of the product of their deviations from their individual expected values:<ref>Oxford Dictionary of Statistics, Oxford University Press, 2002, p. 104.</ref><ref name=KunIlPark>{{cite book | author=Park, Kun Il| title=Fundamentals of Probability and Stochastic Processes with Applications to Communications| publisher=Springer | year=2018 | isbn=9783319680743 }}</ref>{{rp|p = 119}}

<math display=block>\operatorname{cov}(X, Y) = \operatorname{E}{\big[(X - \operatorname{E}[X])(Y - \operatorname{E}[Y])\big]}</math>

where <math>\operatorname{E}[X]</math> is the expected value of <math>X</math>, also known as the mean of <math>X</math>. The covariance is also sometimes denoted <math>\sigma_{XY}</math> or <math>\sigma(X,Y)</math>, in analogy to [[variance]].  By using the linearity property of expectations, this can be simplified to the expected value of their product minus the product of their expected values:
<math display="block">
\begin{align}
\operatorname{cov}(X, Y)
&= \operatorname{E}\left[\left(X - \operatorname{E}\left[X\right]\right) \left(Y - \operatorname{E}\left[Y\right]\right)\right] \\
&= \operatorname{E}\left[X Y - X \operatorname{E}\left[Y\right] - \operatorname{E}\left[X\right] Y + \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right]\right] \\
&= \operatorname{E}\left[X Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] + \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right] \\
&= \operatorname{E}\left[X Y\right] - \operatorname{E}\left[X\right] \operatorname{E}\left[Y\right].
\end{align}
</math>
This identity is useful for mathematical derivations. From the viewpoint of numerical computation, however, it is susceptible to [[catastrophic cancellation]] (see the section on [[#Numerical computation|numerical computation]] below).

The [[unit of measurement|units of measurement]] of the covariance <math>\operatorname{cov}(X, Y)</math> are those of <math>X</math> times those of <math>Y</math>. By contrast, [[correlation|correlation coefficients]], which depend on the covariance, are a [[dimensionless number|dimensionless]] measure of linear dependence. (In fact, correlation coefficients can simply be understood as a normalized version of covariance.)

===Complex random variables===
{{main|Complex random variable#Covariance}}

The covariance between two [[complex random variable]]s <math>Z, W</math> is defined as<ref name=KunIlPark/>{{rp|p= 119}}
<math display="block">\operatorname{cov}(Z, W) =
  \operatorname{E}\left[(Z - \operatorname{E}[Z])\overline{(W - \operatorname{E}[W])}\right] =
  \operatorname{E}\left[Z\overline{W}\right] - \operatorname{E}[Z]\operatorname{E}\left[\overline{W}\right]
</math>

Notice the complex conjugation of the second factor in the definition.

A related ''[[pseudo-covariance]]'' can also be defined.

===Discrete random variables===
If the (real) random variable pair <math>(X,Y)</math> can take on the values <math>(x_i,y_i)</math> for <math>i = 1,\ldots,n</math>, with equal probabilities <math>p_i=1/n</math>, then the covariance can be equivalently written in terms of the means <math>\operatorname{E}[X]</math> and <math>\operatorname{E}[Y]</math> as
<math display="block">\operatorname{cov} (X,Y) = \frac{1}{n}\sum_{i=1}^n (x_i-E(X)) (y_i-E(Y)).</math>

It can also be equivalently expressed, without directly referring to the means, as<ref>{{cite conference |author=Yuli Zhang |author2=Huaiyu Wu |author3=Lei Cheng |title=Some new deformation formulas about variance and covariance|book-title=Proceedings of 4th International Conference on Modelling, Identification and Control(ICMIC2012)|date=June 2012 |pages=987–992}}</ref>
<math display="block"> \operatorname{cov}(X,Y) = \frac{1}{n^2} \sum_{i=1}^n \sum_{j=1}^n \frac{1}{2}(x_i - x_j)(y_i - y_j) = \frac{1}{n^2} \sum_i \sum_{j>i} (x_i-x_j)(y_i - y_j). </math>

More generally, if there are <math>n</math> possible realizations of <math>(X,Y)</math>, namely <math>(x_i,y_i)</math> but with possibly unequal probabilities <math>p_i </math> for <math>i = 1,\ldots,n</math>, then the covariance is
<math display="block">\operatorname{cov} (X,Y) = \sum_{i=1}^n p_i (x_i-E(X)) (y_i-E(Y)).</math>

In the case where two discrete random variables <math>X</math> and <math>Y</math> have a joint probability distribution, represented by elements <math>p_{i,j}</math> corresponding to the joint probabilities of <math>P( X = x_i, Y = y_j )</math>, the covariance is calculated using a double summation over the indices of the matrix:

<math display="block">\operatorname{cov} (X, Y) = \sum_{i=1}^{n}\sum_{j=1}^{n} p_{i,j} (x_i - E[X])(y_j - E[Y]).</math>