Editing Covariance (section)

=== Relationship to inner products ===
Many of the properties of covariance can be extracted elegantly by observing that it satisfies similar properties to those of an [[inner product]]:
# [[bilinear operator|bilinear]]:  for constants <math>a</math> and <math>b</math> and random variables <math>X,Y,Z,</math> <math> \operatorname{cov}(aX+bY,Z) = a \operatorname{cov}(X,Z) + b \operatorname{cov}(Y,Z)</math>
# symmetric: <math>\operatorname{cov}(X,Y) = \operatorname{cov}(Y,X)</math>
# [[definite bilinear form|positive semi-definite]]: <math>\sigma^2(X) = \operatorname{cov}(X,X) \ge 0</math> for all random variables <math>X</math>, and <math>\operatorname{cov}(X,X) = 0</math> implies that <math>X</math> is constant [[almost surely]].

In fact these properties imply that the covariance defines an inner product over the [[quotient space (linear algebra)|quotient vector space]] obtained by taking the subspace of random variables with finite second moment and identifying any two that differ by a constant. (This identification turns the positive semi-definiteness above into positive definiteness.) That quotient vector space is isomorphic to the subspace of random variables with finite second moment and mean zero; on that subspace, the covariance is exactly the [[Lp space|L<sup>2</sup>]] inner product of real-valued functions on the sample space.

As a result, for random variables with finite variance, the inequality
<math display="block">\left|\operatorname{cov}(X, Y)\right| \le \sqrt{\sigma^2(X) \sigma^2(Y)} </math>
holds via the [[Cauchy–Schwarz inequality]].

Proof: If <math>\sigma^2(Y) = 0</math>, then it holds trivially. Otherwise, let random variable
<math display="block"> Z = X - \frac{\operatorname{cov}(X, Y)}{\sigma^2(Y)} Y.</math>

Then we have
<math display="block">\begin{align}
  0 \le \sigma^2(Z)
    &= \operatorname{cov}\left(
         X - \frac{\operatorname{cov}(X, Y)}{\sigma^2(Y)} Y,\;
         X - \frac{\operatorname{cov}(X, Y)}{\sigma^2(Y)} Y
       \right) \\[12pt]
    &= \sigma^2(X) - \frac{(\operatorname{cov}(X, Y))^2}{\sigma^2(Y)} \\
\implies (\operatorname{cov}(X, Y))^2 &\le \sigma^2(X)\sigma^2(Y) \\
\left|\operatorname{cov}(X, Y)\right| &\le \sqrt{\sigma^2(X)\sigma^2(Y)}
\end{align}</math>