Editing Correlation (section)

==Pearson's product-moment coefficient==
{{Main|Pearson product-moment correlation coefficient}}
[[File:Pearson Correlation Coefficient and associated scatterplots.png|thumb|Example scatterplots of various datasets with various correlation coefficients]]

The most familiar measure of dependence between two quantities is the [[Pearson product-moment correlation coefficient]] (PPMCC), or "Pearson's correlation coefficient", commonly called simply "the correlation coefficient". It is obtained by taking the ratio of the covariance of the two variables in question of our numerical dataset, normalized to the square root of their variances.   Mathematically, one simply divides the [[covariance]] of the two variables by the product of their [[standard deviation]]s. [[Karl Pearson]] developed the coefficient from a similar but slightly different idea by [[Francis Galton]].<ref name="thirteenways">{{cite journal | last1 = Rodgers | first1 = J. L. | last2 = Nicewander | first2 = W. A. | year = 1988 | title = Thirteen ways to look at the correlation coefficient | journal = The American Statistician | volume = 42 | issue = 1| pages = 59–66 | jstor=2685263 | doi=10.1080/00031305.1988.10475524}}</ref>

A Pearson product-moment correlation coefficient attempts to establish a line of best fit through a dataset of two variables by essentially laying out the expected values and the resulting Pearson's correlation coefficient indicates how far away the actual dataset is from the expected values.  Depending on the sign of our Pearson's correlation coefficient, we can end up with either a negative or positive correlation if there is any sort of relationship between the variables of our data set.{{cn|date=November 2023}}

The population correlation coefficient <math>\rho_{X,Y}</math> between two [[random variables]] <math>X</math> and <math>Y</math> with [[expected value]]s <math>\mu_X</math> and <math>\mu_Y</math> and [[standard deviation]]s <math>\sigma_X</math> and <math>\sigma_Y</math> is defined as:

<math display=block>\rho_{X,Y} = \operatorname{corr}(X,Y) = {\operatorname{cov}(X,Y) \over \sigma_X \sigma_Y} = {\operatorname{E}[(X-\mu_X)(Y-\mu_Y)] \over \sigma_X\sigma_Y}, \quad \text{if}\ \sigma_{X}\sigma_{Y}>0.</math>

where <math>\operatorname{E}</math> is the [[expected value]] operator, <math>\operatorname{cov}</math> means [[covariance]], and <math>\operatorname{corr}</math> is a widely used alternative notation for the correlation coefficient. The Pearson correlation is defined only if both standard deviations are finite and positive. An alternative formula purely in terms of [[moment (mathematics)|moments]] is:

<math display=block>\rho_{X,Y} =  {\operatorname{E}(XY)-\operatorname{E}(X)\operatorname{E}(Y)\over \sqrt{\operatorname{E}(X^2)-\operatorname{E}(X)^2}\cdot \sqrt{\operatorname{E}(Y^2)-\operatorname{E}(Y)^2} }</math>

===Correlation and independence===
It is a corollary of the [[Cauchy–Schwarz inequality]] that the [[absolute value]] of the Pearson correlation coefficient is not bigger than 1. Therefore, the value of a correlation coefficient ranges between −1 and +1. The correlation coefficient is +1 in the case of a perfect direct (increasing) linear relationship (correlation), −1 in the case of a perfect inverse (decreasing) linear relationship ('''anti-correlation'''),<ref>Dowdy, S. and Wearden, S. (1983). "Statistics for Research", Wiley. {{ISBN|0-471-08602-9}} pp 230</ref> and some value in the [[open interval]] <math>(-1,1)</math> in all other cases, indicating the degree of [[linear dependence]] between the variables. As it approaches zero there is less of a relationship (closer to uncorrelated). The closer the coefficient is to either −1 or 1, the stronger the correlation between the variables.

If the variables are [[statistical independence|independent]], Pearson's correlation coefficient is 0. However, because the correlation coefficient detects only linear dependencies between two variables, the converse is not necessarily true. A correlation coefficient of 0 does not imply that the variables are independent{{Cn|date=May 2024}}.

<math display=block>\begin{align}
X,Y \text{ independent} \quad & \Rightarrow \quad \rho_{X,Y} = 0 \quad (X,Y \text{ uncorrelated})\\
\rho_{X,Y} = 0 \quad (X,Y \text{ uncorrelated})\quad & \nRightarrow \quad X,Y \text{ independent}
\end{align}</math>

For example, suppose the random variable <math>X</math> is symmetrically distributed about zero, and <math>Y=X^2</math>. Then <math>Y</math> is completely determined by <math>X</math>, so that <math>X</math> and <math>Y</math> are perfectly dependent, but their correlation is zero; they are [[uncorrelated]]. However, in the special case when <math>X</math> and <math>Y</math> are [[Joint normality|jointly normal]], uncorrelatedness is equivalent to independence.

Even though uncorrelated data does not necessarily imply independence, one can check if random variables are independent if their [[mutual information]] is 0.


===Sample correlation coefficient===
Given a series of <math>n</math> measurements of the pair <math>(X_i,Y_i)</math> indexed by <math>i=1,\ldots,n</math>, the ''sample correlation coefficient'' can be used to estimate the population Pearson correlation <math>\rho_{X,Y}</math> between <math>X</math> and <math>Y</math>.  The sample correlation coefficient is defined as

:<math>
r_{xy} \quad \overset{\underset{\mathrm{def}}{}}{=} \quad \frac{\sum\limits_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}{(n-1)s_x s_y}
      =\frac{\sum\limits_{i=1}^n (x_i-\bar{x})(y_i-\bar{y})}
            {\sqrt{\sum\limits_{i=1}^n (x_i-\bar{x})^2 \sum\limits_{i=1}^n (y_i-\bar{y})^2}},
</math>

where <math>\overline{x}</math> and <math>\overline{y}</math> are the sample [[arithmetic mean|means]] of <math>X</math> and <math>Y</math>, and <math>s_x</math> and <math>s_y</math> are the [[Standard deviation#Corrected sample standard deviation|corrected sample standard deviations]] of <math>X</math> and <math>Y</math>.

Equivalent expressions for <math>r_{xy}</math> are
:<math>
\begin{align}
r_{xy} &=\frac{\sum x_iy_i-n \bar{x} \bar{y}}{n s'_x s'_y} \\[5pt]
       &=\frac{n\sum x_iy_i-\sum x_i\sum y_i}{\sqrt{n\sum x_i^2-(\sum x_i)^2}~\sqrt{n\sum y_i^2-(\sum y_i)^2}}.
\end{align}
</math>
where <math>s'_x</math> and <math>s'_y</math> are the [[Standard deviation#Uncorrected sample standard deviation|''uncorrected'' sample standard deviations]] of <math>X</math> and <math>Y</math>.

If <math>x</math> and <math>y</math> are results of measurements that contain measurement error, the realistic limits on the correlation coefficient are not −1 to +1 but a smaller range.<ref>{{cite journal|last=Francis|first=DP|author2=Coats AJ|author3=Gibson D|title=How high can a correlation coefficient be?|journal=Int J Cardiol|year=1999|volume=69|pages=185–199|doi=10.1016/S0167-5273(99)00028-5|issue=2|pmid=10549842}}</ref> For the case of a linear model with a single independent variable, the [[Coefficient of determination|coefficient of determination (R squared)]] is the square of <math>r_{xy}</math>, Pearson's product-moment coefficient.

===Example===
Consider the [[joint probability distribution]] of {{mvar|X}} and {{mvar|Y}} given in the table below.

:{| class="wikitable" style="text-align:center;"
|+ <math>\mathrm{P}(X=x,Y=y)</math>
! {{diagonal split header|{{mvar|x}}|{{mvar|y}}}}
!&minus;1
!0
!1
|-
!0
|0
|{{sfrac|1|3}}
|0
|-
!1
|{{sfrac|1|3}}
|0
|{{sfrac|1|3}}
|}

For this joint distribution, the [[marginal distribution]]s are:
:<math>\mathrm{P}(X=x)=
\begin{cases}
\frac 1 3 & \quad \text{for } x=0 \\
\frac 2 3 & \quad \text{for } x=1 
\end{cases}
</math>

:<math>\mathrm{P}(Y=y)=
\begin{cases}
\frac 1 3 & \quad \text{for } y=-1 \\
\frac 1 3 & \quad \text{for } y=0 \\
\frac 1 3 & \quad \text{for } y=1 
\end{cases}
</math>

This yields the following expectations and variances:
:<math>\mu_X = \frac 2 3</math>
:<math>\mu_Y = 0</math>
:<math>\sigma_X^2 = \frac 2 9</math>
:<math>\sigma_Y^2 = \frac 2 3</math>

Therefore:

: <math>
\begin{align}
\rho_{X,Y} & = \frac{1}{\sigma_X \sigma_Y} \mathrm{E}[(X-\mu_X)(Y-\mu_Y)] \\[5pt]
& = \frac{1}{\sigma_X \sigma_Y} \sum_{x,y}{(x-\mu_X)(y-\mu_Y) \mathrm{P}(X=x,Y=y)} \\[5pt]
& = \frac{3\sqrt{3}}{2}\left(\left(1-\frac 2 3\right)(-1-0)\frac{1}{3} + \left(0-\frac 2 3\right)(0-0)\frac{1}{3} + \left(1-\frac 2 3\right)(1-0)\frac{1}{3}\right) = 0.
\end{align}
</math>