Editing Law of large numbers (section)

==Proof of the weak law==
Given ''X''<sub>1</sub>, ''X''<sub>2</sub>, ... an infinite sequence of [[i.i.d.]] random variables with finite expected value <math>E(X_1)=E(X_2)=\cdots=\mu<\infty</math>, we are interested in the convergence of the sample average

<math display="block">\overline{X}_n=\tfrac1n(X_1+\cdots+X_n). </math>

The weak law of large numbers states:
{{NumBlk||<math display="block">
    \overline{X}_n\ \overset{P}{\rightarrow}\ \mu \qquad\textrm{when}\ n \to \infty.
</math>|{{EquationRef|2}}}}

===Proof using Chebyshev's inequality assuming finite variance===
This proof uses the assumption of finite [[variance]] <math> \operatorname{Var} (X_i)=\sigma^2 </math> (for all <math>i</math>). The independence of the random variables implies no correlation between them, and we have that

<math display="block">
\operatorname{Var}(\overline{X}_n) = \operatorname{Var}(\tfrac1n(X_1+\cdots+X_n)) = \frac{1}{n^2} \operatorname{Var}(X_1+\cdots+X_n) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}.
</math>

The common mean μ of the sequence is the mean of the sample average:

<math display="block">
E(\overline{X}_n) = \mu.
</math>

Using [[Chebyshev's inequality]] on <math>\overline{X}_n </math> results in

<math display="block">
\operatorname{P}( \left| \overline{X}_n-\mu \right| \geq \varepsilon) \leq \frac{\sigma^2}{n\varepsilon^2}.
</math>

This may be used to obtain the following:

<math display="block">
\operatorname{P}( \left| \overline{X}_n-\mu \right| < \varepsilon) = 1 - \operatorname{P}( \left| \overline{X}_n-\mu \right| \geq \varepsilon) \geq 1 - \frac{\sigma^2}{n \varepsilon^2 }.
</math>

As ''n'' approaches infinity, the expression approaches 1. And by definition of [[convergence in probability]], we have obtained
{{NumBlk||<math display="block">
    \overline{X}_n\ \overset{P}{\rightarrow}\ \mu \qquad\textrm{when}\ n \to \infty.
</math>|{{EquationRef|2}}}}

===Proof using convergence of characteristic functions===
By [[Taylor's theorem]] for [[complex function]]s, the [[Characteristic function (probability theory)|characteristic function]] of any random variable, ''X'', with finite mean μ, can be written as

<math display="block">\varphi_X(t) = 1 + it\mu + o(t), \quad t \rightarrow 0.</math>

All ''X''<sub>1</sub>, ''X''<sub>2</sub>, ... have the same characteristic function, so we will simply denote this ''φ''<sub>''X''</sub>.

Among the basic properties of characteristic functions there are

<math display="block">\varphi_{\frac 1 n X}(t)= \varphi_X(\tfrac t n) \quad \text{and} \quad
 \varphi_{X+Y}(t) = \varphi_X(t) \varphi_Y(t) \quad </math> if ''X'' and ''Y'' are independent.

These rules can be used to calculate the characteristic function of <math>\overline{X}_n</math> in terms of ''φ''<sub>''X''</sub>:

<math display="block">\varphi_{\overline{X}_n}(t)= \left[\varphi_X\left({t \over n}\right)\right]^n = \left[1 + i\mu{t \over n} + o\left({t \over n}\right)\right]^n \, \rightarrow \, e^{it\mu}, \quad \text{as} \quad n \to \infty.</math>

The limit ''e''<sup>''itμ''</sup> is the characteristic function of the constant random variable μ, and hence by the [[Lévy continuity theorem]], <math> \overline{X}_n</math> [[Convergence in distribution|converges in distribution]] to μ:

<math display="block">\overline{X}_n \, \overset{\mathcal D}{\rightarrow} \, \mu \qquad\text{for}\qquad n \to \infty.</math>

μ is a constant, which implies that convergence in distribution to μ and convergence in probability to μ are equivalent (see [[Convergence of random variables]].) Therefore,
{{NumBlk||<math display="block">
    \overline{X}_n\ \overset{P}{\rightarrow}\ \mu \qquad\textrm{when}\ n \to \infty.
</math>|{{EquationRef|2}}}}

This shows that the sample mean converges in probability to the derivative of the characteristic function at the origin, as long as the latter exists.