Editing Law of large numbers (section)

==Proof of the strong law==
We give a relatively simple proof of the strong law under the assumptions that the <math>X_i</math> are [[Independent and identically distributed random variables|iid]], <math> {\mathbb E}[X_i] =: \mu < \infty </math>, <math> \operatorname{Var} (X_i)=\sigma^2 < \infty</math>, and <math> {\mathbb E}[X_i^4] =: \tau < \infty </math>.

Let us first note that without loss of generality we can assume that <math>\mu = 0</math> by centering. In this case, the strong law says that

<math display="block">
    \Pr\!\left( \lim_{n\to\infty}\overline{X}_n = 0 \right) = 1,
</math>
or
<math display="block">
    \Pr\left(\omega: \lim_{n\to\infty}\frac{S_n(\omega)}n = 0 \right) = 1.
</math>
It is equivalent to show that
<math display="block">
    \Pr\left(\omega: \lim_{n\to\infty}\frac{S_n(\omega)}n \neq 0 \right) = 0,
</math>
Note that
<math display="block">
    \lim_{n\to\infty}\frac{S_n(\omega)}n \neq 0 \iff \exists\epsilon>0, \left|\frac{S_n(\omega)}n\right| \ge \epsilon\ \mbox{infinitely often},
</math>
and thus to prove the strong law we need to show that for every <math>\epsilon > 0</math>, we have
<math display="block">
    \Pr\left(\omega: |S_n(\omega)| \ge n\epsilon \mbox{ infinitely often} \right) = 0.
</math>
Define the events <math> A_n = \{\omega : |S_n| \ge n\epsilon\}</math>, and if we can show that 
<math display="block">
    \sum_{n=1}^\infty \Pr(A_n) <\infty,
</math>
then the Borel-Cantelli Lemma implies the result.  So let us estimate <math>\Pr(A_n)</math>.

We compute 
<math display="block">
    {\mathbb E}[S_n^4] = {\mathbb E}\left[\left(\sum_{i=1}^n X_i\right)^4\right] = {\mathbb E}\left[\sum_{1 \le i,j,k,l\le n} X_iX_jX_kX_l\right].
</math>
We first claim that every term of the form <math>X_i^3X_j, X_i^2X_jX_k, X_iX_jX_kX_l</math> where all subscripts are distinct, must have zero expectation.  This is because <math>{\mathbb E}[X_i^3X_j] = {\mathbb E}[X_i^3]{\mathbb E}[X_j]</math> by independence, and the last term is zero—and similarly for the other terms.  Therefore the only terms in the sum with nonzero expectation are <math>{\mathbb E}[X_i^4]</math> and <math>{\mathbb E}[X_i^2X_j^2]</math>.  Since the <math>X_i</math> are identically distributed, all of these are the same, and moreover <math>{\mathbb E}[X_i^2X_j^2]=({\mathbb E}[X_i^2])^2</math>.  

There are <math>n</math> terms of the form <math>{\mathbb E}[X_i^4]</math> and <math>3 n (n-1)</math> terms of the form <math>({\mathbb E}[X_i^2])^2</math>, and so
<math display="block">
    {\mathbb E}[S_n^4] = n \tau + 3n(n-1)\sigma^4.
</math>
Note that the right-hand side is a quadratic polynomial in <math>n</math>, and as such there exists a <math>C>0</math> such that <math> {\mathbb E}[S_n^4] \le Cn^2</math> for <math>n</math> sufficiently large.  By Markov,
<math display="block">
    \Pr(|S_n| \ge n \epsilon) \le \frac1{(n\epsilon)^4}{\mathbb E}[S_n^4] \le \frac{C}{\epsilon^4 n^2},
</math>
for <math>n</math> sufficiently large, and therefore this series is summable.  Since this holds for any <math>\epsilon > 0</math>, we have established the strong law of large numbers.<ref>Another proof was given by  {{cite journal |last1=Etemadi |first1=Nasrollah |title=An elementary proof of the strong law of large numbers |journal=Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete |date=1981 |volume=55 |pages=119–122 |publisher=Springer|doi=10.1007/BF01013465 |s2cid=122166046 |doi-access=free }}</ref> The proof can be strengthened immensely by dropping all finiteness assumptions on the second and fourth moments. It can also be extended for example to discuss partial sums of distributions without any finite moments. Such proofs use more intricate arguments to prove the same Borel-Cantelli predicate, a strategy attributed to Kolmogorov to conceptually bring the limit inside the probability parentheses. <ref>For a proof without the added assumption of a finite fourth moment, see Section 22 of {{cite book|last = Billingsley | first = Patrick| title = Probability and Measure|date = 1979}}</ref>