Editing Independence (probability theory) (section)

===For events===
====Two events====
Two events <math>A</math> and <math>B</math> are independent (often written as <math>A \perp B</math> or <math>A \perp\!\!\!\perp B</math>, where the latter symbol often is also used for [[conditional independence]]) if and only if their [[joint probability]] equals the product of their probabilities:<ref name=Florescu>{{cite book | author=Florescu, Ionut| title=Probability and Stochastic Processes| publisher=Wiley| year=2014 | isbn=978-0-470-62455-5}}</ref>{{rp|p. 29}}<ref name=Gallager/>{{rp|p. 10}}

{{Equation box 1
|indent =
|title=
|equation = {{NumBlk||<math>\mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B)</math>|{{EquationRef|Eq.1}}}}
|cellpadding= 6
|border
|border colour = #0073CF
|background colour=#F5FFFA}}

<math>A \cap B \neq \emptyset</math> indicates that two independent events <math>A</math> and <math>B</math> have common elements in their [[sample space]] so that they are not [[Mutual exclusivity|mutually exclusive]] (mutually exclusive iff <math>A \cap B = \emptyset</math>). Why this defines independence is made clear by rewriting with [[Conditional probability|conditional probabilities]] <math>P(A \mid B) = \frac{P(A \cap B)}{P(B)}</math> as the probability at which the event <math>A</math> occurs provided that the event <math>B</math> has or is assumed to have occurred:

:<math>\mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B) \iff \mathrm{P}(A\mid B) = \frac{\mathrm{P}(A \cap B)}{\mathrm{P}(B)} = \mathrm{P}(A).</math>

and similarly

:<math>\mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B) \iff\mathrm{P}(B\mid A) = \frac{\mathrm{P}(A \cap B)}{\mathrm{P}(A)} = \mathrm{P}(B).</math>

Thus, the occurrence of <math>B</math> does not affect the probability of <math>A</math>, and vice versa. In other words, <math>A</math> and <math>B</math> are independent of each other. Although the derived expressions may seem more intuitive, they are not the preferred definition, as the conditional probabilities may be undefined if <math>\mathrm{P}(A)</math> or <math>\mathrm{P}(B)</math> are 0. Furthermore, the preferred definition makes clear by symmetry that when <math>A</math> is independent of <math>B</math>, <math>B</math> is also independent of <math>A</math>.

====Odds====
Stated in terms of [[odds]], two events are independent if and only if the [[odds ratio]] of {{tmath|A}} and {{tmath|B}} is unity (1). Analogously with probability, this is equivalent to the conditional odds being equal to the unconditional odds:
:<math>O(A \mid B) = O(A) \text{ and } O(B \mid A) = O(B),</math>
or to the odds of one event, given the other event, being the same as the odds of the event, given the other event not occurring:
:<math>O(A \mid B) = O(A \mid \neg B) \text{ and } O(B \mid A) = O(B \mid \neg A).</math>
The odds ratio can be defined as
:<math>O(A \mid B) : O(A \mid \neg B),</math>
or symmetrically for odds of {{tmath|B}} given {{tmath|A}}, and thus is 1 if and only if the events are independent.

====More than two events====
A finite set of events <math>\{ A_i \} _{i=1}^{n}</math> is [[Pairwise independence|pairwise independent]] if every pair of events is independent<ref name ="Feller">{{cite book | last = Feller | first = W | year = 1971 | title = An Introduction to Probability Theory and Its Applications | publisher = [[John Wiley & Sons|Wiley]] | chapter = Stochastic Independence}}</ref>&mdash;that is, if and only if for all distinct pairs of indices <math>m,k</math>,

{{Equation box 1
|indent =
|title=
|equation = {{NumBlk||<math>\mathrm{P}(A_m \cap A_k) = \mathrm{P}(A_m)\mathrm{P}(A_k)</math>|{{EquationRef|Eq.2}}}}
|cellpadding= 6
|border
|border colour = #0073CF
|background colour=#F5FFFA}}

A finite set of events is '''mutually independent''' if every event is independent of any intersection of the other events<ref name="Feller" /><ref name=Gallager/>{{rp|p. 11}}&mdash;that is, if and only if for every <math>k \leq n</math> and for every k indices <math>1\le i_1 < \dots < i_k \le n</math>,

{{Equation box 1
|indent =
|title=
|equation = {{NumBlk||<math>\mathrm{P}\left(\bigcap_{j=1}^k A_{i_j} \right)=\prod_{j=1}^k \mathrm{P}(A_{i_j} )</math>|{{EquationRef|Eq.3}}}}
|cellpadding= 6
|border
|border colour = #0073CF
|background colour=#F5FFFA}}

This is called the ''multiplication rule'' for independent events. It is [[#Triple-independence but no pairwise-independence|not a single condition]] involving only the product of all the probabilities of all single events; it must hold true for all subsets of events.

For more than two events, a mutually independent set of events is (by definition) pairwise independent; but the converse is [[#Pairwise and mutual independence|not necessarily true]].<ref name=Florescu/>{{rp|p. 30}}

====Log probability and information content====
Stated in terms of [[log probability]], two events are independent if and only if the log probability of the joint event is the sum of the log probability of the individual events:
:<math>\log \mathrm{P}(A \cap B) = \log \mathrm{P}(A) + \log \mathrm{P}(B)</math>
In [[information theory]], negative log probability is interpreted as [[information content]], and thus two events are independent if and only if the information content of the combined event equals the sum of information content of the individual events:
:<math>\mathrm{I}(A \cap B) = \mathrm{I}(A) + \mathrm{I}(B)</math>
See ''{{slink|Information content|Additivity of independent events}}'' for details.