Editing Outlier (section)

== Occurrence and causes ==
[[File:Standard_deviation_diagram_micro.svg|thumb|250px|Relative probabilities in a normal distribution]]
In the case of [[normal distribution|normally distributed]] data, the [[three sigma rule]] means that roughly 1 in 22 observations will differ by twice the [[standard deviation]] or more from the mean, and 1 in 370 will deviate by three times the standard deviation.<ref>{{cite book|last1=Ruan|first1=Da|author1-link=Da Ruan|last2=Chen|first2=Guoqing|last3=Kerre|first3=Etienne|editor1-last=Wets|editor1-first=G.|title=Intelligent Data Mining: Techniques and Applications|url=https://archive.org/details/intelligentdatam00ruan_742|url-access=limited|date=2005|publisher=Springer|isbn=978-3-540-26256-5|page=[https://archive.org/details/intelligentdatam00ruan_742/page/n326 318]|series=Studies in Computational Intelligence Vol. 5}}</ref> In a sample of 1000 observations, the presence of up to five observations deviating from the mean by more than three times the standard deviation is within the range of what can be expected, being less than twice the expected number and hence within 1 standard deviation of the expected number – see [[Poisson distribution]] – and not indicate an anomaly. If the sample size is only 100, however, just three such outliers are already reason for concern, being more than 11 times the expected number.

In general, if the nature of the population distribution is known ''a priori'', it is possible to test if the number of outliers deviate [[Statistical significance|significant]]ly from what can be expected: for a given cutoff (so samples fall beyond the cutoff with probability ''p'') of a given distribution, the number of outliers will follow a [[binomial distribution]] with parameter ''p'', which can generally be well-approximated by the [[Poisson distribution]] with λ = ''pn''. Thus if one takes a normal distribution with cutoff 3 standard deviations from the mean, ''p'' is approximately 0.3%, and thus for 1000 trials one can approximate the number of samples whose deviation exceeds 3 sigmas by a Poisson distribution with λ = 3.

=== Causes ===
Outliers can have many anomalous causes. A physical apparatus for taking measurements may have suffered a transient malfunction. There may have been an error in data transmission or transcription. Outliers arise due to changes in system behaviour, fraudulent behaviour, human error, instrument error or simply through natural deviations in populations. A sample may have been contaminated with elements from outside the population being examined. Alternatively, an outlier could be the result of a flaw in the assumed theory, calling for further investigation by the researcher.  Additionally, the pathological appearance of outliers of a certain form appears in a variety of datasets, indicating that the causative mechanism for the data might differ at the extreme end ([[King effect]]).