Editing Statistical hypothesis test (section)

==={{anchor|NHST}}Null hypothesis significance testing (NHST)===
The modern version of hypothesis testing is generally called the '''null hypothesis significance testing (NHST)'''<ref name=nickerson /> and is a hybrid of the Fisher approach with the Neyman-Pearson approach. In 2000, [[Raymond S. Nickerson]] wrote an article stating that NHST was (at the time) "arguably the most widely used method of analysis of data collected in psychological experiments and has been so for about 70 years" and that it was at the same time "very controversial".<ref name=nickerson />

This fusion resulted from confusion by writers of statistical textbooks (as predicted by Fisher) beginning in the 1940s<ref name="Halpin 625–653">{{cite journal|last1=Halpin|first1=P F|last2=Stam|first2=HJ|date=Winter 2006|title=Inductive Inference or Inductive Behavior: Fisher and Neyman: Pearson Approaches to Statistical Testing in Psychological Research (1940–1960)|journal=The American Journal of Psychology|volume=119|issue=4|pages=625–653|doi=10.2307/20445367|jstor=20445367|pmid=17286092}}</ref> (but [[Detection theory|signal detection]], for example, still uses the Neyman/Pearson formulation). Great conceptual differences and many caveats in addition to those mentioned above were ignored. Neyman and Pearson provided the stronger terminology, the more rigorous mathematics and the more consistent philosophy, but the subject taught today in introductory statistics has more similarities with Fisher's method than theirs.<ref name="Gigerenzer">{{cite book|last=Gigerenzer|first=Gerd|title=The Empire of Chance: How Probability Changed Science and Everyday Life|author2=Zeno Swijtink|author3=Theodore Porter|author4=Lorraine Daston|author5=John Beatty|author6=Lorenz Kruger|publisher=Cambridge University Press|year=1989|isbn=978-0-521-39838-1|pages=70–122|chapter=Part 3: The Inference Experts}}</ref>

Sometime around 1940,<ref name="Halpin 625–653" /> authors of statistical text books began combining the two approaches by using the ''p''-value in place of the [[test statistic]] (or data) to test against the Neyman–Pearson "significance level".

{| class="wikitable"
|+ A comparison between Fisherian, frequentist (Neyman–Pearson)
|-
! #
! Fisher's null hypothesis testing !! Neyman–Pearson decision theory
|-
| 1
| Set up a statistical null hypothesis. The null need not be a nil hypothesis (i.e., zero difference).
| Set up two statistical hypotheses, H1 and H2, and decide about α, β, and sample size before the experiment, based on subjective cost-benefit considerations. These define a rejection region for each hypothesis.
|-
| 2
| Report the exact level of significance (e.g. p = 0.051 or p = 0.049). Do not refer to "accepting" or "rejecting" hypotheses. If the result is "not significant", draw no conclusions and make no decisions, but suspend judgement until further data is available.
| If the data falls into the rejection region of H1, accept H2; otherwise accept H1. Accepting a hypothesis does not mean that you believe in it, but only that you act as if it were true.
|-
| 3
| Use this procedure only if little is known about the problem at hand, and only to draw provisional conclusions in the context of an attempt to understand the experimental situation.
| The usefulness of the procedure is limited among others to situations where you have a disjunction of hypotheses (e.g. either μ1 = 8 or μ2 = 10 is true) and where you can make meaningful cost-benefit trade-offs for choosing alpha and beta.
|}