Editing Statistical significance (section)

== Limitations ==

Researchers focusing solely on whether their results are statistically significant might report findings that are not substantive<ref name="Carver">{{Cite journal | last1 = Carver| first1 = Ronald P. | title = The Case Against Statistical Significance Testing | journal = Harvard Educational Review | volume = 48| issue = 3 | pages = 378–399 | year = 1978| doi = 10.17763/haer.48.3.t490261645281841 | s2cid = 16355113 }}</ref> and not replicable.<ref name="Ioannidis">{{cite journal | last1 = Ioannidis | first1 = John P. A. | title = Why most published research findings are false | journal = PLOS Medicine | volume = 2 | issue = 8 | pages = e124 | year = 2005 | doi=10.1371/journal.pmed.0020124 | pmid=16060722 | pmc=1182327 | doi-access = free }}</ref><ref name="peerj.com">{{cite journal|last1= Amrhein|first1=Valentin|last2=Korner-Nievergelt|first2=Fränzi|last3=Roth|first3=Tobias|title=The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research|journal=PeerJ|date=2017|volume=5|page=e3544|doi=10.7717/peerj.3544|pmid=28698825|pmc=5502092 |doi-access=free }}</ref> There is also a difference between statistical significance and practical significance. A study that is found to be statistically significant may not necessarily be practically significant.<ref name="A Visitor’s Guide to Effect Sizes">{{cite journal|last1=Hojat|first1=Mohammadreza|last2=Xu|first2=Gang|title=A Visitor's Guide to Effect Sizes|journal=Advances in Health Sciences Education|volume=9|issue=3|pages=241–9|date=2004|doi=10.1023/B:AHSE.0000038173.00909.f6|pmid=15316274|s2cid=8045624}}</ref><ref name=":1"/>

=== Effect size ===
{{Main|Effect size}}

Effect size is a measure of a study's practical significance.<ref name="A Visitor’s Guide to Effect Sizes"/> A statistically significant result may have a weak effect. To gauge the research significance of their result, researchers are encouraged to always report an [[effect size]] along with ''p''-values. An effect size measure quantifies the strength of an effect, such as the distance between two means in units of standard deviation (cf. [[Cohen's d]]), the [[Pearson product-moment correlation coefficient|correlation coefficient]] between two variables or [[Coefficient of determination|its square]], and other measures.<ref name=Pedhazur>{{cite book | last1 = Pedhazur | first1 = Elazar J. | last2=Schmelkin|first2=Liora P. | title = Measurement, Design, and Analysis: An Integrated Approach| edition=Student|publisher = Psychology Press |location = New York, NY | year = 1991|isbn =978-0-8058-1063-9 |pages=180–210}}</ref>

=== Reproducibility ===
{{Main|Reproducibility}}

A statistically significant result may not be easy to reproduce.<ref name="peerj.com"/> In particular, some statistically significant results will in fact be false positives. Each failed attempt to reproduce a result increases the likelihood that the result was a false positive.<ref>{{cite book|last1=Stahel|first1=Werner|chapter=Statistical Issue in Reproducibility|title=Principles, Problems, Practices, and Prospects Reproducibility: Principles, Problems, Practices, and Prospects|date=2016|pages=87–114|doi=10.1002/9781118865064.ch5|isbn=978-1-118-86497-5}}</ref>