Editing Overfitting (section)

===Regression===
In [[regression analysis]], overfitting occurs frequently.<ref name="RMS">{{citation| title= Regression Modeling Strategies | last= Harrell | first= F. E. Jr. | year= 2001 | publisher= Springer}}.</ref> As an extreme example, if there are ''p'' variables in a [[linear regression]] with ''p'' data points, the fitted line can go exactly through every point.<ref>{{cite web
| url=http://www.ma.utexas.edu/users/mks/statmistakes/ovefitting.html
| title=Overfitting
| author=Martha K. Smith
| date=2014-06-13
| publisher=[[University of Texas at Austin]]
| access-date=2016-07-31}}</ref> For [[logistic regression]] or Cox [[proportional hazards models]], there are a variety of rules of thumb (e.g. 5–9,<ref name="Vittinghoff et al. (2007)">{{cite journal |first1=E. |last1=Vittinghoff |first2=C. E. |last2=McCulloch |year=2007 |title=Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression |journal=[[American Journal of Epidemiology]] |volume=165 |issue=6 |pages=710–718 |doi=10.1093/aje/kwk052 |pmid=17182981|doi-access= }}</ref> 10<ref>{{cite book
| title = Applied Regression Analysis 
| edition= 3rd 
| last1 = Draper
| first1 = Norman R.
| last2 = Smith
| first2 = Harry
| publisher = [[John Wiley & Sons|Wiley]]
| year = 1998
| isbn = 978-0471170822}}</ref> and 10–15<ref>{{cite web
| url = http://blog.minitab.com/blog/adventures-in-statistics/the-danger-of-overfitting-regression-models
| title = The Danger of Overfitting Regression Models
| author = Jim Frost
| date = 2015-09-03
| access-date = 2016-07-31}}</ref> — the guideline of 10 observations per independent variable is known as the "[[one in ten rule]]"). In the process of regression model selection, the mean squared error of the random regression function can be split into random noise, approximation bias, and variance in the estimate of the regression function. The [[bias–variance tradeoff]] is often used to overcome overfit models.

With a large set of [[explanatory variable]]s that actually have no relation to the [[dependent variable]] being predicted, some variables will in general be falsely found to be [[statistically significant]] and the researcher may thus retain them in the model, thereby overfitting the model. This is known as [[Freedman's paradox]].