Editing Meta-analysis (section)

===Problems related to the statistical approach===

Other weaknesses are that it has not been determined if the statistically most accurate method for combining results is the fixed, IVhet, random or quality effect models, though the criticism against the random effects model is mounting because of the perception that the new random effects (used in meta-analysis) are essentially formal devices to facilitate smoothing or shrinkage and prediction may be impossible or ill-advised.<ref>{{cite web | vauthors = Hodges JS, Clayton MK | title = Random effects old and new  | date = February 2011 | pages = 1–23 | citeseerx = 10.1.1.225.2685 }}</ref> The main problem with the random effects approach is that it uses the classic statistical thought of generating a "compromise estimator" that makes the weights close to the naturally weighted estimator if [[study heterogeneity|heterogeneity]] across studies is large but close to the inverse variance weighted estimator if the between study heterogeneity is small. However, what has been ignored is the distinction between the model ''we choose'' to analyze a given dataset, and the ''mechanism by which the data came into being''.<ref name="ReferenceB">{{cite book | vauthors = Hodges JS | chapter =  Random effects old and new. |title=Richly parameterized linear models: additive, time series, and spatial models using random effects |date=2014 |publisher=CRC Press |location=Boca Raton |isbn=978-1-4398-6683-2 |pages=285–302}}</ref> A random effect can be present in either of these roles, but the two roles are quite distinct. There's no reason to think the analysis model and data-generation mechanism (model) are similar in form, but many sub-fields of statistics have developed the habit of assuming, for theory and simulations, that the data-generation mechanism (model) is identical to the analysis model we choose (or would like others to choose). As a hypothesized mechanisms for producing the data, the random effect model for meta-analysis is silly and it is more appropriate to think of this model as a superficial description and something we choose as an analytical tool – but this choice for meta-analysis may not work because the study effects are a fixed feature of the respective meta-analysis and the probability distribution is only a descriptive tool.<ref name="ReferenceB"/>