Editing Linear model (section)

==Linear regression models==
{{main|Linear regression}}

For the regression case, the [[statistical model]] is as follows. Given a (random) sample <math> (Y_i, X_{i1}, \ldots, X_{ip}), \, i = 1, \ldots, n </math> the relation between the observations <math>Y_i</math> and the [[independent variables]] <math>X_{ij}</math> is formulated as

:<math>Y_i = \beta_0 + \beta_1 \phi_1(X_{i1}) + \cdots + \beta_p \phi_p(X_{ip}) + \varepsilon_i \qquad i = 1, \ldots, n </math>

where <math> \phi_1, \ldots, \phi_p </math> may be [[Nonlinear system|nonlinear]] functions. In the above, the quantities <math>\varepsilon_i</math> are [[random variable]]s representing errors in the relationship. The "linear" part of the designation relates to the appearance of the [[regression coefficient]]s, <math>\beta_j</math> in a linear way in the above relationship. Alternatively, one may say that the predicted values corresponding to the above model, namely
:<math>\hat{Y}_i = \beta_0 + \beta_1 \phi_1(X_{i1}) + \cdots + \beta_p \phi_p(X_{ip}) \qquad (i = 1, \ldots, n), </math>
are linear functions of the <math>\beta_j</math>.

Given that estimation is undertaken on the basis of a [[least squares]] analysis, estimates of the unknown parameters <math>\beta_j</math> are determined by minimising a sum of squares function
:<math>S = \sum_{i = 1}^n \varepsilon_i^2 = \sum_{i = 1}^n \left(Y_i - \beta_0 - \beta_1 \phi_1(X_{i1}) - \cdots - \beta_p \phi_p(X_{ip})\right)^2 .</math>
From this, it can readily be seen that the "linear" aspect of the model means the following:
:*the function to be minimised is a quadratic function of the <math>\beta_j</math> for which minimisation is a relatively simple problem;
:*the derivatives of the function are linear functions of the <math>\beta_j</math> making it easy to find the minimising values;
:*the  minimising values <math>\beta_j</math> are linear functions of the observations <math>Y_i</math>;
:*the  minimising values <math>\beta_j</math> are linear functions of the random errors <math>\varepsilon_i</math> which makes it relatively easy to determine the statistical properties of the estimated values of <math>\beta_j</math>.