Editing Maximum likelihood estimation (section)

=== [[Newton's method|Newton–Raphson method]] ===
<math display="block">\eta_r = 1</math> and <math>\mathbf{d}_r\left(\widehat{\theta}\right) = -\mathbf{H}^{-1}_r\left(\widehat{\theta}\right) \mathbf{s}_r\left(\widehat{\theta}\right)</math>

where <math>\mathbf{s}_{r}(\widehat{\theta})</math> is the [[Score (statistics)|score]] and <math>\mathbf{H}^{-1}_r \left(\widehat{\theta}\right)</math> is the [[Invertible matrix|inverse]] of the [[Hessian matrix]] of the log-likelihood function, both evaluated the <var>r</var>th iteration.<ref>{{cite book |first=Takeshi |last=Amemiya |author-link=Takeshi Amemiya |title=Advanced Econometrics |location=Cambridge |publisher=Harvard University Press |year=1985 |isbn=0-674-00560-0 |pages=[https://archive.org/details/advancedeconomet00amem/page/137 137–138] |url=https://archive.org/details/advancedeconomet00amem/page/137 }}</ref><ref>{{cite book |first=Denis |last=Sargan |author-link=Denis Sargan |chapter=Methods of Numerical Optimization |title=Lecture Notes on Advanced Econometric Theory |location=Oxford |publisher=Basil Blackwell |year=1988 |isbn=0-631-14956-2 |pages=161–169 }}</ref> But because the calculation of the Hessian matrix is [[Computational complexity|computationally costly]], numerous alternatives have been proposed. The popular [[Berndt–Hall–Hall–Hausman algorithm]] approximates the Hessian with the [[outer product]] of the expected gradient, such that

<math display="block">\mathbf{d}_r\left(\widehat{\theta}\right) = - \left[ \frac{1}{n} \sum_{t=1}^n \frac{\partial \ell(\theta;\mathbf{y})}{\partial \theta} \left( \frac{\partial \ell(\theta;\mathbf{y})}{\partial \theta} \right)^{\mathsf{T}} \right]^{-1} \mathbf{s}_r \left(\widehat{\theta}\right)</math>