Editing Maximum likelihood estimation (section)

=== Second-order efficiency after correction for bias ===
However, when we consider the higher-order terms in the [[Edgeworth expansion|expansion]] of the distribution of this estimator, it turns out that {{math|''θ''<sub>mle</sub>}} has bias of order {{frac|1|{{mvar|n}}}}. This bias is equal to (componentwise)<ref>See formula&nbsp;20 in
{{cite journal
  | last1 = Cox | first1 = David R. | author-link1=David R. Cox
  | last2 = Snell | first2 = E. Joyce | author-link2 = Joyce Snell
  | title = A general definition of residuals
  | year = 1968
  | journal = [[Journal of the Royal Statistical Society, Series B]]
  | pages = 248–275
  | jstor = 2984505
  | volume=30
  | issue = 2
}}
</ref>

<math display="block">
    b_h \; \equiv \; \operatorname{\mathbb E} \biggl[ \; \left( \widehat\theta_\mathrm{mle} - \theta_0 \right)_h \; \biggr]
       \; = \; \frac{1}{\,n\,} \, \sum_{i, j, k = 1}^m \; \mathcal{I}^{h i} \; \mathcal{I}^{j k} \left( \frac{1}{\,2\,} \, K_{i j k} \; + \; J_{j,i k} \right)
  </math>

where <math>\mathcal{I}^{j k}</math> (with superscripts) denotes the (''j,k'')-th component of the ''inverse'' Fisher information matrix <math>\mathcal{I}^{-1}</math>, and

<math display="block">
     \frac{1}{\,2\,} \, K_{i j k} \; + \; J_{j,i k} \; = \; \operatorname{\mathbb E}\,\biggl[\;
             \frac12 \frac{\partial^3 \ln f_{\theta_0}(X_t)}{\partial\theta_i\;\partial\theta_j\;\partial\theta_k} +
             \frac{\;\partial\ln f_{\theta_0}(X_t)\;}{\partial\theta_j}\,\frac{\;\partial^2\ln f_{\theta_0}(X_t)\;}{\partial\theta_i \, \partial\theta_k}
             \; \biggr] ~ .
  </math>

Using these formulae it is possible to estimate the second-order bias of the maximum likelihood estimator, and ''correct'' for that bias by subtracting it:
<math display="block">
    \widehat{\theta\,}^*_\text{mle} = \widehat{\theta\,}_\text{mle} - \widehat{b\,} ~ .
  </math>
This estimator is unbiased up to the terms of order {{sfrac|1|&thinsp;{{mvar|n}}&thinsp;}}, and is called the '''bias-corrected maximum likelihood estimator'''.

This bias-corrected estimator is {{em|second-order efficient}} (at least within the curved exponential family), meaning that it has minimal mean squared error among all second-order bias-corrected estimators, up to the terms of the order {{sfrac|1|&thinsp;{{mvar|n}}<sup>2</sup>&thinsp;}}&nbsp;. It is possible to continue this process, that is to derive the third-order bias-correction term, and so on. However, the maximum likelihood estimator is ''not'' third-order efficient.<ref>
{{cite journal
 |last    = Kano
 |first   = Yutaka
 |title   = Third-order efficiency implies fourth-order efficiency
 |year    = 1996
 |journal = Journal of the Japan Statistical Society
 |volume  = 26  |pages = 101–117
 |doi     = 10.14490/jjss1995.26.101
 |doi-access= free
 }}
</ref>