Editing Exponential distribution (section)

===Prediction===
Having observed a sample of ''n'' data points from an unknown exponential distribution a common task is to use these samples to make predictions about future data from the same source. A common predictive distribution over future samples is the so-called plug-in distribution, formed by plugging a suitable estimate for the rate parameter ''λ'' into the exponential density function. A common choice of estimate is the one provided by the principle of maximum likelihood, and using this yields the predictive density over a future sample ''x''<sub>''n''+1</sub>, conditioned on the observed samples ''x'' = (''x''<sub>1</sub>, ..., ''x<sub>n</sub>'') given by
<math display="block">p_{\rm ML}(x_{n+1} \mid x_1, \ldots, x_n) = \left( \frac1{\overline{x}} \right) \exp \left( - \frac{x_{n+1}}{\overline{x}} \right).</math>

The Bayesian approach provides a predictive distribution which takes into account the uncertainty of the estimated parameter, although this may depend crucially on the choice of prior.

A predictive distribution free of the issues of choosing priors that arise under the subjective Bayesian approach is

<math display="block">p_{\rm CNML}(x_{n+1} \mid x_1, \ldots, x_n) = \frac{ n^{n+1} \left( \overline{x} \right)^n }{ \left( n \overline{x} + x_{n+1} \right)^{n+1} },</math>

which can be considered as
# a frequentist [[confidence distribution]], obtained from the distribution of the pivotal quantity <math>{x_{n+1}}/{\overline{x}}</math>;<ref>{{cite journal |last1=Lawless |first1=J. F. |last2=Fredette |first2=M. |title=Frequentist predictions intervals and predictive distributions |journal=Biometrika |year=2005 |volume=92 |issue=3 |pages=529–542 |doi=10.1093/biomet/92.3.529 |doi-access= }}</ref>
# a profile predictive likelihood, obtained by eliminating the parameter ''λ'' from the joint likelihood of ''x''<sub>''n''+1</sub> and ''λ'' by maximization;<ref>{{cite journal | last1 = Bjornstad | first1 = J.F. | year = 1990 | title = Predictive Likelihood: A Review | journal = Statist. Sci. | volume = 5 | issue = 2| pages = 242–254 | doi=10.1214/ss/1177012175| doi-access = free }}</ref>
# an objective Bayesian predictive posterior distribution, obtained using the non-informative [[Jeffreys prior]] 1/''λ'';
# the Conditional Normalized Maximum Likelihood (CNML) predictive distribution, from information theoretic considerations.<ref>D. F. Schmidt and E. Makalic, "[http://www.emakalic.org/blog/wp-content/uploads/2010/04/SchmidtMakalic09b.pdf Universal Models for the Exponential Distribution]", ''[[IEEE Transactions on Information Theory]]'', Volume 55, Number 7, pp. 3087–3090, 2009 {{doi|10.1109/TIT.2009.2018331}}</ref>

The accuracy of a predictive distribution may be measured using the distance or divergence between the true exponential distribution with rate parameter, ''λ''<sub>0</sub>, and the predictive distribution based on the sample ''x''. The [[Kullback–Leibler divergence]] is a commonly used, parameterisation free measure of the difference between two distributions. Letting Δ(''λ''<sub>0</sub>||''p'') denote the Kullback–Leibler divergence between an exponential with rate parameter ''λ''<sub>0</sub> and a predictive distribution ''p'' it can be shown that

<math display="block">\begin{align}
\operatorname{E}_{\lambda_0} \left[ \Delta(\lambda_0\parallel p_{\rm ML}) \right] &= \psi(n) + \frac{1}{n-1} - \log(n) \\
\operatorname{E}_{\lambda_0} \left[ \Delta(\lambda_0\parallel p_{\rm CNML}) \right] &= \psi(n) + \frac{1}{n} - \log(n)
\end{align}</math>

where the expectation is taken with respect to the exponential distribution with rate parameter {{nowrap|''λ''<sub>0</sub> ∈ (0, ∞)}}, and {{nowrap|ψ( · )}} is the digamma function. It is clear that the CNML predictive distribution is strictly superior to the maximum likelihood plug-in distribution in terms of average Kullback–Leibler divergence for all sample sizes {{nowrap|''n'' > 0}}.