Editing Naive Bayes classifier (section)

====Dealing with rare words====
In the case a word has never been met during the learning phase, both the numerator and the denominator are equal to zero, both in the general formula and in the spamicity formula. The software can decide to discard such words for which there is no information available.

More generally, the words that were encountered only a few times during the learning phase cause a problem, because it would be an error to trust blindly the information they provide. A simple solution is to simply avoid taking such unreliable words into account as well.

Applying again Bayes' theorem, and assuming the classification between spam and ham of the emails containing a given word ("replica") is a [[random variable]] with [[beta distribution]], some programs decide to use a corrected probability:

:<math>\Pr'(S|W) = \frac{s \cdot \Pr(S) + n \cdot \Pr(S|W)}{s + n }</math>

where:
*<math>\Pr'(S|W)</math> is the corrected probability for the message to be spam, knowing that it contains a given word ;
* <math>s</math> is the ''strength'' we give to background information about incoming spam ;
* <math>\Pr(S)</math> is the probability of any incoming message to be spam ;
* <math>n</math> is the number of occurrences of this word during the learning phase ;
* <math>\Pr(S|W)</math> is the spamicity of this word.

(Demonstration:<ref>{{cite magazine|url=http://www.linuxjournal.com/article/6467|magazine=Linux Journal|author=Gary Robinson|author-link=Gary Robinson|title=A statistical approach to the spam problem|year=2003|access-date=2007-07-19|archive-url=https://web.archive.org/web/20101022001749/http://www.linuxjournal.com/article/6467|archive-date=2010-10-22|url-status=live}}</ref>)

This corrected probability is used instead of the spamicity in the combining formula.

This formula can be extended to the case where ''n'' is equal to zero (and where the spamicity is not defined), and evaluates in this case to <math>Pr(S)</math>.