Editing Naive Bayes classifier (section)

===Person classification===
Problem: classify whether a given person is a male or a female based on the measured features.
The features include height, weight, and foot size. Although with NB classifier we treat them as independent, they are not in reality.

====Training====
Example training set below.

{| class="wikitable"
|-
! Person !! height (feet) !! weight (lbs) !! foot size (inches)
|-
| male || 6 || 180 || 12
|-
| male || 5.92 (5'11") || 190 || 11
|-
| male || 5.58 (5'7") || 170 || 12
|-
| male || 5.92 (5'11") || 165 || 10
|-
| female || 5 || 100 || 6
|-
| female || 5.5 (5'6") || 150 || 8
|-
| female || 5.42 (5'5") || 130 || 7
|-
| female || 5.75 (5'9") || 150 || 9
|-
|}

The classifier created from the training set using a Gaussian distribution assumption would be (given variances are ''unbiased'' [[Variance#Population variance and sample variance|sample variances]]):
{| class="wikitable"
|-
! Person !! mean (height) !! variance (height) !! mean (weight) !! variance (weight) !! mean (foot size) !! variance (foot size)
|-
| male || 5.855 || 3.5033 × 10<sup>−2</sup> || 176.25 || 1.2292 × 10<sup>2</sup> || 11.25 || 9.1667 × 10<sup>−1</sup>
|-
| female || 5.4175 || 9.7225 × 10<sup>−2</sup> || 132.5 || 5.5833 × 10<sup>2</sup> || 7.5 || 1.6667
|}

The following example assumes equiprobable classes so that P(male)= P(female) = 0.5. This prior [[probability distribution]] might be based on prior knowledge of frequencies in the larger population or in the training set.

====Testing====

Below is a sample to be classified as male or female.

{| class="wikitable"
|-
! Person !! height (feet) !! weight (lbs) !! foot size (inches)
|-
| sample || 6 || 130 || 8
|}

In order to classify the sample, one has to determine which posterior is greater, male or female. For the classification as male the posterior is given by
<math display="block">
\text{posterior (male)} = \frac{P(\text{male}) \, p(\text{height} \mid \text{male}) \, p(\text{weight} \mid \text{male}) \, p(\text{foot size} \mid \text{male})}{\text{evidence}}
</math>

For the classification as female the posterior is given by
<math display="block">
\text{posterior (female)} = \frac{P(\text{female}) \, p(\text{height} \mid \text{female}) \, p(\text{weight} \mid \text{female}) \, p(\text{foot size} \mid \text{female})}{\text{evidence}}
</math>

The evidence (also termed normalizing constant) may be calculated:
<math display="block">\begin{align}
\text{evidence} = P(\text{male}) \, p(\text{height} \mid \text{male}) \, p(\text{weight} \mid \text{male}) \, p(\text{foot size} \mid \text{male}) \\
+ P(\text{female}) \, p(\text{height} \mid \text{female}) \, p(\text{weight} \mid \text{female}) \, p(\text{foot size} \mid \text{female})
\end{align}</math>

However, given the sample, the evidence is a constant and thus scales both posteriors equally. It therefore does not affect classification and can be ignored.  The [[probability distribution]] for the sex of the sample can now be determined:
<math display="block">P(\text{male}) = 0.5</math>
<math display="block">p({\text{height}} \mid \text{male}) = \frac{1}{\sqrt{2\pi \sigma^2}}\exp\left(\frac{-(6-\mu)^2}{2\sigma^2}\right) \approx 1.5789,</math>
where <math>\mu = 5.855</math> and <math>\sigma^2 = 3.5033 \cdot 10^{-2}</math> are the parameters of normal distribution which have been previously determined from the training set. Note that a value greater than 1 is OK here – it is a probability density rather than a probability, because ''height'' is a continuous variable.

<math display="block">p({\text{weight}} \mid \text{male}) = \frac{1}{\sqrt{2\pi \sigma^2}}\exp\left(\frac{-(130-\mu)^2}{2\sigma^2}\right) = 5.9881 \cdot 10^{-6}</math>
<math display="block">p({\text{foot size}} \mid \text{male}) = \frac{1}{\sqrt{2\pi \sigma^2}}\exp\left(\frac{-(8-\mu)^2}{2\sigma^2}\right) = 1.3112 \cdot 10^{-3}</math>
<math display="block">\text{posterior numerator (male)} = \text{their product} = 6.1984 \cdot 10^{-9}</math>

<math display="block">P({\text{female}}) = 0.5</math>
<math display="block">p({\text{height}} \mid {\text{female}}) = 2.23 \cdot 10^{-1}</math>
<math display="block">p({\text{weight}} \mid {\text{female}}) = 1.6789 \cdot 10^{-2}</math>
<math display="block">p({\text{foot size}} \mid {\text{female}}) = 2.8669 \cdot 10^{-1}</math>
<math display="block">\text{posterior numerator (female)} = \text{their product} = 5.3778 \cdot 10^{-4}</math>

Since posterior numerator is greater in the female case, the prediction is that the sample is female.