The Propositional Case

Let $A_i, 1\leq i\leq n,$ be a set of attributes, and let Cl be the class attribute. Given that an individual takes on the values $a_1\ldots a_n$ for attributes $A_i\ldots A_n$, in a Bayesian approach the most likely class value c is the one that maximises

$$P(c\vert a_1\ldots a_n) = \frac{P(a_1\ldots a_n\vert c)P(c)}{P(a_1\ldots a_n)}$$ (1)

Here we write P(a_i) as an abbreviation for P(A_i=a_i).

In order to decrease the number of probabilities involved in this calculation, and to increase the reliability of their estimates, usually the simplifying naive Bayes assumption is made that $P(a_1\ldots a_n\vert c) = P(a_1\vert c)\ldots P(a_n\vert c)$, i.e. the values taken on by the different attributes are conditionally independent given the class value. The predicted class value c is the one that maximises $P(c) P(a_1\vert c)\ldots P(a_n\vert c)$:

$$argmax_c P(c) \prod P(a_i\vert c)$$ (2)

(For a given individual the term $P(a_1\ldots a_n)$ is a constant normalising term that can be ignored if we're only interested in determining the most likely class value.)

The classifier which predicts by maximising the above expression is called the naive Bayesian classifier, or Bayesian classifier for short. Essentially, it reads the description of an individual to be classified, and then tries to estimate how likely it is to observe such an individual among each of the possible classes. Thus, the fundamental problem of a Bayesian classifier (naive or otherwise) is to estimate how likely it is to observe an individual satisfying a particular description among given sub-populations. In our case these estimates are obtained from the training set, under the naive Bayes assumption of conditional independence. Even in cases where this assumption is clearly invalid, the Bayesian classifier has been shown to give good results [5].