Editing Inductive bias (section)

==Types==

The following is a list of common inductive biases in machine learning algorithms.

* '''Maximum [[conditional independence]]''': if the hypothesis can be cast in a [[Bayesian inference|Bayesian]] framework, try to maximize conditional independence. This is the bias used in the [[Naive Bayes classifier]].
* '''Minimum [[Cross-validation (statistics)|cross-validation]] error''': when trying to choose among hypotheses, select the hypothesis with the lowest cross-validation error. Although cross-validation may seem to be free of bias, the [[No free lunch theorem|"no free lunch"]] theorems show that cross-validation must be biased, for example assuming that there is no information encoded in the ordering of the data.
* '''Maximum margin''': when drawing a boundary between two classes, attempt to maximize the width of the boundary. This is the bias used in [[support vector machines]]. The assumption is that distinct classes tend to be separated by wide boundaries.
* '''[[Minimum description length]]''': when forming a hypothesis, attempt to minimize the length of the description of the hypothesis. 
* '''Minimum features''': unless there is good evidence that a [[feature space|feature]] is useful, it should be deleted. This is the assumption behind [[feature selection]] algorithms.
* '''Nearest neighbors''': assume that most of the cases in a small neighborhood in [[feature space]] belong to the same class. Given a case for which the class is unknown, guess that it belongs to the same class as the majority in its immediate neighborhood. This is the bias used in the [[k-nearest neighbors algorithm]]. The assumption is that cases that are near each other tend to belong to the same class.