Editing Support vector machine (section)

==== Target functions ====
The difference between the hinge loss and these other loss functions is best stated in terms of ''target functions -'' the function that minimizes expected risk for a given pair of random variables <math>X,\,y</math>.

In particular, let <math>y_x</math> denote <math>y</math> conditional on the event that <math>X = x</math>.  In the classification setting, we have:

<math display="block">y_x = \begin{cases} 1 & \text{with probability } p_x \\ -1 & \text{with probability } 1-p_x  \end{cases}</math>

The optimal classifier is therefore:

<math display="block">f^*(x) = \begin{cases}1 & \text{if }p_x \geq 1/2 \\ -1 & \text{otherwise}\end{cases} </math>

For the square-loss, the target function is the conditional expectation function, <math>f_{sq}(x) = \mathbb{E}\left[y_x\right]</math>; For the logistic loss, it's the logit function, <math>f_{\log}(x) = \ln\left(p_x / ({1-p_x})\right)</math>. While both of these target functions yield the correct classifier, as <math>\sgn(f_{sq}) = \sgn(f_\log) = f^*</math>, they give us more information than we need. In fact, they give us enough information to completely describe the distribution of <math> y_x</math>.

On the other hand, one can check that the target function for the hinge loss is ''exactly'' <math>f^*</math>. Thus, in a sufficiently rich hypothesis space—or equivalently, for an appropriately chosen kernel—the SVM classifier will converge to the simplest function (in terms of <math>\mathcal{R}</math>) that correctly classifies the data. This extends the geometric interpretation of SVM—for linear classification, the empirical risk is minimized by any function whose margins lie between the support vectors, and the simplest of these is the max-margin classifier.<ref>{{Cite journal |title=Are Loss Functions All the Same? |url=https://ieeexplore.ieee.org/document/6789841 |journal=Neural Computation |date=2004-05-01 |issn=0899-7667 |pages=1063–1076 |volume=16 |issue=5 |doi=10.1162/089976604773135104 |pmid=15070510 |first1=Lorenzo |last1=Rosasco |first2=Ernesto |last2=De Vito |first3=Andrea |last3=Caponnetto |first4=Michele |last4=Piana |first5=Alessandro |last5=Verri |citeseerx=10.1.1.109.6786 |s2cid=11845688 }}</ref>