Editing Support vector machine (section)

=== Regression ===
[[Image:Svr epsilons demo.svg|thumbnail|right|Support vector regression (prediction) with different thresholds ''ε''. As ''ε'' increases, the prediction becomes less sensitive to errors.]]
A version of SVM for [[regression analysis|regression]] was proposed in 1996 by [[Vladimir N. Vapnik]], Harris Drucker, Christopher J. C. Burges, Linda Kaufman and Alexander J. Smola.<ref>Drucker, Harris; Burges, Christ. C.; Kaufman, Linda; Smola, Alexander J.; and Vapnik, Vladimir N. (1997); "[http://papers.nips.cc/paper/1238-support-vector-regression-machines.pdf Support Vector Regression Machines]", in ''Advances in Neural Information Processing Systems 9, NIPS 1996'', 155–161, MIT Press.</ref> This method is called support vector regression (SVR). The model produced by support vector classification (as described above) depends only on a subset of the training data, because the cost function for building the model does not care about training points that lie beyond the margin. Analogously, the model produced by SVR depends only on a subset of the training data, because the cost function for building the model ignores any training data close to the model prediction. Another SVM version known as [[least-squares support vector machine]] (LS-SVM) has been proposed by Suykens and Vandewalle.<ref>Suykens, Johan A. K.; Vandewalle, Joos P. L.; "[https://lirias.kuleuven.be/bitstream/123456789/218716/2/Suykens_NeurProcLett.pdf Least squares support vector machine classifiers]", ''Neural Processing Letters'', vol. 9, no. 3, Jun. 1999, pp. 293–300.</ref>

Training the original SVR means solving<ref>{{cite journal |last1=Smola |first1=Alex J. |first2=Bernhard |last2=Schölkopf |title=A tutorial on support vector regression |journal=Statistics and Computing |volume=14 |issue=3 |year=2004 |pages=199–222 |url=http://eprints.pascal-network.org/archive/00000856/01/fulltext.pdf |url-status=live |archive-url=https://web.archive.org/web/20120131193522/http://eprints.pascal-network.org/archive/00000856/01/fulltext.pdf |archive-date=2012-01-31 |doi=10.1023/B:STCO.0000035301.49549.88 |citeseerx=10.1.1.41.1452 |s2cid=15475 }}</ref>

: minimize <math>\tfrac{1}{2} \|w\|^2 </math>
: subject to <math> | y_i - \langle w, x_i \rangle  - b | \le \varepsilon </math>

where <math>x_i</math> is a training sample with target value <math>y_i</math>. The inner product plus intercept <math>\langle w, x_i \rangle + b</math> is the prediction for that sample, and <math>\varepsilon</math> is a free parameter that serves as a threshold: all predictions have to be within an <math>\varepsilon</math> range of the true predictions. Slack variables are usually added into the above to allow for errors and to allow approximation in the case the above problem is infeasible.