Editing Neural network (machine learning) (section)

===Other===
In a [[Bayesian probability|Bayesian]] framework, a distribution over the set of allowed models is chosen to minimize the cost. [[Evolutionary methods]],<ref>{{cite conference |author1=de Rigo, D. |author2=Castelletti, A. |author3=Rizzoli, A. E. |author4=Soncini-Sessa, R. |author5=Weber, E. |date=January 2005 |title=A selective improvement technique for fastening Neuro-Dynamic Programming in Water Resources Network Management |conference=16th IFAC World Congress |publisher=IFAC |location=Prague, Czech Republic |conference-url=http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Index.html |book-title=Proceedings of the 16th IFAC World Congress – IFAC-PapersOnLine |editor=Pavel Zítek |volume=16 |pages=7–12 |url=http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Papers/Paper4269.html |access-date=30 December 2011 |doi=10.3182/20050703-6-CZ-1902.02172 |isbn=978-3-902661-75-3 |hdl=11311/255236 |hdl-access=free |archive-date=26 April 2012 |archive-url=https://web.archive.org/web/20120426012450/http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Papers/Paper4269.html |url-status=live }}</ref> [[gene expression programming]],<ref>{{cite book |last=Ferreira |first=C. |year=2006 |contribution=Designing Neural Networks Using Gene Expression Programming |url=http://www.gene-expression-programming.com/webpapers/Ferreira-ASCT2006.pdf |editor=A. Abraham |editor2=B. de Baets |editor3=M. Köppen |editor4=B. Nickolay |title=Applied Soft Computing Technologies: The Challenge of Complexity |pages=517–536 |publisher=Springer-Verlag |access-date=8 October 2012 |archive-date=19 December 2013 |archive-url=https://web.archive.org/web/20131219022806/http://www.gene-expression-programming.com/webpapers/Ferreira-ASCT2006.pdf |url-status=live }}</ref> [[simulated annealing]],<ref>{{cite conference |author=Da, Y. |author2=Xiurun, G. |date=July 2005 |title=An improved PSO-based ANN with simulated annealing technique |volume=63 |pages=527–533 |editor=T. Villmann |book-title=New Aspects in Neurocomputing: 11th European Symposium on Artificial Neural Networks |url=http://www.dice.ucl.ac.be/esann/proceedings/electronicproceedings.htm |publisher=Elsevier |doi=10.1016/j.neucom.2004.07.002 |access-date=30 December 2011 |archive-date=25 April 2012 |archive-url=https://web.archive.org/web/20120425233611/http://www.dice.ucl.ac.be/esann/proceedings/electronicproceedings.htm |url-status=dead }}</ref> [[expectation–maximization algorithm|expectation–maximization]], [[non-parametric methods]] and [[particle swarm optimization]]<ref>{{cite conference |author=Wu, J. |author2=Chen, E. |date=May 2009 |title=A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network |series=Lecture Notes in Computer Science |volume=5553 |pages=49–58 |book-title=6th International Symposium on Neural Networks, ISNN 2009 |url=http://www2.mae.cuhk.edu.hk/~isnn2009/ |editor=Wang, H. |editor2=Shen, Y. |editor3=Huang, T. |editor4=Zeng, Z. |publisher=Springer |doi=10.1007/978-3-642-01513-7_6 |isbn=978-3-642-01215-0 |access-date=1 January 2012 |archive-date=31 December 2014 |archive-url=https://web.archive.org/web/20141231221755/http://www2.mae.cuhk.edu.hk/~isnn2009/ |url-status=dead }}</ref> are other learning algorithms. Convergent recursion is a learning algorithm for [[cerebellar model articulation controller]] (CMAC) neural networks.<ref name="Qin1">{{cite journal |author1=Ting Qin |author2=Zonghai Chen |author3=Haitao Zhang |author4=Sifu Li |author5=Wei Xiang |author6=Ming Li |url=http://www-control.eng.cam.ac.uk/Homepage/papers/cued_control_998.pdf |title=A learning algorithm of CMAC based on RLS |journal=Neural Processing Letters |volume=19 |issue=1 |date=2004 |pages=49–61 |doi=10.1023/B:NEPL.0000016847.18175.60 |s2cid=6233899 |access-date=30 January 2019 |archive-date=14 April 2021 |archive-url=https://web.archive.org/web/20210414103815/http://www-control.eng.cam.ac.uk/Homepage/papers/cued_control_998.pdf |url-status=live }}</ref><ref name="Qin2">{{cite journal |author1=Ting Qin |author2=Haitao Zhang |author3=Zonghai Chen |author4=Wei Xiang |url=http://www-control.eng.cam.ac.uk/Homepage/papers/cued_control_997.pdf |title=Continuous CMAC-QRLS and its systolic array |journal=Neural Processing Letters |volume=22 |issue=1 |date=2005 |pages=1–16 |doi=10.1007/s11063-004-2694-0 |s2cid=16095286 |access-date=30 January 2019 |archive-date=18 November 2018 |archive-url=https://web.archive.org/web/20181118122850/http://www-control.eng.cam.ac.uk/Homepage/papers/cued_control_997.pdf |url-status=live }}</ref>

==== Modes ====
{{No footnotes|date=August 2019|section}}
Two modes of learning are available: stochastic and batch. In stochastic learning, each input creates a weight adjustment. In batch learning weights are adjusted based on a batch of inputs, accumulating errors over the batch. Stochastic learning introduces "noise" into the process, using the local gradient calculated from one data point; this reduces the chance of the network getting stuck in local minima. However, batch learning typically yields a faster, more stable descent to a local minimum, since each update is performed in the direction of the batch's average error. A common compromise is to use "mini-batches", small batches with samples in each batch selected stochastically from the entire data set.