Editing Reinforcement learning (section)

==== Function approximation methods ====
In order to address the fifth issue, ''function approximation methods'' are used. ''Linear function approximation'' starts with a mapping <math>\phi</math> that assigns a finite-dimensional vector to each state-action pair. Then, the action values of a state-action pair <math>(s,a)</math> are obtained by linearly combining the components of <math>\phi(s,a)</math> with some ''weights'' <math>\theta</math>:

:<math>Q(s,a) = \sum_{i=1}^d \theta_i \phi_i(s,a).</math>

The algorithms then adjust the weights, instead of adjusting the values associated with the individual state-action pairs. Methods based on ideas from [[nonparametric statistics]] (which can be seen to construct their own features) have been explored.

Value iteration can also be used as a starting point, giving rise to the [[Q-learning]] algorithm and its many variants.<ref>{{cite thesis
| last = Watkins | first = Christopher J.C.H. | author-link = Christopher J.C.H. Watkins
| degree= PhD
| title= Learning from Delayed Rewards
| year= 1989
| publisher = King's College, Cambridge, UK
| url= http://www.cs.rhul.ac.uk/~chrisw/new_thesis.pdf}}</ref> Including Deep Q-learning methods when a neural network is used to represent Q, with various applications in stochastic search problems.<ref name="MBK">{{Cite journal |title = Detection of Static and Mobile Targets by an Autonomous Agent with Deep Q-Learning Abilities | journal=Entropy | year=2022 | volume=24 | issue=8 | page=1168  | doi=10.3390/e24081168 | pmid=36010832 | pmc=9407070 | bibcode=2022Entrp..24.1168M | doi-access=free | last1=Matzliach | first1=Barouch | last2=Ben-Gal | first2=Irad | last3=Kagan | first3=Evgeny }}</ref>

The problem with using action-values is that they may need highly precise estimates of the competing action values that can be hard to obtain when the returns are noisy, though this problem is mitigated to some extent by temporal difference methods. Using the so-called compatible function approximation method compromises generality and efficiency.