Editing Lagrange multiplier (section)

=== Example 3 ===
[[Image:Lagrange simple.svg|thumb|right|300px| Illustration of constrained optimization problem&nbsp;'''3'''.]]

This example deals with more strenuous calculations, but it is still a single constraint problem.

Suppose one wants to find the maximum values of
<math display="block"> f(x, y) = x^2 y</math>
with the condition that the <math>\ x\ </math>- and <math>\ y\ </math>-coordinates lie on the circle around the origin with radius <math>\ \sqrt{3\ } ~.</math> That is, subject to the constraint
<math display="block"> g(x,y) = x^2 + y^2 - 3 = 0 ~.</math>

As there is just a single constraint, there is a single multiplier, say <math>\ \lambda ~.</math>

The constraint <math>\ g(x,y)\ </math> is identically zero on the circle of radius <math>\ \sqrt{3\ } ~.</math> Any multiple of <math>\ g(x,y)\ </math> may be added to <math>\ g(x,y)\ </math> leaving <math>\ g(x,y)\ </math> unchanged in the region of interest (on the circle where our original constraint is satisfied).

Applying the ordinary Lagrange multiplier method yields
<math display="block">\begin{align}
\mathcal{L}(x, y, \lambda) &= f(x,y) + \lambda \cdot g(x, y) \\
&= x^2y +  \lambda (x^2 + y^2 - 3)\ ,
\end{align}</math>
from which the gradient can be calculated:
<math display="block">\begin{align}
\nabla_{x,y,\lambda} \mathcal{L}(x , y, \lambda) &= \left ( \frac{\partial \mathcal{L}}{\partial x}, \frac{\partial \mathcal{L}}{\partial y}, \frac{\partial \mathcal{L}}{\partial \lambda} \right) \\
&= \left ( 2 x y + 2 \lambda x, x^2 + 2 \lambda y, x^2 + y^2 -3 \right) ~.
\end{align}</math>
And therefore:
<math display="block">\nabla_{x,y,\lambda} \mathcal{L}(x , y, \lambda)=0 \quad \iff \quad \begin{cases}
2 x y + 2 \lambda x = 0 \\
x^2 + 2 \lambda y = 0 \\
x^2 + y^2 - 3 = 0
\end{cases} \quad \iff \quad \begin{cases}
x (y + \lambda) = 0 & \text{(i)} \\
x^2 = -2 \lambda y & \text{(ii)} \\
x^2 + y^2 = 3 & \text{(iii)}
\end{cases} </math>
(iii) is just the original constraint. (i) implies <math>\ x=0\ </math> or <math>\ \lambda=-y ~.</math> If <math>x=0</math> then <math>\ y = \pm \sqrt{3\ }\ </math> by (iii) and consequently  <math>\ \lambda=0\ </math> from (ii). If <math>\ \lambda=-y\ ,</math> substituting this into (ii) yields <math>\ x^2 = 2y^2 ~.</math> Substituting this into (iii) and solving for <math>\ y\ </math> gives <math>\ y=\pm1 ~.</math> Thus there are six critical points of <math>\ \mathcal{L}\ :</math>
<math display="block"> (\sqrt{2\ },1,-1); \quad (-\sqrt{2\ },1,-1); \quad (\sqrt{2\ },-1,1); \quad (-\sqrt{2\ },-1,1); \quad (0,\sqrt{3\ }, 0); \quad (0,-\sqrt{3\ }, 0) ~.</math>

Evaluating the objective at these points, one finds that
<math display="block"> f(\pm\sqrt{2\ },1) = 2; \quad f(\pm\sqrt{2\ },-1) = -2; \quad f(0,\pm \sqrt{3\ }) = 0 ~.</math>

Therefore, the  objective function attains the [[global maximum]] (subject to the constraints) at <math>\ (\pm\sqrt{2\ },1\ )</math> and the [[global minimum]] at <math>\ (\pm\sqrt{2\ },-1) ~.</math> The point <math>\ (0,\sqrt{3\ })\ </math> is a [[local minimum]] of <math>\ f\ </math> and <math>\ (0,-\sqrt{3\ })\ </math> is a [[local maximum]] of <math>\ f\ ,</math> as may be determined by consideration of the [[Hessian (mathematics)#Bordered Hessian|Hessian matrix]] of <math>\ \mathcal{L}(x,y,0) ~.</math>

Note that while <math>\ (\sqrt{2\ }, 1, -1)\ </math> is a critical point of <math>\ \mathcal{L}\ ,</math> it is not a local extremum of <math>\ \mathcal{L} ~.</math> We have
<math display="block">\mathcal{L} \left (\sqrt{2\ } + \varepsilon, 1, -1 + \delta \right ) = 2 + \delta \left( \varepsilon^2 + \left (2\sqrt{2\ } \right)\varepsilon \right) ~.</math>

Given any neighbourhood of <math>\ (\sqrt{2\ }, 1, -1)\ ,</math> one can choose a small positive <math>\ \varepsilon\ </math> and a small <math>\ \delta\ </math> of either sign to get <math>\ \mathcal{L}</math> values both greater and less than <math>\ 2 ~.</math> This can also be seen from the Hessian matrix of <math>\ \mathcal{L}\ </math> evaluated at this point (or indeed at any of the critical points) which is an [[indefinite matrix]]. Each of the critical points of <math>\ \mathcal{L}\ </math> is a [[saddle point]] of <math>\ \mathcal{L} ~.</math><ref name=Walsh1975/>