Part 8 - Lagrange multipliers
Introduction
In this part we’ll cover the method of Lagrange multipliers. This is a strategy, for finding local minimum/maximums for a function with a constraint.
Let’s define it properly
Definition
Say we have a function, $f(x, y)$, which we want to find the min/max points. But this function is subject of some constraint of the form, $g(x, y) = k$.
Recall our rectangular box problem from the last part: $$ V(x, y, z) = xyz \newline 2zx + 2zy + yx = 12 $$
So if it’s difficult to try to rewrite this using only two variables, we can instead use this method.
Let’s first get a geometrical sense.
Geometrical sense of Lagrange multipliers
If min/max of $f$, is subject to constraint, $g(x, y) = k$, which occurs at $(x_0, y_0)$, then the level curve containing $(x_0, y_0)$ must touch the constraint curve.
If two curves are touching at $(x_0, y_0)$, in other words they have a common tangent line at $(x_0, y_0)$.
Which is just equivalent of saying that they have a common perpendicular line at $(x_0, y_0)$, and what do we know which is orthogonal to the level curve?
The gradient.
Therefore, this means: $$ \nabla f(x_0, y_0) || \nabla g(x_0, y_0) $$
If two vectors are parallel, it’s equivalent of saying:
$$ \boxed{\nabla f(x_0, y_0) || \lambda \nabla g(x_0, y_0)} $$
All of this assuming that $\nabla g \neq 0$
Let’s properly define our method now.
Lagrange multiplier method
To find minimum/maximum values of function, $f$, subject to constraint, $g(x, y) = k$, assuming minimum/maximum points exists and $\nabla g \neq 0$
-
Find all numbers $x, y$ and $\lambda$, such that: $$ \nabla f(x, y) = \lambda \nabla g(x, y) = k $$
-
Compute values from step 1, choose the minimum/maximum value(s).
Let’s do an example.
Example
Find the extreme values of, $f(x, y) = xy$ on the disc, $x^2 + y^2 \leq 1$.
Step 1 - Find critical points (as usual): $$ \nabla f = 0 \newline \langle y, x \rangle = 0 $$
Which means: $$ \begin{cases} y = 0 \newline x = 0 \end{cases} $$
So we have a critical point at $(0, 0)$. Let’s compute the value: $$ f(0, 0) = \boxed{0} $$
Step 2 - Find absolute minimum/maximum values on boundary.
Since we have a unit circle, with equation, $x^2 + y^2 = 1$, we can view this as a constraint. $$ g(x, y) = k \newline x^2 + y^2 = 1 $$
Let’s first see if all our assumptions hold, since this is a bounded closed set, by the extreme value theorem, absolute values must exists. So we know that they exists.
Let’s check that $\nabla g \neq 0$: $$ \nabla g = \langle 2x, 2y \rangle $$
Which is only $0$ at $(0, 0)$ which isn’t on the boundary of the circle.
So we can try our new method!
$$ \begin{cases} \langle y, x \rangle = \lambda \langle 2x, 2y \rangle \newline x^2 + y^2 = 1 \end{cases} $$
So we have a vector that must be equal to another vector, this means they share coordinates.
Which means: $$ \begin{cases} y = 2\lambda x \newline x = 2\lambda y \newline x^2 + y^2 = 1 \end{cases} $$
substitute and simplify: $$ \begin{cases} x = 4\lambda^2 x \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases} $$
$$ \begin{cases} x - 4\lambda^2 x = 0 \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases} $$
$$ \begin{cases} x(1 - 4\lambda^2) = 0 \newline y = 2\lambda x \newline x^2 + y^2 = 1 \end{cases} $$
Case 1, $x = 0$, therefore, $y = 0$ as well, but our constraint is not fulfilled then.
Case 2, $1 - 4\lambda^2 = 0$: $$ \lambda = \pm \dfrac{1}{2} $$
Case 2.1, $\lambda = \dfrac{1}{2}$ $$ \begin{cases} y = x x = y x^2 + y^2 = 1 \end{cases} $$
$$ 2x^2 = 1 \newline x = \pm \dfrac{1}{\sqrt{2}} \newline y = \pm \dfrac{1}{\sqrt{2}} \newline $$
The constraint holds, so we have found another point, $\left( \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right)$ and $\left( - \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right)$.
Case 2.2, $\lambda = - \dfrac{1}{2}$ $$ \begin{cases} y = -x x = -y x^2 + y^2 = 1 \end{cases} $$
$$ 2x^2 = 1 \newline x = \pm \dfrac{1}{\sqrt{2}} \newline y = \pm \dfrac{1}{\sqrt{2}} \newline $$
The constraint holds, $\left( \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right)$ and $\left( - \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right)$.
Now, we just compute the value of these points: $$ f\left( \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right) = \dfrac{1}{2} \newline f\left( - \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right) = \dfrac{1}{2} \newline f\left( \dfrac{1}{\sqrt{2}}, - \dfrac{1}{\sqrt{2}} \right) = - \dfrac{1}{2} \newline f\left( - \dfrac{1}{\sqrt{2}}, \dfrac{1}{\sqrt{2}} \right) = - \dfrac{1}{2} $$
So, we have finally found our minimum and maximum values, $- \dfrac{1}{2}$ and $\dfrac{1}{2}$, respectively.
Remark
A little remark, it is important that the constraint has to be of the form $g(x, y) = k$. For example, our rectangle problem from last time, we can not describe a rectangular box in that form.
So no lagrange multiplier there. Let’s look at more difficult example.
Example
Find the points on the sphere, $x^2 + y^2 + z^2 = 4$, that are closest and furthest from the point $(3, 1, -1)$.
So we have to calculate the distance between these points, or in other words: $$ d(x, y, z) = \sqrt{(x - 3)^2 + (y - 1)^2 + (z + 1)^2} $$
But this function isn’t really that pleasant, so instead, let’s calculate the distance squared, or: $$ d(x, y, z) = (x - 3)^2 + (y - 1)^2 + (z + 1)^2 $$
We can work with this function instead, so let’s use our method. Let’s first compute gradients: $$ \nabla d = \langle 2(x - 3), 2(y - 1), 2(z + 1) \rangle $$
$$ \nabla g = \langle 2x, 2y, 2z \rangle $$
So we have: $$ \begin{cases} \langle 2(x - 3), 2(y - 1), 2(z + 1) \rangle = \lambda \langle 2x, 2y, 2z \rangle \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
$$ \begin{cases} 2(x - 3) = 2\lambda x \newline 2(y - 1) = 2\lambda y \newline 2(z + 1) = 2\lambda z \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
$$ \begin{cases} x - 3 = \lambda x \newline y - 1 = \lambda y \newline z + 1 = \lambda z \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
$$ \begin{cases} x - \lambda x = 3 \newline y - \lambda y = 1 \newline z - \lambda z = -1 \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
$$ \begin{cases} x(1 - \lambda) = 3 \newline y(1 - \lambda) = 1 \newline z(1 - \lambda) = -1 \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
$$ \begin{cases} x = \dfrac{3}{1 - \lambda} \newline y = \dfrac{1}{1 - \lambda} \newline z = \dfrac{-1}{1 - \lambda} \newline x^2 + y^2 + z^2 = 4 \end{cases} $$
Let’s substitute these into our constraint: $$ \dfrac{9}{(1 - \lambda)^2} + \dfrac{1}{(1 - \lambda)^2} + \dfrac{1}{(1 - \lambda)^2} = 4 $$
$$ \dfrac{11}{(1 - \lambda)^2} = 4 $$
$$ (1 - \lambda)^2 = \dfrac{11}{4} $$
$$ 1 - \lambda = \pm \dfrac{\sqrt{11}}{2} $$
Compute vales of $x, y, z$, in case 1, $1 - \lambda = \dfrac{\sqrt{11}}{2}$: $$ \begin{cases} x = \dfrac{6}{\sqrt{11}} \newline y = \dfrac{2}{\sqrt{11}} \newline z = -\dfrac{2}{\sqrt{11}} \newline \end{cases} $$
Compute vales of $x, y, z$, in case 2, $1 - \lambda = -\dfrac{\sqrt{11}}{2}$: $$ \begin{cases} x = -\dfrac{6}{\sqrt{11}} \newline y = -\dfrac{2}{\sqrt{11}} \newline z = \dfrac{2}{\sqrt{11}} \newline \end{cases} $$
Now, we could plug these values into our $d$ function, however we can intuitively see which point is closest and furthest.
Case 1 is the closest and case 2 is the furthest.