<p>The Lagrangian is not really symmetric; something that is easier to see if you formulate it without the calculus implementation. First-order conditions for maxima and minima might look similar, but maxima and minima are very different.</p>
<p>You have the function <span class="math-container">$f:\mathbb{R}^n\to\mathbb{R}$</span> you want to maximize, subject to the constraint (written using vector notation) that <span class="math-container">$G(x)\leq b$</span> for a function <span class="math-container">$G:\mathbb{R}^n\to\mathbb{R}^m$</span> and some <span class="math-container">$b\in\mathbb{R}^m$</span>. The Lagrangian <span class="math-container">$L:\mathbb{R}^n\times\mathbb{R}^m_+\to\mathbb{R}$</span> is given by <span class="math-container">$$L(x,\lambda)=f(x)+\lambda\cdot (b-G(x)).$$</span>
You can view the Lagrangian as the payoff-function of a zero-sum game. One player, the one controlling <span class="math-container">$x$</span>, wants to maximize the Lagrangian, and the other player, the one controlling <span class="math-container">$\lambda$</span>, wants to minimize the Lagrangian. The sufficiency conditions tell you that if you have an equilibrium of this game, then the payoff of the maximizer is the maximum of the optimization problem. The necessary conditions guarantee that an equilibrium exists and that a minimax theorem holds for this game:
<span class="math-container">$$\sup_{x\in\mathbb{R}^n}\inf_{\lambda\in\mathbb{R}^m_+}L(x,\lambda)=\inf_{\lambda\in\mathbb{R}^m_+}\sup_{x\in\mathbb{R}^n} L(x,\lambda).$$</span>
The sup becomes a max, the inf becomes a min, and the solutions are given by first-order conditions. Here is the idea of why this works: If the maximizer maximizes <span class="math-container">$f$</span> under the constraint, we get <span class="math-container">$b-G(x)\geq 0$</span>, and thus, for every <span class="math-container">$\lambda\geq 0,$</span> one has <span class="math-container">$\lambda\cdot (b-G(x))\geq 0$</span>. Since <span class="math-container">$\lambda=0$</span> is always possible for the minimizer, we must have <span class="math-container">$\lambda\cdot (b-G(x))= 0$</span>. However, if the maximizer would violate the constraint, there must be some coordinate <span class="math-container">$i=1,\ldots,m$</span> such that <span class="math-container">$b_i-G_i(x)&lt;0$</span>. Let <span class="math-container">$e_i\in\mathbb{R}^m$</span> be the vector with a <span class="math-container">$1$</span> in the <span class="math-container">$i$</span>th place, and all other coordinates <span class="math-container">$0$</span>. For <span class="math-container">$\lambda=C e_i$</span> with <span class="math-container">$C$</span> a large positive number, the Lagragian can be made arbitrarily small (negative). So the maximizer has to satisfy the constraint in order for the minimizer not to &quot;win.&quot; Duality tells you that it doesn't matter which player would move first, but it doesn't change the asymmetry between maximizing and minimizing. Of course you can rewrite the result to be one for minimization problems, which is exactly what you get by maximizing <span class="math-container">$-f$</span>.</p>

CC BY-SA 4.0
2024-04-28T10:39:52.573Z
Hi Michael thanks for this detailed answer. Apologies i have't accepted it yet, I'm in the middle of my final exams. I will read through in detail tomorrow - by the look of all the upvotes i'm sure it's perfect!
CormJack
CC BY-SA 4.0
2024-04-28T18:27:29.233Z
I second that statement by CormJack.
mark leeds
CC BY-SA 4.0
2024-04-29T10:19:22.403Z
@MichaelGreinecker Dear Michael, it is interesting the parallelism with zero-sum game, but what the matrix of the game would be (payoff)? Moreover, minimax theorem holds if x and y are both expressed as convex linear combinations (mixed strategies). For pure strategies, the minimax theorem is not valid in general.
marco tognoli
CC BY-SA 4.0
2024-04-29T12:40:05.763Z
@marcotognoli There is no payoff matrix, it is an infinite game, and equilibria are in pure strategies. The convexity assumptions for the necessity part of the KKT-theorem help to guarantee the existence of a saddle point.
Michael Greinecker
CC BY-SA 4.0
2024-04-23T10:02:37.027Z
2024-04-23T11:25:00.840Z
2024-04-23T11:25:00.840Z
Michael Greinecker
<p>My course always converts minimisation problems into maximisation. They give the following reason as outlined in the problem below.</p>
<blockquote>
<p><span class="math-container">$Min\; P_xx + P_yy \; s.t. \; u(x,y) \le x^{\frac{1}{2}} + y$</span></p>
<ul>
<li>&quot;<em><strong>In order to apply the Kuhn-Tucker theorem we can rewrite this problem as</strong></em>&quot;</li>
</ul>
</blockquote>
<p>They would write it as:</p>
<p><span class="math-container">$Max\; -(P_xx + P_yy) \; s.t. \; -u(x,y) \ge -(x^{\frac{1}{2}} + y)$</span></p>
<ol>
<li>As far as i'm aware it's not strictly necessary to do this, so long as our constraint and objective function are both convex (i.e. our Lagrangian is convex), then the KKT / Lagrange technique will find a minimum, just as it would find a maximum, and through duality, these will be the same value. So why the emphasis? I have suspicion that it could be to do with a subtle point regarding non-negativity constraints that they haven't elaborated.</li>
</ol>
<p><strong>Non negativity constraints:</strong> Assuming the possibility of <span class="math-container">$y = 0$</span> I have written the problem below both as a maximisation and a minimisation:</p>
<p>Max: <span class="math-container">$L(,x,t,λ) = -(P_xx + P_yy)  - λ[(\bar{u} -(x^{\frac{1}{2}} + y)] + μy$</span><br />
Min: <span class="math-container">$L(,x,t,λ) = (P_xx + P_yy)  + λ[(\bar{u} -(x^{\frac{1}{2}} + y)] - μy$</span></p>
<p>Taking the FOC with respect to y from the Maximisation problem we get:</p>
<p><span class="math-container">$\frac{\partial L}{\partial y} = -P_yy  + λ + m \le 0$</span> Which as <span class="math-container">$μ \ge 0$</span> implies <span class="math-container">$\frac{\partial L}{\partial y} = -P_yy  + λ \le 0$</span></p>
<ul>
<li><span class="math-container">$P_yy \ge λ$</span></li>
</ul>
<p><strong>And now here is my problem, taking the FOC with a respect to the Minimisation problem we get.</strong></p>
<p><span class="math-container">$\frac{\partial L}{\partial y} = P_yy  - λ - μ \le 0$</span></p>
<p><strong>Question:</strong> I believe that when we have a minimisation problem, and are testing a binding non-negativity constraint i.e. boundary solution on the axis, then the inequality in the FOC becomes <span class="math-container">$\ge 0$</span> Is this correct?</p>
<p>I.e. It should be: <span class="math-container">$\frac{\partial L}{\partial y} = P_yy  - λ - μ \ge 0$</span> Which as <span class="math-container">$μ \ge 0$</span> implies <span class="math-container">$\frac{\partial L}{\partial y} = P_yy  - λ \ge 0$</span></p>
<ul>
<li><span class="math-container">$P_yy \ge λ$</span></li>
</ul>
<p>This is the only way i can get it to make sense otherwise the two versions are giving different results.  This problem goes unnoticed when it's equality contained. And because my course has never written the problem formally as a minimisation problem i haven't been able to see what happens in this boundary case.</p>
<p>if i am correct can someone explain some economic / mathematical intuition behind the  <span class="math-container">$\ge 0$</span>.</p>
<p>Thanks!</p>
<p><strong>Update 23/04/2024</strong></p>
<p>To try and make clear what version of KKT (i think) i'm following as suggested by Michael, i have outlined the nuances in the comments of two different courses i have taken. For context the degree is self taught so i have very little outside help apart from the kindness of fellow enthusiasts like yourselves! It also means differences in methods are hard to distinguish between expedience of a particular module vs fundamental difference.</p>
<p>The images i am attaching here are screenshots from <a href="https://www.youtube.com/watch?v=2syXAny8PfE&amp;t=609s" rel="nofollow noreferrer">this</a> youtube lecture series by Mark Walker, which is the Econ PHD maths prep programme. It's been the most useful reference for me!</p>
<p><a href="https://i.sstatic.net/FKRkX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FKRkX.png" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/q7Y2v.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/q7Y2v.png" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2024-04-22T23:14:24.137Z
There are several versions of the KKM-Theorem. It would be helpful if you would state which one you are using.
Michael Greinecker
CC BY-SA 4.0
2024-04-23T07:13:40.150Z
That is it self really helpful. I’ve never been taught that there are several versions (I’ve just learnt what I felt was the same thing in some times different ways). Could you tell me their names or a reference and I will try and identify which I’m using, thank you!
CormJack
CC BY-SA 4.0
2024-04-23T07:15:24.107Z
Also I am referring to the KKT conditions for inequality constrained optimisation. Karush–Kuhn–Tucker. You mentioned KKM, I’m not sure if that’s a typo, or related concept, or perhaps alternate name?
CormJack
CC BY-SA 4.0
2024-04-23T07:19:17.373Z
Sorry, that was a misspelling. The various versions differ in the form of the constraint qualifications used, whether differentiabikity assumptions are made, what the domain is etc.
Michael Greinecker
CC BY-SA 4.0
2024-04-23T07:39:37.437Z
I see well the first thing to note is I have taken two optimisation modules (It's a Maths & Economics degree). 1) relied on showing whether constraint was quasi-convex/convex for a minimum (I learnt elsewhere because then the Lagrange is the sum of two convex functions and so is also convex) and concave for maximum...Although this was actually flexible depending on how one arranged the constraint and the sign before the Lagrange multiplier That course also implicitly used for a maximum the condition that if say $x_i = 0$ then $\frac{\partial L}{\partial x_i} ≤ 0$.
CormJack
CC BY-SA 4.0
2024-04-23T07:40:11.033Z
The equivalent condition for a minimum was never explicitly made which i am trying to uncover now. And i said "Implicitly" because it was never formerly stated, but in mark schemes it was clear the condition was used!
CormJack
CC BY-SA 4.0
2024-04-23T07:45:46.500Z
Course 2) never make any mention of the above condition instead always we have $\frac{\partial L}{\partial x_i} = 0$, this course also does not require us to check convexity/concavity. However this course does require us to check constraint qualifications. Specifically the LI of all combinations of effective (binding) constraints. Also the first course told us to treat minimums by just maximising the negative version of the problem. Course 2) just states we subtract the constraints (and add them for a maximum) however through a bit of algebra i think these are equivalent.
CormJack
CC BY-SA 4.0
2024-04-23T08:21:38.587Z
The fundamental method still seems identical, and it's not clear to me how much of these discrepancies in nuance is just oversight, expediency of the course, redundancy or subtly different methods. I have also attached screenshots of the youtube series for the Arizona Econ PHD prep programme that i found most helpful. It seems to be representative of the first course, i suppose this is the version i thought i was following: Link in body of text also youtube.com/watch?v=2syXAny8PfE&t=609s
CormJack
CC BY-SA 4.0
2024-04-23T09:39:24.150Z
I don't have an answer to your question but maybe look at a well respected textbook ?  Back in the pre-historic times, Bazarra and Shetty was quite popular. I'm not sure if it's still used in OR departments ? Nowadays there is a book by  Bertsimas and also a book by Boyd that are both quite popular but I can't comment on their exposition of KKT. Nocedal and Wright is another one that just came into my mind.
mark leeds
<mathematical-economics><utility><optimization><lagrangian>
Minimisation problem turned into Maximisation
CormJack
# Minimisation problem turned into Maximisation

(Asked by: username_0 on 2023-05-26)

My course always converts minimisation problems into maximisation. They give the following reason as outlined in the problem below.

> $Min\; P\_xx + P\_yy \; s.t. \; u(x,y) \le x^{\frac{1}{2}} + y$
>
> * "***In order to apply the Kuhn-Tucker theorem we can rewrite this problem as***"

They would write it as:

$Max\; -(P\_xx + P\_yy) \; s.t. \; -u(x,y) \ge -(x^{\frac{1}{2}} + y)$

1. As far as i'm aware it's not strictly necessary to do this, so long as our constraint and objective function are both convex (i.e. our Lagrangian is convex), then the KKT / Lagrange technique will find a minimum, just as it would find a maximum, and through duality, these will be the same value. So why the emphasis? I have suspicion that it could be to do with a subtle point regarding non-negativity constraints that they haven't elaborated.

**Non negativity constraints:** Assuming the possibility of $y = 0$ I have written the problem below both as a maximisation and a minimisation:

Max: $L(,x,t,λ) = -(P\_xx + P\_yy) - λ[(\bar{u} -(x^{\frac{1}{2}} + y)] + μy$  
Min: $L(,x,t,λ) = (P\_xx + P\_yy) + λ[(\bar{u} -(x^{\frac{1}{2}} + y)] - μy$

Taking the FOC with respect to y from the Maximisation problem we get:

$\frac{\partial L}{\partial y} = -P\_yy + λ + m \le 0$ Which as $μ \ge 0$ implies $\frac{\partial L}{\partial y} = -P\_yy + λ \le 0$

* $P\_yy \ge λ$

**And now here is my problem, taking the FOC with a respect to the Minimisation problem we get.**

$\frac{\partial L}{\partial y} = P\_yy - λ - μ \le 0$

**Question:** I believe that when we have a minimisation problem, and are testing a binding non-negativity constraint i.e. boundary solution on the axis, then the inequality in the FOC becomes $\ge 0$ Is this correct?

I.e. It should be: $\frac{\partial L}{\partial y} = P\_yy - λ - μ \ge 0$ Which as $μ \ge 0$ implies $\frac{\partial L}{\partial y} = P\_yy - λ \ge 0$

* $P\_yy \ge λ$

This is the only way i can get it to make sense otherwise the two versions are giving different results. This problem goes unnoticed when it's equality contained. And because my course has never written the problem formally as a minimisation problem i haven't been able to see what happens in this boundary case.

if i am correct can someone explain some economic / mathematical intuition behind the $\ge 0$.

Thanks!

**Update 23/04/2024**

To try and make clear what version of KKT (i think) i'm following as suggested by Michael, i have outlined the nuances in the comments of two different courses i have taken. For context the degree is self taught so i have very little outside help apart from the kindness of fellow enthusiasts like yourselves! It also means differences in methods are hard to distinguish between expedience of a particular module vs fundamental difference.

The images i am attaching here are screenshots from [this](https://www.youtube.com/watch?v=2syXAny8PfE&t=609s) youtube lecture series by Mark Walker, which is the Econ PHD maths prep programme. It's been the most useful reference for me!

[![enter image description here](https://i.sstatic.net/FKRkX.png)](https://i.sstatic.net/FKRkX.png)

[![enter image description here](https://i.sstatic.net/q7Y2v.png)](https://i.sstatic.net/q7Y2v.png)

**Tags:** mathematical-economics, utility, optimization, lagrangian
> **Comments:**
> **username_1**: There are several versions of the KKM-Theorem. It would be helpful if you would state which one you are using.  
> **username_0**: That is it self really helpful. I’ve never been taught that there are several versions (I’ve just learnt what I felt was the same thing in some times different ways). Could you tell me their names or a reference and I will try and identify which I’m using, thank you!  
> **username_0**: Also I am referring to the KKT conditions for inequality constrained optimisation. Karush–Kuhn–Tucker. You mentioned KKM, I’m not sure if that’s a typo, or related concept, or perhaps alternate name?  
> **username_1**: Sorry, that was a misspelling. The various versions differ in the form of the constraint qualifications used, whether differentiabikity assumptions are made, what the domain is etc.  
> **username_0**: I see well the first thing to note is I have taken two optimisation modules (It's a Maths & Economics degree). 1) relied on showing whether constraint was quasi-convex/convex for a minimum (I learnt elsewhere because then the Lagrange is the sum of two convex functions and so is also convex) and concave for maximum...Although this was actually flexible depending on how one arranged the constraint and the sign before the Lagrange multiplier That course also implicitly used for a maximum the condition that if say $x_i = 0$ then $\frac{\partial L}{\partial x_i} ≤ 0$.  
> **username_0**: The equivalent condition for a minimum was never explicitly made which i am trying to uncover now. And i said "Implicitly" because it was never formerly stated, but in mark schemes it was clear the condition was used!  
> **username_0**: Course 2) never make any mention of the above condition instead always we have $\frac{\partial L}{\partial x_i} = 0$, this course also does not require us to check convexity/concavity. However this course does require us to check constraint qualifications. Specifically the LI of all combinations of effective (binding) constraints. Also the first course told us to treat minimums by just maximising the negative version of the problem. Course 2) just states we subtract the constraints (and add them for a maximum) however through a bit of algebra i think these are equivalent.  
> **username_0**: The fundamental method still seems identical, and it's not clear to me how much of these discrepancies in nuance is just oversight, expediency of the course, redundancy or subtly different methods. I have also attached screenshots of the youtube series for the Arizona Econ PHD prep programme that i found most helpful. It seems to be representative of the first course, i suppose this is the version i thought i was following: Link in body of text also youtube.com/watch?v=2syXAny8PfE&t=609s  
> **username_2**: I don't have an answer to your question but maybe look at a well respected textbook ?  Back in the pre-historic times, Bazarra and Shetty was quite popular. I'm not sure if it's still used in OR departments ? Nowadays there is a book by  Bertsimas and also a book by Boyd that are both quite popular but I can't comment on their exposition of KKT. Nocedal and Wright is another one that just came into my mind.  

---

## 1 answers

### Answer by username_1 on 2024-04-23. Score: 9

The Lagrangian is not really symmetric; something that is easier to see if you formulate it without the calculus implementation. First-order conditions for maxima and minima might look similar, but maxima and minima are very different.

You have the function $f:\mathbb{R}^n\to\mathbb{R}$ you want to maximize, subject to the constraint (written using vector notation) that $G(x)\leq b$ for a function $G:\mathbb{R}^n\to\mathbb{R}^m$ and some $b\in\mathbb{R}^m$. The Lagrangian $L:\mathbb{R}^n\times\mathbb{R}^m\_+\to\mathbb{R}$ is given by $$L(x,\lambda)=f(x)+\lambda\cdot (b-G(x)).$$
You can view the Lagrangian as the payoff-function of a zero-sum game. One player, the one controlling $x$, wants to maximize the Lagrangian, and the other player, the one controlling $\lambda$, wants to minimize the Lagrangian. The sufficiency conditions tell you that if you have an equilibrium of this game, then the payoff of the maximizer is the maximum of the optimization problem. The necessary conditions guarantee that an equilibrium exists and that a minimax theorem holds for this game:
$$\sup\_{x\in\mathbb{R}^n}\inf\_{\lambda\in\mathbb{R}^m\_+}L(x,\lambda)=\inf\_{\lambda\in\mathbb{R}^m\_+}\sup\_{x\in\mathbb{R}^n} L(x,\lambda).$$
The sup becomes a max, the inf becomes a min, and the solutions are given by first-order conditions. Here is the idea of why this works: If the maximizer maximizes $f$ under the constraint, we get $b-G(x)\geq 0$, and thus, for every $\lambda\geq 0,$ one has $\lambda\cdot (b-G(x))\geq 0$. Since $\lambda=0$ is always possible for the minimizer, we must have $\lambda\cdot (b-G(x))= 0$. However, if the maximizer would violate the constraint, there must be some coordinate $i=1,\ldots,m$ such that $b\_i-G\_i(x)<0$. Let $e\_i\in\mathbb{R}^m$ be the vector with a $1$ in the $i$th place, and all other coordinates $0$. For $\lambda=C e\_i$ with $C$ a large positive number, the Lagragian can be made arbitrarily small (negative). So the maximizer has to satisfy the constraint in order for the minimizer not to "win." Duality tells you that it doesn't matter which player would move first, but it doesn't change the asymmetry between maximizing and minimizing. Of course you can rewrite the result to be one for minimization problems, which is exactly what you get by maximizing $-f$.

> **Comments:**
> **username_0**: Hi Michael thanks for this detailed answer. Apologies i have't accepted it yet, I'm in the middle of my final exams. I will read through in detail tomorrow - by the look of all the upvotes i'm sure it's perfect!  
> **username_2**: I second that statement by CormJack.  
> **username_3**: @MichaelGreinecker Dear Michael, it is interesting the parallelism with zero-sum game, but what the matrix of the game would be (payoff)? Moreover, minimax theorem holds if x and y are both expressed as convex linear combinations (mixed strategies). For pure strategies, the minimax theorem is not valid in general.  
> **username_1**: @marcotognoli There is no payoff matrix, it is an infinite game, and equilibria are in pure strategies. The convexity assumptions for the necessity part of the KKT-theorem help to guarantee the existence of a saddle point.  

---

