<p>The main two tools for economists solving infinite-horizon constrained optimization problems in discrete-time, as in your example problem, are:</p>
<ol>
<li>Karush–Kuhn–Tucker (KKT) conditions - which is a variational method, as you noticed, and work for linear and non-linear problems.</li>
<li>Dynamic programming - which can also be used to solve linear and non-linear problems, as well.</li>
</ol>
<p>Bellman's Principle of Optimality gives conditions for the the former approach (sequence problem - SP) to give the same solution as the latter (functional equation problem - FP), that is, SP=FE.</p>
<p>When looking for an analytical solution the former approach is often more convenient. It is often natural to write a problem as a sequence problem and the first-derivative approach does not require knowledge about the functional form of the solution or potentially tedious iteration.</p>
<p>However, the second approach has advantages when one wants to take advantage of a computer to solve a potentially more complicated problem with the use of a computer. While multiple state variables makes solving dynamic programming problems more computationally intensive, this has not deterred many economists and many notable economic models are highly dimensional (see Krusell and Smith (1998) for an early example). Deep-learning methods are also being used to speed up the solving of these large-scale problems.</p>
<p>Another benefit of the dynamic programming approach is that it is easier to adapt to uncertainty. An example given <a href="http://faculty.econ.ucdavis.edu/faculty/kdsalyer/LECTURES/Ecn200e/dynprog.pdf" rel="nofollow noreferrer">here</a>.</p>
<p>Lastly, it is possible to extend your constrained optimization problem to a greater degree of complexity, such that you may want to use a more advanced technique, such as the Kalman Filter. An example of this may be a problem in which the <a href="https://python.quantecon.org/kalman.html" rel="nofollow noreferrer">state variable is unobserved to the agent</a>.</p>
<p>Note: there exist continuous-time substitutes to these methods (Pontryagin's Maximum Principle and the Hamiltonian-Jacobi-Bellman equations).</p>

CC BY-SA 4.0
2022-07-20T17:53:21.257Z
thanks for you response, it was very helpful. The thing that I am still confused about is the idea of nonlinear system. So if the model for the dynamics or utility function is nonlinear--which is often the case--then what do you do? If you want to use a Linear Quadratic Control method, the you would need to linearize around the fixed point right? But then the control method would only optimize the consumption and labor vectors to remain at that fixed point. That is the way that optimal control would work in applied math world, but I am not sure how Econ models deal with that.
krishnab
CC BY-SA 4.0
2022-07-20T17:59:57.340Z
I guess part of the question is whether the objective function $u(c_t, h_t)$ has only one fixed point, or whether it has more than one fixed point? If there is only one fixed point, then linearizing around that fixed point makes sense. But if there are more than one fixed point, then linear control will not help to design a control law to move from one fixed point to another.
krishnab
CC BY-SA 4.0
2022-07-21T03:25:13.220Z
There is no problem with the utility being nonlinear, as long as we have monotonicity and discounting (Blackwell’s sufficiency conditions) the function we are solving will be a contraction mapping and have a unique fixed point. For a primer on how to solve these problems I recommend: https://researchspace.auckland.ac.nz/bitstream/handle/2292/190/230.pdf?sequence=1&isAllowed=y
Fića
CC BY-SA 4.0
2022-07-21T03:26:27.203Z
If you are looking for a more thorough treatment of *why* these methods work you could refer to a more advanced source, such as: Stokey, N., Lucas, R.E., with Prescott, E., Recursive Methods for Economic Dynamics, Harvard University Press (1989)
Fića
CC BY-SA 4.0
2022-07-19T22:01:21.430Z
2022-07-19T22:27:54.793Z
2022-07-19T22:27:54.793Z
Fića
<p>I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics concepts, but I have a pretty good grasp of linear and nonlinear optimal control methods.</p>
<p>My fundamental question is <strong>when do economists use variational methods to solve DSGE models, versus using dynamic programming or linear optimal control methods, versus using nonlinear optimal control methods?</strong> It seems like economists use all of the above methods, but I was not sure when and why they choose one method over the other.</p>
<p>I was looking at some <a href="https://www.youtube.com/watch?v=sOG3YW0iCQg&amp;list=PLsZ8kVwX52ZHewGr0jLceiyCazP0GWhoB" rel="noreferrer">videos</a> that present a nice simple DSGE RBC model, and solve it using variational methods--meaning the Lagrange multiplier method. The solution is analytical and derived first order conditions for the system like I learned as an undergraduate. The basic problem they are working on is the infinite and finite time horizon optimization problem:</p>
<p><span class="math-container">$$
max_{c, h} \quad \mathbb{E}(\sum^{\infty}_{t=0} \beta^tu(c_t, h_t))
$$</span></p>
<p>In this case, <span class="math-container">$c_t$</span> is the consumption at time <span class="math-container">$t$</span>, and <span class="math-container">$h_t$</span> is the labor provided at time <span class="math-container">$t$</span>. Of course we can add budget constraints and such to this problem, as the author did in the videos.</p>
<p>However, when looking at the McCandless &quot;ABCs of RBCs&quot; book, the author in chapter 4 uses dynamic programming methods. Now dynamic programming methods like value iteration don't work for high dimensional problems--meaning that they are too computationally expensive.</p>
<p>I could not tell if McCandless gets into linear quadratic control methods and Kalman filters to solve the optimization problem above--I did not find references to &quot;Kalman&quot; in the index. However, Kalman filter methods and nonlinear control methods can be used to solve the objective function--especially since that function <span class="math-container">$u(\cdot)$</span> is usually nonlinear.</p>
<p>Hence I was hoping that someone could explain how economists choose which method to solve the optimization problem above.</p>

CC BY-SA 4.0
2022-07-19T11:15:18.640Z
What exactly do you mean by *solving* DSGE models? Are you always assuming parameters are known, or do they have to be estimated?
BrsG
CC BY-SA 4.0
2022-07-19T11:16:11.213Z
Are you talking about solving these models analytically on paper or actually programming the model? With analytical solutions people just use what they are most comfortable with, if you are talking about programming then its a method that is sufficiently quick enough and easiest to apply
1muflon1
CC BY-SA 4.0
2022-07-19T13:59:54.227Z
@BrsG By solving I mean estimating the consumption and labor vectors $c_t, h_t$. In a real world problem, I would probably look to estimate the control variables. By estimation, mean the potentially recursive optimization of those vectors. Estimating the parameters is important, but there are well known methods for that: I can use gradient descent to minimize the squared loss between the model predictions and the ground truth data, plus any additional constraints I want to impose on the solution. First I just want to efficiently estimate those two vectors.
krishnab
CC BY-SA 4.0
2022-07-19T14:03:31.193Z
@1muflon1 I am thinking about computational or numerical solutions. The variational method is analytical, but requires some conditions to work--the existence of the first derivatives, etc. Like I was saying, I wanted to precisely understand what I gain by using more complex DP or Kalman or even nonlinear control methods? Do I gain accuracy because I can estimate more complex functions? Do I have less variance in the computed solution versus the analytical solution? Or are there other issues as well?
krishnab
<optimization><dsge><dynamic-programming><optimal-control><recursive-macroeconomics>
When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models
krishnab
# When does it make sense to use variational methods, versus dynamic programming, versus nonlinear control methods so solve DSGE models

(Asked by: username_0 on 2022-07-19)

I come from a statistics and applied math background, but have been looking at some problems related to macroeconomic DSGE models lately. So I am still trying to understand the ideas and economics concepts, but I have a pretty good grasp of linear and nonlinear optimal control methods.

My fundamental question is **when do economists use variational methods to solve DSGE models, versus using dynamic programming or linear optimal control methods, versus using nonlinear optimal control methods?** It seems like economists use all of the above methods, but I was not sure when and why they choose one method over the other.

I was looking at some [videos](https://www.youtube.com/watch?v=sOG3YW0iCQg&list=PLsZ8kVwX52ZHewGr0jLceiyCazP0GWhoB) that present a nice simple DSGE RBC model, and solve it using variational methods--meaning the Lagrange multiplier method. The solution is analytical and derived first order conditions for the system like I learned as an undergraduate. The basic problem they are working on is the infinite and finite time horizon optimization problem:

$$
max\_{c, h} \quad \mathbb{E}(\sum^{\infty}\_{t=0} \beta^tu(c\_t, h\_t))
$$

In this case, $c\_t$ is the consumption at time $t$, and $h\_t$ is the labor provided at time $t$. Of course we can add budget constraints and such to this problem, as the author did in the videos.

However, when looking at the McCandless "ABCs of RBCs" book, the author in chapter 4 uses dynamic programming methods. Now dynamic programming methods like value iteration don't work for high dimensional problems--meaning that they are too computationally expensive.

I could not tell if McCandless gets into linear quadratic control methods and Kalman filters to solve the optimization problem above--I did not find references to "Kalman" in the index. However, Kalman filter methods and nonlinear control methods can be used to solve the objective function--especially since that function $u(\cdot)$ is usually nonlinear.

Hence I was hoping that someone could explain how economists choose which method to solve the optimization problem above.

**Tags:** optimization, dsge, dynamic-programming, optimal-control, recursive-macroeconomics
> **Comments:**
> **username_1**: What exactly do you mean by *solving* DSGE models? Are you always assuming parameters are known, or do they have to be estimated?  
> **username_2**: Are you talking about solving these models analytically on paper or actually programming the model? With analytical solutions people just use what they are most comfortable with, if you are talking about programming then its a method that is sufficiently quick enough and easiest to apply  
> **username_0**: @BrsG By solving I mean estimating the consumption and labor vectors $c_t, h_t$. In a real world problem, I would probably look to estimate the control variables. By estimation, mean the potentially recursive optimization of those vectors. Estimating the parameters is important, but there are well known methods for that: I can use gradient descent to minimize the squared loss between the model predictions and the ground truth data, plus any additional constraints I want to impose on the solution. First I just want to efficiently estimate those two vectors.  
> **username_0**: @1muflon1 I am thinking about computational or numerical solutions. The variational method is analytical, but requires some conditions to work--the existence of the first derivatives, etc. Like I was saying, I wanted to precisely understand what I gain by using more complex DP or Kalman or even nonlinear control methods? Do I gain accuracy because I can estimate more complex functions? Do I have less variance in the computed solution versus the analytical solution? Or are there other issues as well?  

---

## 1 answers

### Answer by username_3 on 2022-07-19. Score: 2 | Accepted answer

The main two tools for economists solving infinite-horizon constrained optimization problems in discrete-time, as in your example problem, are:

1. Karush–Kuhn–Tucker (KKT) conditions - which is a variational method, as you noticed, and work for linear and non-linear problems.
2. Dynamic programming - which can also be used to solve linear and non-linear problems, as well.

Bellman's Principle of Optimality gives conditions for the the former approach (sequence problem - SP) to give the same solution as the latter (functional equation problem - FP), that is, SP=FE.

When looking for an analytical solution the former approach is often more convenient. It is often natural to write a problem as a sequence problem and the first-derivative approach does not require knowledge about the functional form of the solution or potentially tedious iteration.

However, the second approach has advantages when one wants to take advantage of a computer to solve a potentially more complicated problem with the use of a computer. While multiple state variables makes solving dynamic programming problems more computationally intensive, this has not deterred many economists and many notable economic models are highly dimensional (see Krusell and Smith (1998) for an early example). Deep-learning methods are also being used to speed up the solving of these large-scale problems.

Another benefit of the dynamic programming approach is that it is easier to adapt to uncertainty. An example given [here](http://faculty.econ.ucdavis.edu/faculty/kdsalyer/LECTURES/Ecn200e/dynprog.pdf).

Lastly, it is possible to extend your constrained optimization problem to a greater degree of complexity, such that you may want to use a more advanced technique, such as the Kalman Filter. An example of this may be a problem in which the [state variable is unobserved to the agent](https://python.quantecon.org/kalman.html).

Note: there exist continuous-time substitutes to these methods (Pontryagin's Maximum Principle and the Hamiltonian-Jacobi-Bellman equations).

> **Comments:**
> **username_0**: thanks for you response, it was very helpful. The thing that I am still confused about is the idea of nonlinear system. So if the model for the dynamics or utility function is nonlinear--which is often the case--then what do you do? If you want to use a Linear Quadratic Control method, the you would need to linearize around the fixed point right? But then the control method would only optimize the consumption and labor vectors to remain at that fixed point. That is the way that optimal control would work in applied math world, but I am not sure how Econ models deal with that.  
> **username_0**: I guess part of the question is whether the objective function $u(c_t, h_t)$ has only one fixed point, or whether it has more than one fixed point? If there is only one fixed point, then linearizing around that fixed point makes sense. But if there are more than one fixed point, then linear control will not help to design a control law to move from one fixed point to another.  
> **username_3**: There is no problem with the utility being nonlinear, as long as we have monotonicity and discounting (Blackwell’s sufficiency conditions) the function we are solving will be a contraction mapping and have a unique fixed point. For a primer on how to solve these problems I recommend: https://researchspace.auckland.ac.nz/bitstream/handle/2292/190/230.pdf?sequence=1&isAllowed=y  
> **username_3**: If you are looking for a more thorough treatment of *why* these methods work you could refer to a more advanced source, such as: Stokey, N., Lucas, R.E., with Prescott, E., Recursive Methods for Economic Dynamics, Harvard University Press (1989)  

---

