<p>5 years later I came across this question myself and I think there is alot to add to the accepted answer: In particular, in general we also want to stick to the case in which <span class="math-container">$r_{t+1}$</span> is unknown to the information at time <span class="math-container">$t$</span>, i.e.
<span class="math-container">\begin{align*}
\lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})] \neq \mathbb{E}_t[\lambda_{t+1} ](1+r_{t+1}).
\end{align*}</span>
But as pointed out in the comments, assuming short term interest rates such that indeed the interest rate <span class="math-container">$r_{t+1}$</span> is measurable w.r.t. the information at time <span class="math-container">$t$</span>, i.e.
<span class="math-container">\begin{align*}
\lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})] = \mathbb{E}_t[\lambda_{t+1} ](1+r_{t+1}),
\end{align*}</span>
isn't the real issue:</p>
<p>Consider an agent facing the following optimization problem
<span class="math-container">\begin{align*}
    \max_{(C_t)_{t \in \mathbb{N}}} \mathbb{E}_0 \left[ \sum_{t=0}^\infty e^{-\delta t} u(C_t, C_{t-1})\right]
\end{align*}</span>
where <span class="math-container">$u(C_t, C_{t-1}) = \frac{1}{1-\gamma} (\alpha C_t + (1- \alpha)C_{t-1})^{1-\gamma}$</span> for some constant <span class="math-container">$\alpha \in (0,1]$</span>. The agent's budget constraint is given by
<span class="math-container">\begin{align*}
    X_{t+1} = (1+r_t)X_t + Y_t - C_t
\end{align*}</span>
where <span class="math-container">$Y_t &gt; 0$</span> is a stochastic income process.</p>
<p>Then, we usually (as in the deterministic case) want to consider the Lagrangian, i.e. the Lagrangian is given by
<span class="math-container">\begin{align*}
    &amp;\mathcal{L}(\{C_t\}_{t=1}^\infty, \{X_{t+1}\}_{t=0}^\infty, \{\lambda_t\}_{t=0}^\infty)\\
    &amp;\hspace{4em}= \mathbb{E}_0 \left[\sum_{t=0}^\infty e^{-\delta t} u(C_t, C_{t-1}) - \lambda_t (X_{t+1} - X_t(1+r_t) - Y_t + C_t)\right].
\end{align*}</span>
Denote by <span class="math-container">$u_1$</span> and <span class="math-container">$u_2$</span> the derivative of <span class="math-container">$u$</span> w.r.t. the first and second variable, respectively. Then, you will read something like &quot;now taking the FOC at period <span class="math-container">$t$</span>&quot; (i.e. w.r.t <span class="math-container">$C_t$</span>, and <span class="math-container">$X_{t+1}$</span>) and &quot;time shift to <span class="math-container">$t+1$</span>&quot; (w.r.t <span class="math-container">$C_{t+1}$</span>), we obtain the following
<span class="math-container">\begin{align}
    &amp;\text{w.r.t. } C_t \hspace{3em} e^{-\delta t} u_1 (C_t, C_{t-1}) + e^{- \delta (t+1)}\mathbb{E}_t [u_2 (C_{t+1}, C_t)] = \lambda_t\tag{1}\label{eqA}\\
    &amp;\text{w.r.t. } C_{t+1} \qquad\mathbb{E}_{t+1}\left[e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})\right] = \lambda_{t+1}\tag{2}\label{eqB}\\
    &amp;\text{w.r.t. } X_{t+1} \qquad\lambda_t = \mathbb{E}_t[\lambda_{t+1} (1+r_{t+1})]\tag{3}\label{eqC}.
\end{align}</span>
Then, as in the corresponding deterministic problem, we insert \eqref{eqA} and \eqref{eqB} into \eqref{eqC}, use the tower property <span class="math-container">$\mathbb{E}_{t}[\mathbb{E}_{t+1}[\cdot]] = \mathbb{E}_t[\cdot]$</span>, and obtain the following:
<span class="math-container">\begin{equation}
\begin{split}
    &amp;u_1 (C_t, C_{t-1}) + e^{- \delta}\mathbb{E}_t [u_2 (C_{t+1}, C_t)] \\
    &amp;\hspace{4em}= \mathbb{E}_t\left[\left(e^{-\delta} u_1 (C_{t+1}, C_t) + e^{-2\delta } u_2 (C_{t+2}, C_{t+1})\right) (1+r_{t+1})\right]. 
\end{split}
\label{eqWant} \tag{4}
\end{equation}</span>
Since the conditional expectation w.r.t the information at time <span class="math-container">$t$</span> is itself a measurable random variable w.r.t. the information at time <span class="math-container">$t$</span>, we can reformulate above equation to the Euler equation:
<span class="math-container">\begin{equation}
    1= \mathbb{E}_t\left[\frac{e^{-\delta} u_1 (C_{t+1}, C_t) + e^{-2\delta } u_2 (C_{t+2}, C_{t+1})}{u_1 (C_t, C_{t-1}) + e^{- \delta}\mathbb{E}_t [u_2 (C_{t+1}, C_t)]} (1+r_{t+1})\right].
\label{eqReform} \tag{5}
\end{equation}</span></p>
<p><strong>Edit</strong>: I think I made a mistake below (see the strikethrough). The approach of &quot;conditioning on the past&quot; seems to be okay (although I lack all the rigorous details). Anyway, I personally would avoid the Langrangian approach and would go with the so called &quot;standard perturbation argument&quot; (see e.g. Financial Asset pricing theory by Claus Munk, Chapter 6.3.2 Habit formation utility, version: September 26, 2007). I leave it for review.</p>
<p>Essentially, Wickens is saying that the simple inserting approach does not work in general if the constraint (compared with the original post, we have <span class="math-container">$X=x$</span> and <span class="math-container">$C=z$</span>)
<span class="math-container">\begin{align*}
     X_{t+1} = f(X_t, C_t) = (1+r_t)X_t + Y_t - C_t
\end{align*}</span>
is more &quot;complicated&quot;, e.g. in the sense that
<span class="math-container">\begin{align*}
     \frac{\partial f(X_{t+s}, C_{t+s})}{\partial C_{t+s}} \neq \text{const}.
\end{align*}</span>
In the example above we have <span class="math-container">$\partial f(X_{t+s}, C_{t+s}) / \partial C_{t+s} = -1$</span> and <span class="math-container">$\partial f(X_{t+s}, C_{t+s}) / \partial X_{t+s} = 1 + r_{t+s}$</span> for <span class="math-container">$s\geq 0$</span>.</p>
<p><s>But the <strong>actual FOC</strong> which we deduce from the Lagrangian are (w.r.t <span class="math-container">$C_t$</span>, <span class="math-container">$C_{t+1}$</span>, and <span class="math-container">$X_{t+1}$</span> at <span class="math-container">$t=0,1,2, \dots$</span>):
<span class="math-container">\begin{align}
    &amp;\text{w.r.t. } C_t \hspace{3em} \mathbb{E}_0 \left[ e^{-\delta t} u_1 (C_t, C_{t-1}) + e^{- \delta (t+1)}u_2 (C_{t+1}, C_t)\right] = \mathbb{E}_0[ \lambda_t]\tag{6}\label{eq1}\\
    &amp;\text{w.r.t. } C_{t+1} \qquad\mathbb{E}_0\left[e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})\right] = \mathbb{E}_0 [\lambda_{t+1}]\tag{7}\label{eq2}\\
    &amp;\text{w.r.t. } X_{t+1} \qquad \mathbb{E}_0[\lambda_t]= \mathbb{E}_0[\lambda_{t+1} (1+r_{t+1})]\tag{8}\label{eq3}.
\end{align}</span></p>
<p>In principle, we <strong>want</strong> (as above) insert \eqref{eq1} and \eqref{eq2} into \eqref{eq3}, to obtain \eqref{eqReform}.</p>
<p><strong>But this cannot be deduced in general with the Lagrangian approach</strong>, <strong>unless</strong> we assume that the multiplier <span class="math-container">$\lambda_{t+1}$</span> and the interest rate <span class="math-container">$r_{t+1}$</span>, as well as interest rate <span class="math-container">$r_{t+1}$</span> and the marginal utility <span class="math-container">$e^{-\delta(t+1)} u_1 (C_{t+1}, C_t) + e^{-\delta (t+2)} u_2 (C_{t+2}, C_{t+1})$</span> are conditionally uncorrelated (w.r.t. to the information at time <span class="math-container">$t=0$</span>), or e.g. the interest rate is constant. <strong>But</strong> still then, we have the problem that our expectation is conditioned on the information set at time <span class="math-container">$t=0$</span>.</p>
<p>Using the Lagrangian approach essentially neglects the fact that we want to have a sequential condition for the optimal consumption flow. As a side remark, the first FOC I gave, would correspond to a different problem:
<span class="math-container">\begin{align*}
\max_{(C_t)_{t \in \mathbb{N}}} \sum_{t=0}^\infty\mathbb{E}_t \left[  e^{-\delta t} u(C_t, C_{t-1})\right]
\end{align*}</span></p>
<p><strong>How to solve the issue of the first problem?</strong> This issue can be solved by the use of the &quot;Stochastic Dynamic Optimization&quot; technique (see Section 15.6 of Wickens), i.e. going to the Bellman equation. In fact, in the case of the first example I gave, you are able to deduce the Euler equation rigorously. For this, consider equation (15.33) in Wickens.</p>
<p>Be aware that the &quot;wrong&quot; Lagrangian approach (i.e. just &quot;inserting&quot; equations \eqref{eqA} and \eqref{eqB} for example) does not lead to the same solution as in the dynamic programming approach in general.</s></p>
<p><strong>Example that the analogous stochastic optimisation problem does not lead to a &quot;naive stochastic version&quot; of the deterministic solution:</strong> Consider Example 15.2 in Wickens' Macroeconomic book:</p>
<p>Maximize
<span class="math-container">\begin{align*}
V = \sum_{t=0}^T \beta^t \ln{c_t}
\end{align*}</span>
subject to
<span class="math-container">\begin{align*}
s_{t+1} - s_t = \alpha (s_t - c_t), \qquad s_{T+1} = 0, \qquad 0&lt;\alpha &lt;1
\end{align*}</span>
The Lagrangian is given by
<span class="math-container">\begin{align*}
\mathcal{L} = \sum_{t=0}^T \{\beta^t \ln{c_t} + \lambda_t[(1+\alpha)s_t - \alpha c_t - s_{t+1}]\}.
\end{align*}</span>
You can easily deduce that (FOC w.r.t. <span class="math-container">$c_t$</span>, <span class="math-container">$c_{t+1}$</span> and <span class="math-container">$s_{t+1}$</span>)
<span class="math-container">\begin{align*}
c_{t+1} = \beta (1+\alpha)c_t.
\end{align*}</span>
If we now consider the stochastic version of this problem (see Example 15.7)
<span class="math-container">\begin{align*}
V = \mathbb{E}_0\left[\sum_{t=0}^T \beta^t \ln{c_t}\right]
\end{align*}</span>
One might be tempted to deduce that
<span class="math-container">\begin{align*}
\mathbb{E}_t[c_{t+1}] = \beta (1+\alpha)c_t.
\end{align*}</span>
But we are only able to deduce that
<span class="math-container">\begin{align*}
\frac{1}{c_t} = \beta (1+\alpha)\mathbb{E}_t\left[\frac{1}{c_{t+1}}\right],
\end{align*}</span>
since in general <span class="math-container">$\mathbb{E}_t[\frac{1}{c_{t+1}}] \neq \frac{1}{\mathbb{E}_t[c_{t+1}]}$</span>.</p>

CC BY-SA 4.0
2021-11-17T21:08:39.860Z
Thanks for the answer. Is there a reason that you chose a durable consumption model of utility? I don't immediately see why it might be important to your broader point.
jmbejara
CC BY-SA 4.0
2021-11-17T21:35:44.923Z
I'm trying to see where your conclusions differ from that in the notes linked by @Alecos's answer (https://www3.nd.edu/~esims1/rbc_notes_2016.pdf). Following the procedure from those notes applied to your problem, you would derive equation (4) by shifting the time subscripts on equation (1) before substituting. I'm internally debating what I think about that step.
jmbejara
CC BY-SA 4.0
2021-11-17T21:36:24.507Z
It seems innocuous, but I think it would amount to assuming that we derive the FOCs for the agent one step in the future and using this assumed optimal future behavior to determine optimal behavior today. I assume that's what dynamic programming is doing more explicitly. Am I missing something?
jmbejara
CC BY-SA 4.0
2021-11-17T21:41:22.727Z
@jmbejara There is no reason behind the choice of the model. I already had written down this model and wanted to save some time.
Philip
CC BY-SA 4.0
2021-11-17T21:44:18.530Z
@jmbejara Your second question: In Alecos's notes you want to replace $r_{t-1}$ by $r_t$ such that you cannot "put out" the interest rate out of the conditional expectations. In Aleco's notes it holds that $\mathbb{E}_t[1+r_t]=1+r_t$ but general you want to consider $\mathbb{E}_t[1+r_{t+1}]\neq1+r_{t+1}$. This makes the problem harder to solve and you often run into the problem of equation (1)-(3)
Philip
CC BY-SA 4.0
2021-11-17T21:54:35.057Z
@jmbejara I added a comment concerning your second question. I hope this is what you meant. Concerning your third comment: I couldn't directly follow your argumentation but dynamic programming usually works (in a finite time horizon setting) by so called backwards induction, and then you "start from the future" in some sense.
Philip
CC BY-SA 4.0
2021-11-18T17:55:47.057Z
Also, the FOCs in your first example don't follow from the Lagrangian as it's written. To fix it, the expectation in the Lagrangian should be conditioned on $t$ and the summation fixed accordingly. (Or, write the FOCs with expectations conditioned on $t=0$.) Kinda nitpicking, but it's important to this discussion.
jmbejara
CC BY-SA 4.0
2021-11-18T17:59:36.443Z
I understand, but you've missed my point. Where in https://www3.nd.edu/~esims1/rbc_notes_2016.pdf does it assume that the interest rate is constant? Now, consider this. Take (3). Substitute in (1). Now, shift the time subscripts of (1) to get $e^{-\delta (t+1)} u_1 (C_{t+1}, C_{t}) + e^{- \delta (t+2)}\mathbb{E}_t [u_2 (C_{t+2}, C_{t+1})] = \lambda_{t+1}$. Now, substitute this expression into (3). The result is (4).
jmbejara
CC BY-SA 4.0
2021-11-18T21:20:36.693Z
@jmbejara Thanks for your suggestions. You are totally right, there were some mistakes about the time conditioning. Do you now agree with the FOC? Or did I something wrong? I edited my post. To your second point: No, he didn't assume that the interest rate is constant, I only wanted to stress that it "behaves" like a constant since you can pull it out of the conditional expectation.
Philip
CC BY-SA 4.0
2021-11-19T03:58:14.760Z
Regarding the timing convention in Sims' notes---fair enough. In that case, the interest rate is in the information set that's conditioned on. However,  You could just say that "we really want to stick to the case in which $r_{t+1}$ is unknown to the information at time $t$." IMO, this is not as important in many macro models as you would think. Most macro models proxy a risk-free short-term interest rate for $r_{t+1}$. Thus, in many cases $r_{t+1}$ is in the information set $\mathcal F_{t}$
jmbejara
CC BY-SA 4.0
2021-11-19T04:00:27.447Z
Your derivation of the FOCs isn't quite what I had in mind nor is it standard. IMO, you should follow the convention in the image in the original post. If you want, I can go ahead and make the edit for you in your post. I obviously can't fit it in here in the comments.
jmbejara
CC BY-SA 4.0
2021-11-19T04:01:39.120Z
Could you comment on my proposal for substitution above? I've mentioned it twice. It's the proposal starting with "Take (3). Substitute in (1)..."
jmbejara
CC BY-SA 4.0
2021-11-19T16:37:58.307Z
@jmbejara To your second comment: Please go ahead, I really want to see what you had in mind. I guess, just replace $0$ by $t$ and $t$ by $t+s$ for the variables and summation over $s$? To your third comment: I implicitly answered this by editing the post. Basically, your substitution approach is equation (1) to (4). I think you made a small mistake by "shifting the time subscripts" since you didn't change the time on the cond. expectation. O/w you would come up with eq. (2). With this approach you obtain the Euler equation but the point is that you do not get eq. (1)-(3) from the Lagrangian.
Philip
CC BY-SA 4.0
2021-11-19T17:51:24.580Z
Ok, I didn't see the edit. I see now why the "time shift" doesn't work. Thanks! After working out my alternative setup, I realize that it's not going to make a difference. My idea was to set up the Lagrangian for the program from the perspective of the agent at time $t$ and a separate Lagrangian from the perspective of the agent at time $t+1$.  It'll take to long to explain, but I realized that it won't help to do this anyway.
jmbejara
CC BY-SA 4.0
2021-11-19T17:53:06.257Z
Also, BTW, when you discuss how the interest rate and lambda or lambda and the marginal utility should be uncorrelated, you should be sure to mention that they need to be **conditionally** uncorrelated. "Unconditionally uncorrelated" is too strong a condition to require for the Lagrangian approach to work.
jmbejara
CC BY-SA 4.0
2021-11-19T20:05:10.527Z
I edited the post about the conditional uncorrelatedness. So, now I think that the post is more or less complete. Thanks again for the useful suggestions!
Philip
CC BY-SA 4.0
2024-12-20T12:35:39.800Z
Thanks for the detailed answer. Would you mind elaborating on your latest edit? What's the problem with the strikethrough text?
Junnan
CC BY-SA 4.0
2021-11-17T17:25:08.393Z
2022-01-09T19:12:06.770Z
2022-01-09T19:12:06.770Z
Philip
<p>Following a comment exchange below another answer, the critical detail <a href="https://www3.nd.edu/~esims1/rbc_notes_2016.pdf" rel="nofollow">of a link offered by the OP</a> as an example of by-passing/ignoring Wicken's comment/argument, and one different from Wickens' formulation, is that in the link's eq. $(4)$, </p>

<p>$$\lambda_t = \beta E_t[\lambda_{t+1}(1+r_t)]$$</p>

<p>the multiplier $\lambda_{t+1}$ appears together with the interest rate <em>of period $t$</em>, which is assumed part of the information set at $t$ and so not a random variable in the specific equation. So the authors can proceed in the 2nd line after eq. $(6)$ to write the marginal rate of substitution as they do, without the need to assume uncorrelatedness between the multiplier/marginal utility of consumption, and the interest rate.  </p>

<p>This goes back to how they formulate the income resource constraint (page 2 middle), where it is essentially assumed that the household has, <em>at the beginning</em> of period $t$, available assets or debt $(1+r_{t-1})B_t$. The authors explicitly discuss this "timing convention" immediately after eq. $(1)$ </p>

<blockquote>
  <p><em>"Note a timing convention -$r_{t-1}$ is the interest you have to pay today on existing debt. $r_t$ is what you will have to pay tomorrow,
  but you choose how much debt to take into tomorrow today. Hence, we
  assume that households observe $r_t$ in time $t$. Hence we can treat
  $r_t$ as known from the perspective of time $t$."</em></p>
</blockquote>

CC BY-SA 3.0
2016-10-02T20:45:53.910Z
Alecos, do you have an opinion on this book by Michael Wickens?
An old man in the sea.
CC BY-SA 3.0
2016-10-02T20:56:42.463Z
I haven't checked it thoroughly. But my first impression when I browsed its pages (I have the first edition), is that it is good for the basic tools (so graduate beginners level). Sometimes such books are very useful because the keep the core very visible.
Alecos Papadopoulos
CC BY-SA 3.0
2016-10-02T20:36:10.780Z
2016-10-02T20:36:10.780Z
Alecos Papadopoulos
<p>In Wickens' Macroeconomics book, in page 552, the author states the following:</p>

<p>«The stochastic problem can be solved using the method of Lagrange multipliers, but there is a problem with this solution.»</p>

<p>However, if I'm not mistaken, it's very usual to see other books using lagrange multipliers to solve this type of problems. Are they wrong? Am I wrong? Is there a way to go around this apparent limitation?</p>

<p>Any help would be appreciated.</p>

<p>Here's a the image:</p>

<p><a href="https://i.sstatic.net/YiXQ5.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/YiXQ5.png" alt="enter image description here"></a></p>

<macroeconomics><general-equilibrium>
Solving Stochastic Dynamic Optimization Problems: A difficulty with Lagrange Multipliers
An old man in the sea.
# Solving Stochastic Dynamic Optimization Problems: A difficulty with Lagrange Multipliers

(Asked by: username_0 on 2016-10-01)

In Wickens' Macroeconomics book, in page 552, the author states the following:

«The stochastic problem can be solved using the method of Lagrange multipliers, but there is a problem with this solution.»

However, if I'm not mistaken, it's very usual to see other books using lagrange multipliers to solve this type of problems. Are they wrong? Am I wrong? Is there a way to go around this apparent limitation?

Any help would be appreciated.

Here's a the image:

[![enter image description here](https://i.sstatic.net/YiXQ5.png)](https://i.sstatic.net/YiXQ5.png)

**Tags:** macroeconomics, general-equilibrium

---

## 2 answers

### Answer by username_4 on 2016-10-02. Score: 3 | Accepted answer

Following a comment exchange below another answer, the critical detail [of a link offered by the OP](https://www3.nd.edu/~esims1/rbc_notes_2016.pdf) as an example of by-passing/ignoring Wicken's comment/argument, and one different from Wickens' formulation, is that in the link's eq. $(4)$,

$$\lambda\_t = \beta E\_t[\lambda\_{t+1}(1+r\_t)]$$

the multiplier $\lambda\_{t+1}$ appears together with the interest rate *of period $t$*, which is assumed part of the information set at $t$ and so not a random variable in the specific equation. So the authors can proceed in the 2nd line after eq. $(6)$ to write the marginal rate of substitution as they do, without the need to assume uncorrelatedness between the multiplier/marginal utility of consumption, and the interest rate.

This goes back to how they formulate the income resource constraint (page 2 middle), where it is essentially assumed that the household has, *at the beginning* of period $t$, available assets or debt $(1+r\_{t-1})B\_t$. The authors explicitly discuss this "timing convention" immediately after eq. $(1)$

> *"Note a timing convention -$r\_{t-1}$ is the interest you have to pay today on existing debt. $r\_t$ is what you will have to pay tomorrow,
> but you choose how much debt to take into tomorrow today. Hence, we
> assume that households observe $r\_t$ in time $t$. Hence we can treat
> $r\_t$ as known from the perspective of time $t$."*

> **Comments:**
> **username_0**: Alecos, do you have an opinion on this book by Michael Wickens?  
> **username_4**: I haven't checked it thoroughly. But my first impression when I browsed its pages (I have the first edition), is that it is good for the basic tools (so graduate beginners level). Sometimes such books are very useful because the keep the core very visible.  

---

### Answer by username_1 on 2021-11-17. Score: 3

5 years later I came across this question myself and I think there is alot to add to the accepted answer: In particular, in general we also want to stick to the case in which $r\_{t+1}$ is unknown to the information at time $t$, i.e.
\begin{align\*}
\lambda\_t = \mathbb{E}\_t[\lambda\_{t+1} (1+r\_{t+1})] \neq \mathbb{E}\_t[\lambda\_{t+1} ](1+r\_{t+1}).
\end{align\*}
But as pointed out in the comments, assuming short term interest rates such that indeed the interest rate $r\_{t+1}$ is measurable w.r.t. the information at time $t$, i.e.
\begin{align\*}
\lambda\_t = \mathbb{E}\_t[\lambda\_{t+1} (1+r\_{t+1})] = \mathbb{E}\_t[\lambda\_{t+1} ](1+r\_{t+1}),
\end{align\*}
isn't the real issue:

Consider an agent facing the following optimization problem
\begin{align\*}
\max\_{(C\_t)\_{t \in \mathbb{N}}} \mathbb{E}\_0 \left[ \sum\_{t=0}^\infty e^{-\delta t} u(C\_t, C\_{t-1})\right]
\end{align\*}
where $u(C\_t, C\_{t-1}) = \frac{1}{1-\gamma} (\alpha C\_t + (1- \alpha)C\_{t-1})^{1-\gamma}$ for some constant $\alpha \in (0,1]$. The agent's budget constraint is given by
\begin{align\*}
X\_{t+1} = (1+r\_t)X\_t + Y\_t - C\_t
\end{align\*}
where $Y\_t > 0$ is a stochastic income process.

Then, we usually (as in the deterministic case) want to consider the Lagrangian, i.e. the Lagrangian is given by
\begin{align\*}
&\mathcal{L}(\{C\_t\}\_{t=1}^\infty, \{X\_{t+1}\}\_{t=0}^\infty, \{\lambda\_t\}\_{t=0}^\infty)\\
&\hspace{4em}= \mathbb{E}\_0 \left[\sum\_{t=0}^\infty e^{-\delta t} u(C\_t, C\_{t-1}) - \lambda\_t (X\_{t+1} - X\_t(1+r\_t) - Y\_t + C\_t)\right].
\end{align\*}
Denote by $u\_1$ and $u\_2$ the derivative of $u$ w.r.t. the first and second variable, respectively. Then, you will read something like "now taking the FOC at period $t$" (i.e. w.r.t $C\_t$, and $X\_{t+1}$) and "time shift to $t+1$" (w.r.t $C\_{t+1}$), we obtain the following
\begin{align}
&\text{w.r.t. } C\_t \hspace{3em} e^{-\delta t} u\_1 (C\_t, C\_{t-1}) + e^{- \delta (t+1)}\mathbb{E}\_t [u\_2 (C\_{t+1}, C\_t)] = \lambda\_t\tag{1}\label{eqA}\\
&\text{w.r.t. } C\_{t+1} \qquad\mathbb{E}\_{t+1}\left[e^{-\delta(t+1)} u\_1 (C\_{t+1}, C\_t) + e^{-\delta (t+2)} u\_2 (C\_{t+2}, C\_{t+1})\right] = \lambda\_{t+1}\tag{2}\label{eqB}\\
&\text{w.r.t. } X\_{t+1} \qquad\lambda\_t = \mathbb{E}\_t[\lambda\_{t+1} (1+r\_{t+1})]\tag{3}\label{eqC}.
\end{align}
Then, as in the corresponding deterministic problem, we insert \eqref{eqA} and \eqref{eqB} into \eqref{eqC}, use the tower property $\mathbb{E}\_{t}[\mathbb{E}\_{t+1}[\cdot]] = \mathbb{E}\_t[\cdot]$, and obtain the following:
\begin{equation}
\begin{split}
&u\_1 (C\_t, C\_{t-1}) + e^{- \delta}\mathbb{E}\_t [u\_2 (C\_{t+1}, C\_t)] \\
&\hspace{4em}= \mathbb{E}\_t\left[\left(e^{-\delta} u\_1 (C\_{t+1}, C\_t) + e^{-2\delta } u\_2 (C\_{t+2}, C\_{t+1})\right) (1+r\_{t+1})\right].
\end{split}
\label{eqWant} \tag{4}
\end{equation}
Since the conditional expectation w.r.t the information at time $t$ is itself a measurable random variable w.r.t. the information at time $t$, we can reformulate above equation to the Euler equation:
\begin{equation}
1= \mathbb{E}\_t\left[\frac{e^{-\delta} u\_1 (C\_{t+1}, C\_t) + e^{-2\delta } u\_2 (C\_{t+2}, C\_{t+1})}{u\_1 (C\_t, C\_{t-1}) + e^{- \delta}\mathbb{E}\_t [u\_2 (C\_{t+1}, C\_t)]} (1+r\_{t+1})\right].
\label{eqReform} \tag{5}
\end{equation}

**Edit**: I think I made a mistake below (see the strikethrough). The approach of "conditioning on the past" seems to be okay (although I lack all the rigorous details). Anyway, I personally would avoid the Langrangian approach and would go with the so called "standard perturbation argument" (see e.g. Financial Asset pricing theory by Claus Munk, Chapter 6.3.2 Habit formation utility, version: September 26, 2007). I leave it for review.

Essentially, Wickens is saying that the simple inserting approach does not work in general if the constraint (compared with the original post, we have $X=x$ and $C=z$)
\begin{align\*}
X\_{t+1} = f(X\_t, C\_t) = (1+r\_t)X\_t + Y\_t - C\_t
\end{align\*}
is more "complicated", e.g. in the sense that
\begin{align\*}
\frac{\partial f(X\_{t+s}, C\_{t+s})}{\partial C\_{t+s}} \neq \text{const}.
\end{align\*}
In the example above we have $\partial f(X\_{t+s}, C\_{t+s}) / \partial C\_{t+s} = -1$ and $\partial f(X\_{t+s}, C\_{t+s}) / \partial X\_{t+s} = 1 + r\_{t+s}$ for $s\geq 0$.

~~But the **actual FOC** which we deduce from the Lagrangian are (w.r.t $C\_t$, $C\_{t+1}$, and $X\_{t+1}$ at $t=0,1,2, \dots$):
\begin{align}
&\text{w.r.t. } C\_t \hspace{3em} \mathbb{E}\_0 \left[ e^{-\delta t} u\_1 (C\_t, C\_{t-1}) + e^{- \delta (t+1)}u\_2 (C\_{t+1}, C\_t)\right] = \mathbb{E}\_0[ \lambda\_t]\tag{6}\label{eq1}\\
&\text{w.r.t. } C\_{t+1} \qquad\mathbb{E}\_0\left[e^{-\delta(t+1)} u\_1 (C\_{t+1}, C\_t) + e^{-\delta (t+2)} u\_2 (C\_{t+2}, C\_{t+1})\right] = \mathbb{E}\_0 [\lambda\_{t+1}]\tag{7}\label{eq2}\\
&\text{w.r.t. } X\_{t+1} \qquad \mathbb{E}\_0[\lambda\_t]= \mathbb{E}\_0[\lambda\_{t+1} (1+r\_{t+1})]\tag{8}\label{eq3}.
\end{align}~~

In principle, we **want** (as above) insert \eqref{eq1} and \eqref{eq2} into \eqref{eq3}, to obtain \eqref{eqReform}.

**But this cannot be deduced in general with the Lagrangian approach**, **unless** we assume that the multiplier $\lambda\_{t+1}$ and the interest rate $r\_{t+1}$, as well as interest rate $r\_{t+1}$ and the marginal utility $e^{-\delta(t+1)} u\_1 (C\_{t+1}, C\_t) + e^{-\delta (t+2)} u\_2 (C\_{t+2}, C\_{t+1})$ are conditionally uncorrelated (w.r.t. to the information at time $t=0$), or e.g. the interest rate is constant. **But** still then, we have the problem that our expectation is conditioned on the information set at time $t=0$.

Using the Lagrangian approach essentially neglects the fact that we want to have a sequential condition for the optimal consumption flow. As a side remark, the first FOC I gave, would correspond to a different problem:
\begin{align\*}
\max\_{(C\_t)\_{t \in \mathbb{N}}} \sum\_{t=0}^\infty\mathbb{E}\_t \left[ e^{-\delta t} u(C\_t, C\_{t-1})\right]
\end{align\*}

**How to solve the issue of the first problem?** This issue can be solved by the use of the "Stochastic Dynamic Optimization" technique (see Section 15.6 of Wickens), i.e. going to the Bellman equation. In fact, in the case of the first example I gave, you are able to deduce the Euler equation rigorously. For this, consider equation (15.33) in Wickens.

Be aware that the "wrong" Lagrangian approach (i.e. just "inserting" equations \eqref{eqA} and \eqref{eqB} for example) does not lead to the same solution as in the dynamic programming approach in general.

**Example that the analogous stochastic optimisation problem does not lead to a "naive stochastic version" of the deterministic solution:** Consider Example 15.2 in Wickens' Macroeconomic book:

Maximize
\begin{align\*}
V = \sum\_{t=0}^T \beta^t \ln{c\_t}
\end{align\*}
subject to
\begin{align\*}
s\_{t+1} - s\_t = \alpha (s\_t - c\_t), \qquad s\_{T+1} = 0, \qquad 0<\alpha <1
\end{align\*}
The Lagrangian is given by
\begin{align\*}
\mathcal{L} = \sum\_{t=0}^T \{\beta^t \ln{c\_t} + \lambda\_t[(1+\alpha)s\_t - \alpha c\_t - s\_{t+1}]\}.
\end{align\*}
You can easily deduce that (FOC w.r.t. $c\_t$, $c\_{t+1}$ and $s\_{t+1}$)
\begin{align\*}
c\_{t+1} = \beta (1+\alpha)c\_t.
\end{align\*}
If we now consider the stochastic version of this problem (see Example 15.7)
\begin{align\*}
V = \mathbb{E}\_0\left[\sum\_{t=0}^T \beta^t \ln{c\_t}\right]
\end{align\*}
One might be tempted to deduce that
\begin{align\*}
\mathbb{E}\_t[c\_{t+1}] = \beta (1+\alpha)c\_t.
\end{align\*}
But we are only able to deduce that
\begin{align\*}
\frac{1}{c\_t} = \beta (1+\alpha)\mathbb{E}\_t\left[\frac{1}{c\_{t+1}}\right],
\end{align\*}
since in general $\mathbb{E}\_t[\frac{1}{c\_{t+1}}] \neq \frac{1}{\mathbb{E}\_t[c\_{t+1}]}$.

> **Comments:**
> **username_2**: Thanks for the answer. Is there a reason that you chose a durable consumption model of utility? I don't immediately see why it might be important to your broader point.  
> **username_2**: I'm trying to see where your conclusions differ from that in the notes linked by @Alecos's answer (https://www3.nd.edu/~esims1/rbc_notes_2016.pdf). Following the procedure from those notes applied to your problem, you would derive equation (4) by shifting the time subscripts on equation (1) before substituting. I'm internally debating what I think about that step.  
> **username_2**: It seems innocuous, but I think it would amount to assuming that we derive the FOCs for the agent one step in the future and using this assumed optimal future behavior to determine optimal behavior today. I assume that's what dynamic programming is doing more explicitly. Am I missing something?  
> **username_1**: @jmbejara There is no reason behind the choice of the model. I already had written down this model and wanted to save some time.  
> **username_1**: @jmbejara Your second question: In Alecos's notes you want to replace $r_{t-1}$ by $r_t$ such that you cannot "put out" the interest rate out of the conditional expectations. In Aleco's notes it holds that $\mathbb{E}_t[1+r_t]=1+r_t$ but general you want to consider $\mathbb{E}_t[1+r_{t+1}]\neq1+r_{t+1}$. This makes the problem harder to solve and you often run into the problem of equation (1)-(3)  
> **username_1**: @jmbejara I added a comment concerning your second question. I hope this is what you meant. Concerning your third comment: I couldn't directly follow your argumentation but dynamic programming usually works (in a finite time horizon setting) by so called backwards induction, and then you "start from the future" in some sense.  
> **username_2**: Also, the FOCs in your first example don't follow from the Lagrangian as it's written. To fix it, the expectation in the Lagrangian should be conditioned on $t$ and the summation fixed accordingly. (Or, write the FOCs with expectations conditioned on $t=0$.) Kinda nitpicking, but it's important to this discussion.  
> **username_2**: I understand, but you've missed my point. Where in https://www3.nd.edu/~esims1/rbc_notes_2016.pdf does it assume that the interest rate is constant? Now, consider this. Take (3). Substitute in (1). Now, shift the time subscripts of (1) to get $e^{-\delta (t+1)} u_1 (C_{t+1}, C_{t}) + e^{- \delta (t+2)}\mathbb{E}_t [u_2 (C_{t+2}, C_{t+1})] = \lambda_{t+1}$. Now, substitute this expression into (3). The result is (4).  
> **username_1**: @jmbejara Thanks for your suggestions. You are totally right, there were some mistakes about the time conditioning. Do you now agree with the FOC? Or did I something wrong? I edited my post. To your second point: No, he didn't assume that the interest rate is constant, I only wanted to stress that it "behaves" like a constant since you can pull it out of the conditional expectation.  
> **username_2**: Regarding the timing convention in Sims' notes---fair enough. In that case, the interest rate is in the information set that's conditioned on. However,  You could just say that "we really want to stick to the case in which $r_{t+1}$ is unknown to the information at time $t$." IMO, this is not as important in many macro models as you would think. Most macro models proxy a risk-free short-term interest rate for $r_{t+1}$. Thus, in many cases $r_{t+1}$ is in the information set $\mathcal F_{t}$  
> **username_2**: Your derivation of the FOCs isn't quite what I had in mind nor is it standard. IMO, you should follow the convention in the image in the original post. If you want, I can go ahead and make the edit for you in your post. I obviously can't fit it in here in the comments.  
> **username_2**: Could you comment on my proposal for substitution above? I've mentioned it twice. It's the proposal starting with "Take (3). Substitute in (1)..."  
> **username_1**: @jmbejara To your second comment: Please go ahead, I really want to see what you had in mind. I guess, just replace $0$ by $t$ and $t$ by $t+s$ for the variables and summation over $s$? To your third comment: I implicitly answered this by editing the post. Basically, your substitution approach is equation (1) to (4). I think you made a small mistake by "shifting the time subscripts" since you didn't change the time on the cond. expectation. O/w you would come up with eq. (2). With this approach you obtain the Euler equation but the point is that you do not get eq. (1)-(3) from the Lagrangian.  
> **username_2**: Ok, I didn't see the edit. I see now why the "time shift" doesn't work. Thanks! After working out my alternative setup, I realize that it's not going to make a difference. My idea was to set up the Lagrangian for the program from the perspective of the agent at time $t$ and a separate Lagrangian from the perspective of the agent at time $t+1$.  It'll take to long to explain, but I realized that it won't help to do this anyway.  
> **username_2**: Also, BTW, when you discuss how the interest rate and lambda or lambda and the marginal utility should be uncorrelated, you should be sure to mention that they need to be **conditionally** uncorrelated. "Unconditionally uncorrelated" is too strong a condition to require for the Lagrangian approach to work.  
> **username_1**: I edited the post about the conditional uncorrelatedness. So, now I think that the post is more or less complete. Thanks again for the useful suggestions!  
> **username_3**: Thanks for the detailed answer. Would you mind elaborating on your latest edit? What's the problem with the strikethrough text?  

---

