<p>The comment by user @MaartenPunt is accurate. I don't think that in general one can identify situations where one should have a clear preference over one formulation over the other. It is more of a case-specific issue (and maybe for some twisted problems where one of the two may fail for usually technical reasons). See this post for some related discussion, <a href="https://economics.stackexchange.com/a/14289/61">https://economics.stackexchange.com/a/14289/61</a>.</p>
<p>...Or sometimes one may get a bit confused, for example, in the specific problem, one could momentarily stop and wonder &quot;what is the derivative of the Hamiltonian with respect to the state variable?&quot;</p>
<p>Well, it is what it appears to be: zero. Because</p>
<p><span class="math-container">$$\frac{\partial \mathcal H}{\partial k}= \frac{\partial \lambda \dot k}{\partial k} = -\frac{\partial \lambda c}{\partial k} = 0,$$</span></p>
<p>because we do not differentiate the decision variable, or the multiplier, with respect to the state variable. Now, optimally, we have</p>
<p><span class="math-container">$$\frac{\partial \mathcal H}{\partial k} = -\dot \lambda,$$</span></p>
<p>and so it follows that the multiplier is constant along the time axis, <span class="math-container">$\dot \lambda = 0$</span>. Then for the other first-order condition, we have</p>
<p><span class="math-container">$$\frac{\partial \mathcal H}{\partial c} = 0 \implies e^{-rt} \frac 1 c = \lambda.$$</span></p>
<p>Differentiating this with respect to time we get</p>
<p><span class="math-container">$$-re^{-rt} \frac 1 c - e^{-rt} \frac{\dot c}{c^2} = 0 \implies \dot c = -rc,$$</span></p>
<p>which is what we get from HJB as &quot;policy&quot; function.</p>
<p>As for whether this is a maximum, it is, because the Hamiltonian is jointly concave in <span class="math-container">$c$</span> and <span class="math-container">$k$</span>, see, <a href="https://economics.stackexchange.com/a/6063/61">https://economics.stackexchange.com/a/6063/61</a>.</p>

CC BY-SA 4.0
2020-11-19T00:39:07.780Z
The word "policy" with the quotation marks is making me wonder, what else would you call this in such a context.
EconJohn
CC BY-SA 4.0
2020-11-19T01:07:10.993Z
@EconJohn We don't usually call it a "policy function" when we do the Hamiltonian, although of course it is. We usually call it the "optimal path", the "optimal rule", things like that. That's why I used quotation marks.
Alecos Papadopoulos
CC BY-SA 4.0
2020-11-18T16:51:14.517Z
2020-11-18T19:32:23.073Z
2020-11-18T19:32:23.073Z
Alecos Papadopoulos
<p>Your standard continuous time cake eating problem is defined as follows:</p>
<p><span class="math-container">$$\max_{c(t)}\int_0^\infty e^{-rt} \ln (c(t)) dt$$</span>
subject to
<span class="math-container">$$f(k(t))=k(t)$$</span>
<span class="math-container">$$\dot{k}(t)=-c(t)$$</span></p>
<p>Approaching this problem by using the Hamilton Jacobi Bellman Equation (the continuous time analogue of the discrete time bellman) with the appropriate drop of time parametrization we have:
<span class="math-container">$$rv(k)=ln(c)+v'(k)\dot{k}$$</span></p>
<p>However in the context of using the (present valued) Hamiltonian proper we have:
<span class="math-container">$$\mathcal{H}=e^{-rt} \ln(c(t))+\lambda(t)\dot{k}(t)$$</span></p>
<p>My question is, under what circumstances would one use the Hamiltonian over the HJB?</p>
<p>Note for more detail on this question see:<br>
<a href="https://math.stackexchange.com/questions/3293825/transversality-condition-with-unbounded-value-function">https://math.stackexchange.com/questions/3293825/transversality-condition-with-unbounded-value-function</a></p>

CC BY-SA 4.0
2020-11-18T10:59:38.220Z
What kind of "different circumstances" do you have in mind? It is too vague for the moment.
Alecos Papadopoulos
CC BY-SA 4.0
2020-11-18T14:35:29.760Z
@AlecosPapadopoulos I'm wondering if there are differences in terms of deriving policy functions VS deriving Euler Equations. As you could see I have little knowledge regarding the solution concept in HJB environments even though I've been working with them for some time now.
EconJohn
CC BY-SA 4.0
2020-11-18T15:12:44.863Z
I seem to recall that the HJB solution is more general and also allows to derive policies off the optimal path. The Hamiltonian does not, but is generally easier to solve and allows for more intuition on its FOC.
Maarten Punt
<macroeconomics><dynamic-programming>
A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?
EconJohn
# A Cake Eating Problem in Continuous Time: Hamiltonian or HJB?

(Asked by: username_0 on 2020-11-18)

Your standard continuous time cake eating problem is defined as follows:

$$\max\_{c(t)}\int\_0^\infty e^{-rt} \ln (c(t)) dt$$
subject to
$$f(k(t))=k(t)$$
$$\dot{k}(t)=-c(t)$$

Approaching this problem by using the Hamilton Jacobi Bellman Equation (the continuous time analogue of the discrete time bellman) with the appropriate drop of time parametrization we have:
$$rv(k)=ln(c)+v'(k)\dot{k}$$

However in the context of using the (present valued) Hamiltonian proper we have:
$$\mathcal{H}=e^{-rt} \ln(c(t))+\lambda(t)\dot{k}(t)$$

My question is, under what circumstances would one use the Hamiltonian over the HJB?

Note for more detail on this question see:  
<https://math.stackexchange.com/questions/3293825/transversality-condition-with-unbounded-value-function>

**Tags:** macroeconomics, dynamic-programming
> **Comments:**
> **username_1**: What kind of "different circumstances" do you have in mind? It is too vague for the moment.  
> **username_0**: @AlecosPapadopoulos I'm wondering if there are differences in terms of deriving policy functions VS deriving Euler Equations. As you could see I have little knowledge regarding the solution concept in HJB environments even though I've been working with them for some time now.  
> **username_2**: I seem to recall that the HJB solution is more general and also allows to derive policies off the optimal path. The Hamiltonian does not, but is generally easier to solve and allows for more intuition on its FOC.  

---

## 1 answers

### Answer by username_1 on 2020-11-18. Score: 7 | Accepted answer

The comment by user @MaartenPunt is accurate. I don't think that in general one can identify situations where one should have a clear preference over one formulation over the other. It is more of a case-specific issue (and maybe for some twisted problems where one of the two may fail for usually technical reasons). See this post for some related discussion, <https://economics.stackexchange.com/a/14289/61>.

...Or sometimes one may get a bit confused, for example, in the specific problem, one could momentarily stop and wonder "what is the derivative of the Hamiltonian with respect to the state variable?"

Well, it is what it appears to be: zero. Because

$$\frac{\partial \mathcal H}{\partial k}= \frac{\partial \lambda \dot k}{\partial k} = -\frac{\partial \lambda c}{\partial k} = 0,$$

because we do not differentiate the decision variable, or the multiplier, with respect to the state variable. Now, optimally, we have

$$\frac{\partial \mathcal H}{\partial k} = -\dot \lambda,$$

and so it follows that the multiplier is constant along the time axis, $\dot \lambda = 0$. Then for the other first-order condition, we have

$$\frac{\partial \mathcal H}{\partial c} = 0 \implies e^{-rt} \frac 1 c = \lambda.$$

Differentiating this with respect to time we get

$$-re^{-rt} \frac 1 c - e^{-rt} \frac{\dot c}{c^2} = 0 \implies \dot c = -rc,$$

which is what we get from HJB as "policy" function.

As for whether this is a maximum, it is, because the Hamiltonian is jointly concave in $c$ and $k$, see, <https://economics.stackexchange.com/a/6063/61>.

> **Comments:**
> **username_0**: The word "policy" with the quotation marks is making me wonder, what else would you call this in such a context.  
> **username_1**: @EconJohn We don't usually call it a "policy function" when we do the Hamiltonian, although of course it is. We usually call it the "optimal path", the "optimal rule", things like that. That's why I used quotation marks.  

---

