<p>Since you mention Walton, here is something from his notes page 111 Definition 4:</p>
<blockquote>
<p>The Hamiltonian and is defined
<span class="math-container">$H(t,x,\rho) := min _v \{c(t,x,v) - \rho v\}$</span></p>
</blockquote>
<p>Notice the analogue to the Hamiltonian you have written.</p>
<p><span class="math-container">$v$</span> in here is the control (your <span class="math-container">$c$</span>), <span class="math-container">$c(t,x,v)$</span> in here is the <span class="math-container">$u(c)$</span> function you have and <span class="math-container">$-\rho = \partial_xL(t,x)$</span> in here where <span class="math-container">$L$</span> is the minimized loss, analogous to your maximized value.</p>
<p>Answer to your second question: That is simply how a Hamiltonian is defined.</p>
<p>Answer to your first question:
When the HJB is derived it has that specific form. For example see Benjamin Moll's notes, slides 8-10.</p>
<p>I hope this helps.</p>
<p>References:</p>
<p>Walton:  <a href="https://appliedprobability.files.wordpress.com/2021/01/stochastic_control_jan29.pdf" rel="nofollow noreferrer">https://appliedprobability.files.wordpress.com/2021/01/stochastic_control_jan29.pdf</a></p>
<p>Moll: <a href="https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf" rel="nofollow noreferrer">https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf</a></p>

CC BY-SA 4.0
2021-07-23T09:59:38.033Z
2021-07-23T09:59:38.033Z
erik
<p>Deterministic Optimal Control Problem<br />
<span class="math-container">$
V(a_t) 
= 
\underset{c}{\max}
\int_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c_{\tau}) d\tau
$</span><br />
s.t.
<span class="math-container">$
\frac{da}{d\tau} = \left( r a_{\tau}  - c_{\tau} \right)  $</span></p>
<p><strong>CV Hamiltonian</strong>:
<span class="math-container">$H^{CV} \equiv  u(c_t) + \lambda_{t} \times (r a_{t}  - c_{t})$</span><br />
<span class="math-container">$H^{CV}_c=0$</span>, <span class="math-container">$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$</span>,
<span class="math-container">$H^{CV}_\lambda = \frac{da}{dt}$</span><br />
Boils down to system of boundary-value DAEs.<br />
If you can eliminate <span class="math-container">$\lambda$</span>, system of BV-ODEs.</p>
<p><strong>HJB</strong>:
<span class="math-container">$\rho V(a_t) = \underset{c}{\max}
\{ u(c_{t}) + V_a \times (r a_{t}  - c_{t}) \}$</span></p>
<p>Authors often define the term in the HJB which depends on the control as &quot;Hamiltonian&quot;:
<span class="math-container">$H\left( V_a\right) \equiv 
\underset{c}{\max} \{ u(c) - c_{t} \times  V_a  \} \Rightarrow c(a_t) =u'^{-1} \left( V_a(a_t) \right)$</span><br />
Hence <span class="math-container">$\Leftrightarrow \rho V(a_t) = H\left( V_a\right) + V_a \times r a_{t}$</span></p>
<p>Adrien Bilal (<a href="https://drive.google.com/file/d/1vXgezoAHRCRnybN9Yis68RcLCghYW3gu/view" rel="nofollow noreferrer">p9</a>) and <a href="https://appliedprobability.blog/" rel="nofollow noreferrer">Neil Walton</a> &amp; others, call the term inside the HJB
<span class="math-container">$H\left( V_a\right)$</span> a Hamiltonian.<br />
If we set <span class="math-container">$\lambda_{t} \equiv V_a$</span> then <span class="math-container">$H^{CV} = H\left( V_a\right) + \lambda  \times r a_{t}$</span>.</p>
<p><strong>Q1</strong>: why do they add the term <span class="math-container">$H\left( V_a\right)$</span> inside the HJB?<br />
-guess: it separates the non-linear part of HJB<br />
-guess: it separates the part of the HJB that depends on the control<br />
<strong>Q2</strong>: why do they call <span class="math-container">$H\left( V_a\right)$</span> a &quot;hamiltonian&quot;?</p>
<p><strong>Note</strong>: I am NOT asking if the solution to the HJB is the same as to the Hamiltonian.<br />
Sol to HJB: <span class="math-container">$V(a_t)$</span> gives policy <span class="math-container">$c(a_t) = u'^{-1}(V_a)$</span><br />
Combine policy w/ transition eqn: you get <span class="math-container">$c^{HJB}(t)$</span><br />
Sol to CV Hamiltonian: <span class="math-container">$c^{H}(t)$</span><br />
I can show (under appropriate assumptions) that <span class="math-container">$c^{HJB}(t)=c^{H}(t)$</span>.</p>
<p>(side-node) Recall: <span class="math-container">$H^{CV}_c=0$</span> &amp; <span class="math-container">$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$</span><br />
<span class="math-container">$H^{CV}_a=\rho \lambda - \frac{d\lambda}{dt}$</span>:
implies <span class="math-container">$\lambda r = \rho \lambda - \frac{d\lambda}{dt}$</span>
implies <span class="math-container">$V_a (\rho-r) = V_{aa} (r a_{t}  - c_{t})$</span></p>

<optimal-control><continuous-time>
What is the relationship between the HJB and "Hamiltonian"? Why is the Hamiltonian H(p) inside the HJB?
Albert Zevelev
# What is the relationship between the HJB and "Hamiltonian"? Why is the Hamiltonian H(p) inside the HJB?

(Asked by: username_0 on 2021-07-21)

Deterministic Optimal Control Problem  
$
V(a\_t)
=
\underset{c}{\max}
\int\_{\tau =t}^{\tau = \infty} e^{-\rho (\tau - t) } u(c\_{\tau}) d\tau
$  
s.t.
$
\frac{da}{d\tau} = \left( r a\_{\tau} - c\_{\tau} \right) $

**CV Hamiltonian**:
$H^{CV} \equiv u(c\_t) + \lambda\_{t} \times (r a\_{t} - c\_{t})$  
$H^{CV}\_c=0$, $H^{CV}\_a=\rho \lambda - \frac{d\lambda}{dt}$,
$H^{CV}\_\lambda = \frac{da}{dt}$  
Boils down to system of boundary-value DAEs.  
If you can eliminate $\lambda$, system of BV-ODEs.

**HJB**:
$\rho V(a\_t) = \underset{c}{\max}
\{ u(c\_{t}) + V\_a \times (r a\_{t} - c\_{t}) \}$

Authors often define the term in the HJB which depends on the control as "Hamiltonian":
$H\left( V\_a\right) \equiv
\underset{c}{\max} \{ u(c) - c\_{t} \times V\_a \} \Rightarrow c(a\_t) =u'^{-1} \left( V\_a(a\_t) \right)$  
Hence $\Leftrightarrow \rho V(a\_t) = H\left( V\_a\right) + V\_a \times r a\_{t}$

Adrien Bilal ([p9](https://drive.google.com/file/d/1vXgezoAHRCRnybN9Yis68RcLCghYW3gu/view)) and [Neil Walton](https://appliedprobability.blog/) & others, call the term inside the HJB
$H\left( V\_a\right)$ a Hamiltonian.  
If we set $\lambda\_{t} \equiv V\_a$ then $H^{CV} = H\left( V\_a\right) + \lambda \times r a\_{t}$.

**Q1**: why do they add the term $H\left( V\_a\right)$ inside the HJB?  
-guess: it separates the non-linear part of HJB  
-guess: it separates the part of the HJB that depends on the control  
**Q2**: why do they call $H\left( V\_a\right)$ a "hamiltonian"?

**Note**: I am NOT asking if the solution to the HJB is the same as to the Hamiltonian.  
Sol to HJB: $V(a\_t)$ gives policy $c(a\_t) = u'^{-1}(V\_a)$  
Combine policy w/ transition eqn: you get $c^{HJB}(t)$  
Sol to CV Hamiltonian: $c^{H}(t)$  
I can show (under appropriate assumptions) that $c^{HJB}(t)=c^{H}(t)$.

(side-node) Recall: $H^{CV}\_c=0$ & $H^{CV}\_a=\rho \lambda - \frac{d\lambda}{dt}$  
$H^{CV}\_a=\rho \lambda - \frac{d\lambda}{dt}$:
implies $\lambda r = \rho \lambda - \frac{d\lambda}{dt}$
implies $V\_a (\rho-r) = V\_{aa} (r a\_{t} - c\_{t})$

**Tags:** optimal-control, continuous-time

---

## 1 answers

### Answer by username_1 on 2021-07-23. Score: 1

Since you mention Walton, here is something from his notes page 111 Definition 4:

> The Hamiltonian and is defined
> $H(t,x,\rho) := min \_v \{c(t,x,v) - \rho v\}$

Notice the analogue to the Hamiltonian you have written.

$v$ in here is the control (your $c$), $c(t,x,v)$ in here is the $u(c)$ function you have and $-\rho = \partial\_xL(t,x)$ in here where $L$ is the minimized loss, analogous to your maximized value.

Answer to your second question: That is simply how a Hamiltonian is defined.

Answer to your first question:
When the HJB is derived it has that specific form. For example see Benjamin Moll's notes, slides 8-10.

I hope this helps.

References:

Walton: <https://appliedprobability.files.wordpress.com/2021/01/stochastic_control_jan29.pdf>

Moll: <https://benjaminmoll.com/wp-content/uploads/2019/07/Lecture3_2149.pdf>


---

