<p>The first panel shows the final value function after iterating 80k (!). Should probably try an implicit method. The second panel shows the evolution from frist guess to final result $v^0\to v^j$, $j=80k$. Third panel is the distance between the value functions and fourth shows the error. </p>

<p>Note that the value function is convex-concave. This is due to the fact that for the limit case the law of motion tends to infinty $\lim_{k\to 0} \dot{k} = \infty$. </p>

<p>By the maximum principle optimality requires that the Hamiltonian 
\begin{align}
H_1(k,\tau_1,\tau_2^*,\lambda_1) := F(k,\tau_1,\tau_2^*) + \lambda_1 f(k,\tau_1,\tau_2^*)
\end{align}
is concave in $(k,\tau_1,\tau_2)$ (sufficient condition). Now we have<br>
\begin{align}
\rho v(k) = \max_{\tau_1} H_1(k,\tau_1,\tau_2^*,\lambda_1).
\end{align} </p>

<ul>
<li>I was wondering if the value function must be (strictly) concave to ensure optimality?</li>
</ul>

<p>Any reference is appreciated.    </p>

<p><img src="https://i.sstatic.net/SFGwR.jpg" alt="vfi"></p>

CC BY-SA 3.0
2015-06-08T11:29:55.653Z
2015-06-08T11:29:55.653Z
clueless
<p>There are two agents $i=1,2$. The state $k$ is governed by $\tau_i\in[0,1]$ where 
\begin{align}
\dot{k} = f(k,\tau_1,\tau_2).
\end{align}</p>

<p>Define the value function of player $i$ by
\begin{align}
v_i(k) := \sup_{\tau_i}\int^\infty_0{e^{-\rho t}F(k,\tau_i,\sigma_{-i})dt}.
\end{align}
where $\sigma_{-i}(k)$ is the policy function or a markovian stratgey profile of player $-i$. And $\rho&gt;0$ is the time preference rate.</p>

<p>The Hamilton-Jacobi-Bellman equation reads
\begin{align}
\rho v_i(k) = \sup_{\tau_i}\{F(k,\tau_i,\sigma_{-i}) + v'(k)f(k,\tau_i,\sigma_{-i})\}.
\end{align}</p>

<p>Now assume I approximated a function $\hat{v}_i$ by value function iteration. How do I know that it's the true value function? </p>

CC BY-SA 3.0
2015-06-05T09:17:20.047Z
Check the distance with the last iteration. If you like it, you can stop. If it's too big, keep iterating.
VicAche
CC BY-SA 3.0
2015-06-08T12:20:31.000Z
first of all you can solve an analytical example and then test your algorithm against it. Second, the value function is a fixed point, so you can google for some tests of convergence to a fixed point, at least one heurisitic rules comes to mind, the distance between i and i+1 is decreasing , strict concavity ensures global optimality
user157623
<mathematical-economics><dynamic-games><dynamic-programming>
How to verify Value Function in nonzero sum two player Differential Game?
clueless
# How to verify Value Function in nonzero sum two player Differential Game?

(Asked by: username_0 on 2015-06-01)

There are two agents $i=1,2$. The state $k$ is governed by $\tau\_i\in[0,1]$ where
\begin{align}
\dot{k} = f(k,\tau\_1,\tau\_2).
\end{align}

Define the value function of player $i$ by
\begin{align}
v\_i(k) := \sup\_{\tau\_i}\int^\infty\_0{e^{-\rho t}F(k,\tau\_i,\sigma\_{-i})dt}.
\end{align}
where $\sigma\_{-i}(k)$ is the policy function or a markovian stratgey profile of player $-i$. And $\rho>0$ is the time preference rate.

The Hamilton-Jacobi-Bellman equation reads
\begin{align}
\rho v\_i(k) = \sup\_{\tau\_i}\{F(k,\tau\_i,\sigma\_{-i}) + v'(k)f(k,\tau\_i,\sigma\_{-i})\}.
\end{align}

Now assume I approximated a function $\hat{v}\_i$ by value function iteration. How do I know that it's the true value function?

**Tags:** mathematical-economics, dynamic-games, dynamic-programming
> **Comments:**
> **username_1**: Check the distance with the last iteration. If you like it, you can stop. If it's too big, keep iterating.  
> **username_2**: first of all you can solve an analytical example and then test your algorithm against it. Second, the value function is a fixed point, so you can google for some tests of convergence to a fixed point, at least one heurisitic rules comes to mind, the distance between i and i+1 is decreasing , strict concavity ensures global optimality  

---

## 1 answers

### Answer by username_0 on 2015-06-08. Score: 1

The first panel shows the final value function after iterating 80k (!). Should probably try an implicit method. The second panel shows the evolution from frist guess to final result $v^0\to v^j$, $j=80k$. Third panel is the distance between the value functions and fourth shows the error.

Note that the value function is convex-concave. This is due to the fact that for the limit case the law of motion tends to infinty $\lim\_{k\to 0} \dot{k} = \infty$.

By the maximum principle optimality requires that the Hamiltonian
\begin{align}
H\_1(k,\tau\_1,\tau\_2^\*,\lambda\_1) := F(k,\tau\_1,\tau\_2^\*) + \lambda\_1 f(k,\tau\_1,\tau\_2^\*)
\end{align}
is concave in $(k,\tau\_1,\tau\_2)$ (sufficient condition). Now we have  
\begin{align}
\rho v(k) = \max\_{\tau\_1} H\_1(k,\tau\_1,\tau\_2^\*,\lambda\_1).
\end{align}

* I was wondering if the value function must be (strictly) concave to ensure optimality?

Any reference is appreciated.

![vfi](https://i.sstatic.net/SFGwR.jpg)


---

