<p>It seems that your transition equation is not standard. I took the liberty to change it like this:
<span class="math-container">$$
\begin{align}
&amp;\max \sum^T_{t=0} \beta^t ( u(c_t) + \gamma u(c_{t-1}) ), \\
&amp;\text{s.t. } c_t + w_{t+1} = (1+r)w_t + y_t.
\end{align}
$$</span>
In your formulation, both income and consumption yield interest, which is not normal. So, I wrote in a form that the LHS shows the expenses, and the RHS shows the source of funding. And <span class="math-container">$y_t = f(w_t)$</span> where <span class="math-container">$f'&gt;0,f''&lt;0$</span>. (By the way, <span class="math-container">$w$</span> seems like a weird choice of the variable for assets because it normally represents wage. I think it's better to use <span class="math-container">$k$</span>.)</p>
<p>There will be 2 state variables <span class="math-container">$(c_{t-1}, w_t)$</span>. Based on that, you can write the Bellman equation</p>
<p><span class="math-container">$$
V(c_{t-1},w_t) = \max_{c_t, w_{t+1}} \left( u(c_t) + \gamma u(c_{t-1}) + \beta V(c_t, w_{t+1}) \right).
$$</span>
Using the constraint, we can reduce it to a problem of 1 choice variable
<span class="math-container">$$
V(c_{t-1},w_t) = \max_{w_{t+1}} \left( u( \underbrace{(1+r)w_t + f(w_t) - w_{t+1}}_{c_t} ) + \gamma u(c_{t-1}) + \beta V( \underbrace{(1+r)w_t + f(w_t) - w_{t+1}}_{c_t}, w_{t+1}) \right).
$$</span>
Your choice variable is <span class="math-container">$w_{t+1}$</span>, so you take FOC with respect to it
<span class="math-container">$$
u'(c_t) + \beta V_1(c_t, w_{t+1}) = \beta V_2(c_t, w_{t+1}).
$$</span>
The envelope theorem conditions are
<span class="math-container">$$
\begin{align}
 V_1(c_{t-1}, w_t) &amp;= \frac{\partial V}{\partial c_{t-1}} = \gamma u'(c_{t-1}), \\
 V_2(c_{t-1}, w_t) &amp;= \frac{\partial V}{\partial w_t} = [(1+r) + f'(w_t)]u'(c_t) + \beta [(1+r) + f'(w_t)] V_1 (c_t, w_{t+1}).
\end{align}
$$</span>
Forwarding one period, we have
<span class="math-container">$$
\begin{align}
  V_1(c_t, w_{t+1}) &amp;= \gamma u'(c_t), \\
  V_2(c_t, w_{t+1}) &amp;= [(1+r) + f'(w_{t+1})](u'(c_{t+1}) + \beta V_1(c_{t+1}, w_{t+2})].
\end{align}
$$</span>
Combining these 2 yields
<span class="math-container">$$
V_2 (c_t, w_{t+1}) = [(1+r) + f'(w_{t+1})](1 + \beta \gamma) u'(c_{t+1}).
$$</span>
Replacing <span class="math-container">$V_1(c_t, w_{t+1})$</span> and <span class="math-container">$V_2(c_t, w_{t+1})$</span> back to the FOC to get
<span class="math-container">$$
u'(c_t) + \beta\gamma u'(c_t) = \beta[(1+r) + f'(w_{t+1})](1 + \beta \gamma) u'(c_{t+1}),
$$</span>
which can be written as
<span class="math-container">$$
(1+\beta\gamma) u'(c_t) = \beta [(1+r) + f'(w_{t+1})](1 + \beta \gamma) u'(c_{t+1}).
$$</span>
Cancelling common terms and rearranging give you the Euler equation
<span class="math-container">$$
 \frac{u'(c_t)}{u'(c_{t+1})} = \beta[(1+r) + f'(w_{t+1})],
$$</span>
which does not depend on the lagged consumption term. <a href="https://www.hup.harvard.edu/catalog.php?isbn=9780674274761" rel="nofollow noreferrer">Sargent (1987)</a> explains that the choice of <span class="math-container">$c$</span> involves comparing the loss in current marginal utility and the gain in future consumption so the state <span class="math-container">$c_{-1}$</span> does not influence the choice of <span class="math-container">$c$</span>. It only has a scale effect on the utility function.</p>
<p>From here, you can use Guess and Verify to work out the Value function. Since we have 2 state variables, the usual form should be
<span class="math-container">$$
 V(c_{t-1}, w_t) = E + F \ln w_t + G \ln c_{t-1}.
$$</span>
Since the Euler equation is the same as a problem without lagged consumption, you can guess that the Value Function is identical to a problem without Habit Formation.</p>

CC BY-SA 4.0
2023-08-25T05:45:01.813Z
2023-08-25T05:45:01.813Z
teddi
<p>Taking the habit formation model of consumption, as a standard dynamic programming problem.</p>
<p><strong>Bellman Value Function for Habit Model</strong></p>
<p>Max<span class="math-container">$\sum_{t=1}^Tβ^tu(c_t - γc_{t-1})$</span> <span class="math-container">$\qquad \qquad (1)$</span></p>
<p>s.t.
<span class="math-container">$w_{t+1} = (1+r)(w_t + y_t - c_t)$</span>,</p>
<p>Here <span class="math-container">$c_t$</span> is consumption at time <span class="math-container">$t$</span>, <span class="math-container">$w_t$</span> are financial assets yielding constant net interest <span class="math-container">$r$</span>, <span class="math-container">$y_t$</span> is labour income.</p>
<ol>
<li><p><strong>How would I determine the state variables</strong>. They are stated as <span class="math-container">$w_t$</span> and <span class="math-container">$c_{t-1}$</span> I believe this is because these change endogenously within the model, and they both impact future consumption. However I'm not satisfied with this answer, and i'm not 100% certain why <span class="math-container">$y_t$</span> isn't a state variable.</p>
</li>
<li><p><strong>Discount Factor:</strong></p>
</li>
</ol>
<p>In general these models always end up with a Bellman Equation looking something like:</p>
<p><span class="math-container">$V(w_t, c_{t-1}, t) = max_{c_t} {u(c_t, c_{t-1}) + β V(w_{t+1}, c_t, t+1)}$</span> <span class="math-container">$\qquad \qquad (2)$</span></p>
<p><span class="math-container">$s.t. w_{t+1} = (1+r)(w_t + y_t - c_t)$</span></p>
<p>I understand how to derive the Bellman equation when we don't have a discount factor, but how do we derive the equation with a discount factor, particularly so that the <span class="math-container">$t$</span> power on <span class="math-container">$β$</span>  goes away.</p>
<ol start="3">
<li><strong>Is my understanding of the model correct:</strong></li>
</ol>
<ul>
<li><span class="math-container">$V(w_t, c_{t-1},t)$</span> is the value function, which represents the lifetime utility of an agent who starts with initial wealth from financial assets <span class="math-container">$w_t$</span> and previous consumption <span class="math-container">$c_{t-1}$</span> at time <span class="math-container">$t$</span> and makes optimal consumption decisions over their lifetime.</li>
<li><span class="math-container">$V(w_t, c_{t-1},t+1)$</span> is the value function for our <span class="math-container">$t+1$</span> up to time <span class="math-container">$T$</span> i.e. it captures the value from optimal decision making, working backwards from terminal time point <span class="math-container">$T$</span> .i.e. through backwards induction?</li>
<li>Our problem is to maximise consumption decisions maximise value today as given by  <span class="math-container">$u(c_t - γc_{t-1})$</span> and future value as given by <span class="math-container">$V(w_t, c_{t-1},t+1)$</span></li>
<li><span class="math-container">$u(c_t - γc_{t-1})$</span> is the utility function, which captures the instantaneous utility that an individual derives from consuming <span class="math-container">$c_t$</span> at time <span class="math-container">$t$</span>, given their previous consumption <span class="math-container">$c_{t-1}$</span>.</li>
<li><span class="math-container">$\beta$</span> is the discount factor, which determines the individual's time preference for consumption.</li>
</ul>

CC BY-SA 4.0
2024-01-22T23:39:17.607Z
A "state variable" is called "state" because it characterizes the "state" of the situation, prior to flow activity. Hence wealth at the beginning of each period describes the state at which you start the period as regards accumulated resources, and past consumption describes/quantifies the state as regards habit formation. The $y$ variable, apparently income for the period is a flow that can affect the state you started at.
Alecos Papadopoulos
<optimization><dynamic-programming><model>
Habit forming Model & State Variables
CormJack
# Habit forming Model & State Variables

(Asked by: username_0 on 2023-04-14)

Taking the habit formation model of consumption, as a standard dynamic programming problem.

**Bellman Value Function for Habit Model**

Max$\sum\_{t=1}^Tβ^tu(c\_t - γc\_{t-1})$ $\qquad \qquad (1)$

s.t.
$w\_{t+1} = (1+r)(w\_t + y\_t - c\_t)$,

Here $c\_t$ is consumption at time $t$, $w\_t$ are financial assets yielding constant net interest $r$, $y\_t$ is labour income.

1. **How would I determine the state variables**. They are stated as $w\_t$ and $c\_{t-1}$ I believe this is because these change endogenously within the model, and they both impact future consumption. However I'm not satisfied with this answer, and i'm not 100% certain why $y\_t$ isn't a state variable.
2. **Discount Factor:**

In general these models always end up with a Bellman Equation looking something like:

$V(w\_t, c\_{t-1}, t) = max\_{c\_t} {u(c\_t, c\_{t-1}) + β V(w\_{t+1}, c\_t, t+1)}$ $\qquad \qquad (2)$

$s.t. w\_{t+1} = (1+r)(w\_t + y\_t - c\_t)$

I understand how to derive the Bellman equation when we don't have a discount factor, but how do we derive the equation with a discount factor, particularly so that the $t$ power on $β$  goes away.

3. **Is my understanding of the model correct:**

* $V(w\_t, c\_{t-1},t)$ is the value function, which represents the lifetime utility of an agent who starts with initial wealth from financial assets $w\_t$ and previous consumption $c\_{t-1}$ at time $t$ and makes optimal consumption decisions over their lifetime.
* $V(w\_t, c\_{t-1},t+1)$ is the value function for our $t+1$ up to time $T$ i.e. it captures the value from optimal decision making, working backwards from terminal time point $T$ .i.e. through backwards induction?
* Our problem is to maximise consumption decisions maximise value today as given by $u(c\_t - γc\_{t-1})$ and future value as given by $V(w\_t, c\_{t-1},t+1)$
* $u(c\_t - γc\_{t-1})$ is the utility function, which captures the instantaneous utility that an individual derives from consuming $c\_t$ at time $t$, given their previous consumption $c\_{t-1}$.
* $\beta$ is the discount factor, which determines the individual's time preference for consumption.

**Tags:** optimization, dynamic-programming, model
> **Comments:**
> **username_1**: A "state variable" is called "state" because it characterizes the "state" of the situation, prior to flow activity. Hence wealth at the beginning of each period describes the state at which you start the period as regards accumulated resources, and past consumption describes/quantifies the state as regards habit formation. The $y$ variable, apparently income for the period is a flow that can affect the state you started at.  

---

## 1 answers

### Answer by username_2 on 2023-08-25. Score: 1 | Accepted answer

It seems that your transition equation is not standard. I took the liberty to change it like this:
$$
\begin{align}
&\max \sum^T\_{t=0} \beta^t ( u(c\_t) + \gamma u(c\_{t-1}) ), \\
&\text{s.t. } c\_t + w\_{t+1} = (1+r)w\_t + y\_t.
\end{align}
$$
In your formulation, both income and consumption yield interest, which is not normal. So, I wrote in a form that the LHS shows the expenses, and the RHS shows the source of funding. And $y\_t = f(w\_t)$ where $f'>0,f''<0$. (By the way, $w$ seems like a weird choice of the variable for assets because it normally represents wage. I think it's better to use $k$.)

There will be 2 state variables $(c\_{t-1}, w\_t)$. Based on that, you can write the Bellman equation

$$
V(c\_{t-1},w\_t) = \max\_{c\_t, w\_{t+1}} \left( u(c\_t) + \gamma u(c\_{t-1}) + \beta V(c\_t, w\_{t+1}) \right).
$$
Using the constraint, we can reduce it to a problem of 1 choice variable
$$
V(c\_{t-1},w\_t) = \max\_{w\_{t+1}} \left( u( \underbrace{(1+r)w\_t + f(w\_t) - w\_{t+1}}\_{c\_t} ) + \gamma u(c\_{t-1}) + \beta V( \underbrace{(1+r)w\_t + f(w\_t) - w\_{t+1}}\_{c\_t}, w\_{t+1}) \right).
$$
Your choice variable is $w\_{t+1}$, so you take FOC with respect to it
$$
u'(c\_t) + \beta V\_1(c\_t, w\_{t+1}) = \beta V\_2(c\_t, w\_{t+1}).
$$
The envelope theorem conditions are
$$
\begin{align}
V\_1(c\_{t-1}, w\_t) &= \frac{\partial V}{\partial c\_{t-1}} = \gamma u'(c\_{t-1}), \\
V\_2(c\_{t-1}, w\_t) &= \frac{\partial V}{\partial w\_t} = [(1+r) + f'(w\_t)]u'(c\_t) + \beta [(1+r) + f'(w\_t)] V\_1 (c\_t, w\_{t+1}).
\end{align}
$$
Forwarding one period, we have
$$
\begin{align}
V\_1(c\_t, w\_{t+1}) &= \gamma u'(c\_t), \\
V\_2(c\_t, w\_{t+1}) &= [(1+r) + f'(w\_{t+1})](u'(c\_{t+1}) + \beta V\_1(c\_{t+1}, w\_{t+2})].
\end{align}
$$
Combining these 2 yields
$$
V\_2 (c\_t, w\_{t+1}) = [(1+r) + f'(w\_{t+1})](1 + \beta \gamma) u'(c\_{t+1}).
$$
Replacing $V\_1(c\_t, w\_{t+1})$ and $V\_2(c\_t, w\_{t+1})$ back to the FOC to get
$$
u'(c\_t) + \beta\gamma u'(c\_t) = \beta[(1+r) + f'(w\_{t+1})](1 + \beta \gamma) u'(c\_{t+1}),
$$
which can be written as
$$
(1+\beta\gamma) u'(c\_t) = \beta [(1+r) + f'(w\_{t+1})](1 + \beta \gamma) u'(c\_{t+1}).
$$
Cancelling common terms and rearranging give you the Euler equation
$$
\frac{u'(c\_t)}{u'(c\_{t+1})} = \beta[(1+r) + f'(w\_{t+1})],
$$
which does not depend on the lagged consumption term. [Sargent (1987)](https://www.hup.harvard.edu/catalog.php?isbn=9780674274761) explains that the choice of $c$ involves comparing the loss in current marginal utility and the gain in future consumption so the state $c\_{-1}$ does not influence the choice of $c$. It only has a scale effect on the utility function.

From here, you can use Guess and Verify to work out the Value function. Since we have 2 state variables, the usual form should be
$$
V(c\_{t-1}, w\_t) = E + F \ln w\_t + G \ln c\_{t-1}.
$$
Since the Euler equation is the same as a problem without lagged consumption, you can guess that the Value Function is identical to a problem without Habit Formation.


---

