<p>I can think of two big ways.</p>

<ol>
<li>Formal tests of model fit: If you are using SMM, GMM or indirect influence check out the <a href="http://en.wikipedia.org/wiki/Generalized_method_of_moments#J-test" rel="nofollow">J-stat</a>. If you are using maximum likelihood you want a <a href="http://en.wikipedia.org/wiki/Likelihood-ratio_test" rel="nofollow">likelihood ratio</a>. <a href="http://en.wikipedia.org/wiki/Bayes_factor" rel="nofollow">Bayesian model testing</a> is similar to likelihood ratios, but more complicated. Many of the DSGE models were rejected on these bases (you can see Sargent discussing it briefly in <a href="https://files.nyu.edu/ts43/public/research/SargentinterviewMD.pdf" rel="nofollow">this interview</a>). These tests are all very similar.</li>
<li>Out of sample performance: Estimate the model, then use it to predict out of sample data. This is a tough test, but it is super useful. They aren't so common in economics, but statisticians use them a lot (often times they leave out part of the data and try the model out there). You probably want your model to pass a formal test in sample before you do an out of sample test.</li>
</ol>

<p>All these tests rely on the same, extremely basic, intuition: If the model is good then it should fit the data well. If the model does not fit the data well it is probably not good.</p>

CC BY-SA 3.0
2015-02-17T19:27:57.323Z
2015-02-17T19:27:57.323Z
jayk
<p>According to Spanos 2014 <a href="https://secure.hosting.vt.edu/www.econ.vt.edu/directory/spanos/spanos9.pdf">Revisiting Haavelmo's Structural econometrics: Bridging the gap between theory and data</a> Dynamic Stochastic General Equilibrium models are statistically inadequate, in such an order of magnitude as to render them useless.</p>

<blockquote>
  <p>The most serious problem with DSGE models is that they are often statistically inadequate. Advocates of DSGE modeling, however, have
  put forward a number of arguments to explain away the unreliability of
  their evidence, such as ‘the price one has to pay for rigorous and
  policy-relevant theoretical models’ (Lucas, 1980, p. 696), or simply
  inevitable: “The models constructed within this theoretical framework
  are necessarily highly abstract. Consequently, they are necessarily
  false, and statistical hypothesis testing will reject them. This does
  not imply, however, that nothing can be learned from such quantitative
  theoretical exercises.” (Prescott, 1986, p. 10)</p>
</blockquote>

<p>Part of my research involves using a partial equilibrium models to simulate a particular segment of an economy.</p>

<p>Considering the above, what methods would you start with to show that a given equilibrium model is statistically unreliable?</p>

<macroeconomics><econometrics><general-equilibrium><dsge>
What methods - inspired by Haavelmo’s Structural Econometrics - can show that a partial equilibrium model is unreliable?
Paul Rougieux
# What methods - inspired by Haavelmo’s Structural Econometrics - can show that a partial equilibrium model is unreliable?

(Asked by: username_0 on 2015-02-13)

According to Spanos 2014 [Revisiting Haavelmo's Structural econometrics: Bridging the gap between theory and data](https://secure.hosting.vt.edu/www.econ.vt.edu/directory/spanos/spanos9.pdf) Dynamic Stochastic General Equilibrium models are statistically inadequate, in such an order of magnitude as to render them useless.

> The most serious problem with DSGE models is that they are often statistically inadequate. Advocates of DSGE modeling, however, have
> put forward a number of arguments to explain away the unreliability of
> their evidence, such as ‘the price one has to pay for rigorous and
> policy-relevant theoretical models’ (Lucas, 1980, p. 696), or simply
> inevitable: “The models constructed within this theoretical framework
> are necessarily highly abstract. Consequently, they are necessarily
> false, and statistical hypothesis testing will reject them. This does
> not imply, however, that nothing can be learned from such quantitative
> theoretical exercises.” (Prescott, 1986, p. 10)

Part of my research involves using a partial equilibrium models to simulate a particular segment of an economy.

Considering the above, what methods would you start with to show that a given equilibrium model is statistically unreliable?

**Tags:** macroeconomics, econometrics, general-equilibrium, dsge

---

## 1 answers

### Answer by username_1 on 2015-02-17. Score: 2

I can think of two big ways.

1. Formal tests of model fit: If you are using SMM, GMM or indirect influence check out the [J-stat](http://en.wikipedia.org/wiki/Generalized_method_of_moments#J-test). If you are using maximum likelihood you want a [likelihood ratio](http://en.wikipedia.org/wiki/Likelihood-ratio_test). [Bayesian model testing](http://en.wikipedia.org/wiki/Bayes_factor) is similar to likelihood ratios, but more complicated. Many of the DSGE models were rejected on these bases (you can see Sargent discussing it briefly in [this interview](https://files.nyu.edu/ts43/public/research/SargentinterviewMD.pdf)). These tests are all very similar.
2. Out of sample performance: Estimate the model, then use it to predict out of sample data. This is a tough test, but it is super useful. They aren't so common in economics, but statisticians use them a lot (often times they leave out part of the data and try the model out there). You probably want your model to pass a formal test in sample before you do an out of sample test.

All these tests rely on the same, extremely basic, intuition: If the model is good then it should fit the data well. If the model does not fit the data well it is probably not good.


---

