<p>There is one theoretically sound methodology to do what you are asking.  When you have multiple models to compare, the only choice is Bayesian model selection and Bayesian model averaging.  </p>

<p>Bayesian methods have several properties that you would want.  Any statistic created is intrinsically "admissible," and "coherent."  The disadvantage, which does not exist for Frequentist methods, is that it is built on inductive reasoning rather than deductive reasoning.  An article on this can be found at <a href="https://plato.stanford.edu/entries/induction-problem/" rel="nofollow noreferrer">https://plato.stanford.edu/entries/induction-problem/</a></p>

<p>Induction only allows you to test models you have, but cannot test models you do not have and so will always be incomplete.  If you reject a Frequentist null hypothesis, then you reject all theories except the null.  The problem with this happens when the model is complex and cannot be turned into a pure binary hypothesis.  It is worse when more than one model rejects its null.  If that happens then, Frequentist theory is silent on what to do.</p>

<p>A statistic is admissible if there is no less risky way to create it and it is coherent if you could place a fair gamble on it.  The idea of a parameter is more flexible in Bayesian thinking than Frequentist.  A model is a parameter in Bayesian thinking.  As it is a parameter, you can construct a probability statement around it.  Bayes theorem naturally rewards better fitting models, while penalizing models for more structure.  It is a natural Occam's razor with a mathematical interpretation.  It also simply sets aside Friedman's defense as unnecessary.</p>

<p>One of the better axiomatic constructions for Bayesian statistics is Cox's.  You can find it at </p>

<blockquote>
  <p>Richard T. Cox. The Algebra of Probable Inference. 1961. Johns Hopkins University Press. Baltimore, MD</p>
</blockquote>

<p>It assigns "plausibilities" to logical assertions.  Usually as odds or probabilities.  Surprisingly, it is worth reading.  If you are doing model or theory comparison, then, more so, this is the case.</p>

CC BY-SA 3.0
2017-07-25T05:22:24.073Z
2017-07-25T05:22:24.073Z
Dave Harris
<p>Based on <a href="https://economics.stackexchange.com/questions/15688/which-are-the-different-schools-of-economic-thought">this question</a> about alternative schools of thought, <a href="https://economics.stackexchange.com/questions/17554/most-important-but-untested-theories-in-economics">this question</a> about untested economic theories, and the rather unsatisfactory answer (imo) to <a href="https://economics.stackexchange.com/questions/5352/what-are-some-growth-theories-that-are-considered-the-most-correct-at-the-moment">this question</a> about which is the "most correct" growth theory at the moment, it would be great to learn more about <strong>how to compare economic models or theories</strong>. Anyone familiar with this literature, which I think belongs to philosophy and methodology of economics?</p>

<p>I have rudimentary ideas here, like <a href="https://en.wikipedia.org/wiki/Occam%27s_razor" rel="nofollow noreferrer">Occam's razor</a> and Friedman's <a href="https://en.wikipedia.org/wiki/Essays_in_Positive_Economics#The_Methodology_of_Positive_Economics" rel="nofollow noreferrer">defense of unrealistic assumptions</a>, but this is deeply unsatisfactory for such an important question (imo).</p>

<methodology><philosophy-of-economics>
Which are some proven methods to compare alternative economic theories?
luchonacho
# Which are some proven methods to compare alternative economic theories?

(Asked by: username_0 on 2017-07-24)

Based on [this question](https://economics.stackexchange.com/questions/15688/which-are-the-different-schools-of-economic-thought) about alternative schools of thought, [this question](https://economics.stackexchange.com/questions/17554/most-important-but-untested-theories-in-economics) about untested economic theories, and the rather unsatisfactory answer (imo) to [this question](https://economics.stackexchange.com/questions/5352/what-are-some-growth-theories-that-are-considered-the-most-correct-at-the-moment) about which is the "most correct" growth theory at the moment, it would be great to learn more about **how to compare economic models or theories**. Anyone familiar with this literature, which I think belongs to philosophy and methodology of economics?

I have rudimentary ideas here, like [Occam's razor](https://en.wikipedia.org/wiki/Occam%27s_razor) and Friedman's [defense of unrealistic assumptions](https://en.wikipedia.org/wiki/Essays_in_Positive_Economics#The_Methodology_of_Positive_Economics), but this is deeply unsatisfactory for such an important question (imo).

**Tags:** methodology, philosophy-of-economics

---

## 1 answers

### Answer by username_1 on 2017-07-25. Score: 3

There is one theoretically sound methodology to do what you are asking. When you have multiple models to compare, the only choice is Bayesian model selection and Bayesian model averaging.

Bayesian methods have several properties that you would want. Any statistic created is intrinsically "admissible," and "coherent." The disadvantage, which does not exist for Frequentist methods, is that it is built on inductive reasoning rather than deductive reasoning. An article on this can be found at <https://plato.stanford.edu/entries/induction-problem/>

Induction only allows you to test models you have, but cannot test models you do not have and so will always be incomplete. If you reject a Frequentist null hypothesis, then you reject all theories except the null. The problem with this happens when the model is complex and cannot be turned into a pure binary hypothesis. It is worse when more than one model rejects its null. If that happens then, Frequentist theory is silent on what to do.

A statistic is admissible if there is no less risky way to create it and it is coherent if you could place a fair gamble on it. The idea of a parameter is more flexible in Bayesian thinking than Frequentist. A model is a parameter in Bayesian thinking. As it is a parameter, you can construct a probability statement around it. Bayes theorem naturally rewards better fitting models, while penalizing models for more structure. It is a natural Occam's razor with a mathematical interpretation. It also simply sets aside Friedman's defense as unnecessary.

One of the better axiomatic constructions for Bayesian statistics is Cox's. You can find it at

> Richard T. Cox. The Algebra of Probable Inference. 1961. Johns Hopkins University Press. Baltimore, MD

It assigns "plausibilities" to logical assertions. Usually as odds or probabilities. Surprisingly, it is worth reading. If you are doing model or theory comparison, then, more so, this is the case.


---

