<p>I will note that I am outsider with an interest in macroeconomics, and so this attempt at an answer only applies to the situation in macroecomics as I see it. </p>

<p>There are effectively three questions here.</p>

<ol>
<li>Was Roehner right (in 2002)? As I discuss below, this question is the most critical, but any answer is going to be controversial.</li>
<li>Has the situation changed since 2002? I don't see any reason why this would be true, but see discussion below.</li>
<li>Are there <em>journals</em> that are keen to publish puzzling empirical data? From the standpoint of macro, that's the wrong question. The right question is: are there <em>articles</em> published on empirical analysis in macro? And the answer is that there is a lot of empirical research published by central banks, supranational bodies, and "think tanks."  If you want to see such empirical work, that is where you look - a reality that Roehner's characterisation skips over.</li>
</ol>

<p>Was Roehner right in 2002? I would decompose his claims into a few levels.</p>

<p>The first level is that the pattern of publications in economics journals (which ones?) is differrent than in physics. I imagine you could look at statistics on the classifications of journal articles, and give an authoritative answer to that question. However, why would anyone expect different academic fields to display the same distribution of article types?</p>

<p>The next level is the question: are there "enough" empirical papers in economics? If you look at developed economy macro, we are only accumulating data points at the rate of 4 quarterly observations per year. Given the general weakness of economic forecasting models, the amount of new "puzzling" data that can be possibly generated is extremely low. There are some empirical puzzles in macro, but the point is that those puzzles have been around for decades, and so there is no value in publishing new articles that just append a few observations to the previous data set. This is a data-generating environment that is utterly unlike the situation in physics. (How many articles on avalanches in sand piles could possibly be published if we only have records of three such avalanches?) Furthermore, such empirical studies largely end up in the central bank research environment, which was excluded by Roehner for unknown reasons. I do not see a good way of publishing authorative study on whether this provides a sufficient weight on empirical work.</p>

<p>(Other areas of economics/finance have less data constraints. For example, finance is heavily empirical. However, the journals Roehner looked at would probably not been the venue for publishing empirical financial papers. That's a defect of Roehner's sampling process, not finance. Some areas are also highly experiment-based, like behavioural economics. My understanding is that these are published in different journals, some of which may have come into existence after 2002. Someone else is going to have to comment on the areas other than finance; I do not think the criticisms carry much weight for the financial literature.) </p>

<p>Finally, one could infer from the quote the implicit criticism that economics journals have a deliberate policy of excluding empirical studies that allegedly contradict the existing paradigm. Such a charge is obviously extremely controversial; in the absence of evidence that such a suppression effort exists, you are not going to get a good answer here. (Can you cite any such "puzzling" data? If not, how do we even know that it exists to be written up in an article?) The controversial nature of this question suggests that it might be broken out into a new, very focussed question. Something like: "what empirical data cannot be observed by standard theories in (some sub-field of economics)?" If nobody here can find anything (that is not in the literature), then it does indicate that there may not be a large potential literature for "puzzling empirical data." That said, the answers will be different for every subfield of economics; there are tons of "puzzles" discussed in finance.</p>

CC BY-SA 3.0
2017-08-03T04:40:50.267Z
Thanks, I've upvoted. But what's about microeconomics? There should be much more data points there....
zer0hedge
CC BY-SA 3.0
2017-08-03T04:44:29.627Z
If it is difficult to find an article with "puzzling empirical observations" then  a suppression effort may be present.
zer0hedge
CC BY-SA 3.0
2017-08-03T11:37:44.280Z
I added some comments about data; not sure about "micro", but finance or behavioural economics are empirical. The idea of suppression is interesting, but the objective of this website is to be authoritative. (My answer is not authoritative, but I am trying to give a starting point for pursuing the question, and to explain why getting answers is awkward. You may need to decompose the question, as I discuss.)
Brian Romanchuk
CC BY-SA 3.0
2017-08-02T19:22:48.647Z
2017-08-03T11:41:30.600Z
2017-08-03T11:41:30.600Z
Brian Romanchuk
<p>Was he right? No.</p>

<p>Has the situation changed since 2002? Yes, there are lots more journals, and much higher diversity of publishing. Sadly, that also means there are also a lot more venues that publish at the bottom of the quality scale: one even shares a name with you.</p>

<p>Are there decent economic journals which are keen to publish puzzling empirical observations? No, generally they'll look for something well-written, novel, intelligent, and within the journal's scope. Just being a puzzling empirical observation simply isn't enough for any decent venue. You'll want a blog for that.</p>

<p>As for a slice of historical publishing of puzzling empirical observations, why not try a <a href="https://scholar.google.co.uk/scholar?q=&quot;productivity+puzzle&quot;&amp;btnG=&amp;hl=en" rel="nofollow noreferrer">Google Scholar search for "productivity puzzle"</a>?</p>

CC BY-SA 3.0
2017-07-31T19:14:05.577Z
You may try to google "avalanches in sandpiles". Much more results than about "productivity puzzle"and all of them about ...  avalanches in sandpiles, not about "More than 50 studies of scientists in various fields show that women publish less than men."  (second link in your search)
zer0hedge
CC BY-SA 3.0
2017-08-02T20:14:10.673Z
How sure are you of your second No? I think I have seen papers with "puzzling" empirical results. I will try to search for references. In any case, it is always more complicated to prove a no than a yes.
luchonacho
CC BY-SA 3.0
2017-08-03T04:32:55.717Z
@luchonacho I'd omitted a crucial word, which I've now edited in.
410 gone
CC BY-SA 3.0
2017-08-03T05:42:18.783Z
I think "decent" is a rather subjective word.
luchonacho
CC BY-SA 3.0
2017-07-31T18:42:32.430Z
2017-08-03T04:32:23.250Z
2017-08-03T04:32:23.250Z
410 gone
<p>In his book <a href="http://www.cambridge.org/catalogue/catalogue.asp?isbn=0521675731" rel="nofollow noreferrer">&quot;Patterns of Speculation: A Study in Observational Econophysics&quot;(2002)</a> Bertrand Roehner makes a bold assertion:</p>
<blockquote>
<p>Physical journals welcome all kinds of experimental observations, especially if they are puzzling and cannot be explained by existing theories. Recent illustrations are the numerous papers about avalanches in sandpiles. Economic journals, on the contrary, are very <strong>reluctant to publish empirical observations, especially when they have no clear interpretation within the existing theoretical framework.</strong></p>
<p>Another feature of great importance is the small volume of economic journals:
the total number of pages published annually in physical journals is probably of the order of 10–20 times larger than for economic journals. As a result the acceptance ratios for papers in economic journals are between 5 percent and 25 percent, which in turn <strong>induces authors to write on the most fashionable topics in the hope of raising the interest of referees, rather than to pursue an original and consistent research project.</strong></p>
</blockquote>
<p>Was he right? Has the situation changed since 2002?  Are there economic journals which are publishing puzzling empirical observations, i.e. those observations which have no clear interpretation within the existing theoretical framework?</p>

CC BY-SA 3.0
2017-07-31T21:13:37.273Z
Noah Smith (Blomberg contributor) has been writing about a shift towards empirical economics within the journals; you could search to find his articles. My interests within economics are narrow; mainly macroeconomics. I am not a fan of mainstream macro, but I do not see this as an issue. I would be hard-pressed to think of data that cannot be "interpreted" by existing macro theoretical frameworks; the usual problem is that the standard frameworks have limited explanatory power. That is, we have the reverse problem than suggested: a difficulty of finding data that could falsify theories.
Brian Romanchuk
CC BY-SA 3.0
2017-08-01T14:25:18.177Z
@BrianRomanchuk We all heard about Karl Popper, falsifiability etc - I actually don't want to go that far. I will accept a binary answer  - Yes(he was right),  No(the situation has not changed) - if the answer will receive a  reasonable number of upvotes from the community. Otherwise I expect name(s) of the journal(s)
zer0hedge
CC BY-SA 3.0
2017-08-01T15:57:13.057Z
Strictly speaking, given the huge number of journals, it is very likely that there are journals publishing apparent puzzling results. I imagine that what Bertrand had in mind were "mainstream/top" journals. There, the question is more complicated (and perhaps more interesting),
luchonacho
CC BY-SA 3.0
2017-08-01T18:37:00.310Z
@luchonacho Good point - I've extended the citation
zer0hedge
CC BY-SA 3.0
2017-08-02T02:04:01.297Z
My point is that if you can't falsify the theory, puzzling data does not exist. Therefore, the lack of such articles is not a surprise, and does not reflect editorial choices. Unless you have some puzzling data that has not been published, why question publication decisions?
Brian Romanchuk
CC BY-SA 3.0
2017-08-02T06:55:33.757Z
@BrianRomanchuk Is there any economic theory which is able to predict?
zer0hedge
CC BY-SA 3.0
2017-08-02T10:57:50.870Z
@zer0hedge The economists here may come up with examples. My opinion is that macro "forecasting" is difficult, but making "predictions" based on conditions ("X will happen if there is a recession") is feasible. My opinions are my take on post-Keynesian theory.
Brian Romanchuk
CC BY-SA 3.0
2017-08-02T18:09:24.493Z
@BrianRomanchuk I doubt that the economists will do. Anyway I feel that my question has been answered already
zer0hedge
CC BY-SA 3.0
2017-08-02T19:44:59.927Z
This question seems like it should be closed for being opinion-based. Also, I'm not sure it's strictly on-topic. Maybe academica.SE would better?
jmbejara
CC BY-SA 3.0
2017-08-03T04:31:07.563Z
@jmbejara I changed the question and removed the word "keen".  I hope it can not be considered "opinion-based" question any more.
zer0hedge
<empirical-evidence><empirical-methods>
Are there economic journals which are keen to publish puzzling empirical observations?
zer0hedge
# Are there economic journals which are keen to publish puzzling empirical observations?

(Asked by: username_0 on 2017-07-31)

In his book ["Patterns of Speculation: A Study in Observational Econophysics"(2002)](http://www.cambridge.org/catalogue/catalogue.asp?isbn=0521675731) Bertrand Roehner makes a bold assertion:

> Physical journals welcome all kinds of experimental observations, especially if they are puzzling and cannot be explained by existing theories. Recent illustrations are the numerous papers about avalanches in sandpiles. Economic journals, on the contrary, are very **reluctant to publish empirical observations, especially when they have no clear interpretation within the existing theoretical framework.**
>
> Another feature of great importance is the small volume of economic journals:
> the total number of pages published annually in physical journals is probably of the order of 10–20 times larger than for economic journals. As a result the acceptance ratios for papers in economic journals are between 5 percent and 25 percent, which in turn **induces authors to write on the most fashionable topics in the hope of raising the interest of referees, rather than to pursue an original and consistent research project.**

Was he right? Has the situation changed since 2002? Are there economic journals which are publishing puzzling empirical observations, i.e. those observations which have no clear interpretation within the existing theoretical framework?

**Tags:** empirical-evidence, empirical-methods
> **Comments:**
> **username_1**: Noah Smith (Blomberg contributor) has been writing about a shift towards empirical economics within the journals; you could search to find his articles. My interests within economics are narrow; mainly macroeconomics. I am not a fan of mainstream macro, but I do not see this as an issue. I would be hard-pressed to think of data that cannot be "interpreted" by existing macro theoretical frameworks; the usual problem is that the standard frameworks have limited explanatory power. That is, we have the reverse problem than suggested: a difficulty of finding data that could falsify theories.  
> **username_0**: @BrianRomanchuk We all heard about Karl Popper, falsifiability etc - I actually don't want to go that far. I will accept a binary answer  - Yes(he was right),  No(the situation has not changed) - if the answer will receive a  reasonable number of upvotes from the community. Otherwise I expect name(s) of the journal(s)  
> **username_2**: Strictly speaking, given the huge number of journals, it is very likely that there are journals publishing apparent puzzling results. I imagine that what Bertrand had in mind were "mainstream/top" journals. There, the question is more complicated (and perhaps more interesting),  
> **username_0**: @luchonacho Good point - I've extended the citation  
> **username_1**: My point is that if you can't falsify the theory, puzzling data does not exist. Therefore, the lack of such articles is not a surprise, and does not reflect editorial choices. Unless you have some puzzling data that has not been published, why question publication decisions?  
> **username_0**: @BrianRomanchuk Is there any economic theory which is able to predict?  
> **username_1**: @zer0hedge The economists here may come up with examples. My opinion is that macro "forecasting" is difficult, but making "predictions" based on conditions ("X will happen if there is a recession") is feasible. My opinions are my take on post-Keynesian theory.  
> **username_0**: @BrianRomanchuk I doubt that the economists will do. Anyway I feel that my question has been answered already  
> **username_3**: This question seems like it should be closed for being opinion-based. Also, I'm not sure it's strictly on-topic. Maybe academica.SE would better?  
> **username_0**: @jmbejara I changed the question and removed the word "keen".  I hope it can not be considered "opinion-based" question any more.  

---

## 2 answers

### Answer by username_1 on 2017-08-02. Score: 1 | Accepted answer

I will note that I am outsider with an interest in macroeconomics, and so this attempt at an answer only applies to the situation in macroecomics as I see it.

There are effectively three questions here.

1. Was Roehner right (in 2002)? As I discuss below, this question is the most critical, but any answer is going to be controversial.
2. Has the situation changed since 2002? I don't see any reason why this would be true, but see discussion below.
3. Are there *journals* that are keen to publish puzzling empirical data? From the standpoint of macro, that's the wrong question. The right question is: are there *articles* published on empirical analysis in macro? And the answer is that there is a lot of empirical research published by central banks, supranational bodies, and "think tanks." If you want to see such empirical work, that is where you look - a reality that Roehner's characterisation skips over.

Was Roehner right in 2002? I would decompose his claims into a few levels.

The first level is that the pattern of publications in economics journals (which ones?) is differrent than in physics. I imagine you could look at statistics on the classifications of journal articles, and give an authoritative answer to that question. However, why would anyone expect different academic fields to display the same distribution of article types?

The next level is the question: are there "enough" empirical papers in economics? If you look at developed economy macro, we are only accumulating data points at the rate of 4 quarterly observations per year. Given the general weakness of economic forecasting models, the amount of new "puzzling" data that can be possibly generated is extremely low. There are some empirical puzzles in macro, but the point is that those puzzles have been around for decades, and so there is no value in publishing new articles that just append a few observations to the previous data set. This is a data-generating environment that is utterly unlike the situation in physics. (How many articles on avalanches in sand piles could possibly be published if we only have records of three such avalanches?) Furthermore, such empirical studies largely end up in the central bank research environment, which was excluded by Roehner for unknown reasons. I do not see a good way of publishing authorative study on whether this provides a sufficient weight on empirical work.

(Other areas of economics/finance have less data constraints. For example, finance is heavily empirical. However, the journals Roehner looked at would probably not been the venue for publishing empirical financial papers. That's a defect of Roehner's sampling process, not finance. Some areas are also highly experiment-based, like behavioural economics. My understanding is that these are published in different journals, some of which may have come into existence after 2002. Someone else is going to have to comment on the areas other than finance; I do not think the criticisms carry much weight for the financial literature.)

Finally, one could infer from the quote the implicit criticism that economics journals have a deliberate policy of excluding empirical studies that allegedly contradict the existing paradigm. Such a charge is obviously extremely controversial; in the absence of evidence that such a suppression effort exists, you are not going to get a good answer here. (Can you cite any such "puzzling" data? If not, how do we even know that it exists to be written up in an article?) The controversial nature of this question suggests that it might be broken out into a new, very focussed question. Something like: "what empirical data cannot be observed by standard theories in (some sub-field of economics)?" If nobody here can find anything (that is not in the literature), then it does indicate that there may not be a large potential literature for "puzzling empirical data." That said, the answers will be different for every subfield of economics; there are tons of "puzzles" discussed in finance.

> **Comments:**
> **username_0**: Thanks, I've upvoted. But what's about microeconomics? There should be much more data points there....  
> **username_0**: If it is difficult to find an article with "puzzling empirical observations" then  a suppression effort may be present.  
> **username_1**: I added some comments about data; not sure about "micro", but finance or behavioural economics are empirical. The idea of suppression is interesting, but the objective of this website is to be authoritative. (My answer is not authoritative, but I am trying to give a starting point for pursuing the question, and to explain why getting answers is awkward. You may need to decompose the question, as I discuss.)  

---

### Answer by username_4 on 2017-07-31. Score: 1

Was he right? No.

Has the situation changed since 2002? Yes, there are lots more journals, and much higher diversity of publishing. Sadly, that also means there are also a lot more venues that publish at the bottom of the quality scale: one even shares a name with you.

Are there decent economic journals which are keen to publish puzzling empirical observations? No, generally they'll look for something well-written, novel, intelligent, and within the journal's scope. Just being a puzzling empirical observation simply isn't enough for any decent venue. You'll want a blog for that.

As for a slice of historical publishing of puzzling empirical observations, why not try a [Google Scholar search for "productivity puzzle"](https://scholar.google.co.uk/scholar?q=)?

> **Comments:**
> **username_0**: You may try to google "avalanches in sandpiles". Much more results than about "productivity puzzle"and all of them about ...  avalanches in sandpiles, not about "More than 50 studies of scientists in various fields show that women publish less than men."  (second link in your search)  
> **username_2**: How sure are you of your second No? I think I have seen papers with "puzzling" empirical results. I will try to search for references. In any case, it is always more complicated to prove a no than a yes.  
> **username_4**: @luchonacho I'd omitted a crucial word, which I've now edited in.  
> **username_2**: I think "decent" is a rather subjective word.  

---

