<p>There is no problem with infinite variance regressors. The assumption is made because it results in consistency at the rate of the square root of the sample size. This is standard and common, thus it is presented in this way in most textbooks.</p>
<p>In the case of an infinite variance regressor, the OLS estimate converges to the truth at a rate faster than <span class="math-container">$\sqrt{N}$</span>, and is &quot;superconsistent&quot;.</p>
<p>Consider the model: <span class="math-container">$y_i =\beta_0 +\beta_1 x_{1i}+u_i$</span>.</p>
<p>The OLS estimator of <span class="math-container">$\beta_1$</span> is <span class="math-container">$$\hat{\beta_1} =\beta_1 \frac{Var(x)}{Var(x)} + \frac{Cov(x,u)}{Var(x)}$$</span></p>
<p>Consider the case of a mean-zero regressor for convenience: <span class="math-container">$$\hat{\beta_1} =\beta_1 +\frac{\frac{1}{N}\sum_{i=1}^Nx_iu_i}{\frac{1}{N}\sum_{i=1}^Nx_i^2} $$</span></p>
<p><span class="math-container">$$\sqrt{N}(\hat{\beta_1} -\beta_1)=\frac{\frac{1}{\sqrt{N}}\sum_{i=1}^Nx_iu_i}{\frac{1}{N}\sum_{i=1}^Nx_i^2} $$</span></p>
<p>If a CLT applies to the numerator and the LLN applies to the denominator (i.e., the regressor has finite variance), then the OLS estimate converges at the rate of <span class="math-container">$\sqrt{N}$</span>. If there is an infinite variance regressor, then the denominator in the above expression does not converge. Suppose however, that <span class="math-container">$\frac{1}{N^2}\sum_{i=1}^Nx_i^2$</span> converges to a non-zero constant, then,</p>
<p><span class="math-container">$$N^{3/2}(\hat{\beta_1} -\beta_1)=\frac{\frac{1}{\sqrt{N}}\sum_{i=1}^Nx_iu_i}{\frac{1}{N^2}\sum_{i=1}^Nx_i^2} $$</span></p>
<p>Again, a CLT applies to the numerator, and the denominator converges. Thus, the OLS estimate converges at the rate <span class="math-container">$N^{3/2}$</span>.  This is &quot;superconsistency&quot; and a case that most basic textbooks rule out for ease of exposition.</p>

CC BY-SA 4.0
2022-01-31T16:08:47.163Z
Do you have any textbooks or extended reading on this topic? I note that this answer has much less divergence - I am not interested in finance per se so it is quite clear.
RegressForward
CC BY-SA 4.0
2022-01-31T16:21:36.410Z
Hamilton's Time Series Analysis, section “Asymptotic Properties of a First-Order Autoregression when the True Coefficient is Unity” (17.4).
Michael Gmeiner
CC BY-SA 4.0
2022-01-31T16:21:58.287Z
This is most prominent in time series econometrics in which regressors have unit roots or are otherwise non-stationary, giving them infinite variance.
Michael Gmeiner
CC BY-SA 4.0
2022-01-31T16:49:03.390Z
Ah, I should have known better than to think it would be anywhere else!
RegressForward
CC BY-SA 4.0
2022-01-30T17:58:27.960Z
2022-01-31T07:58:04.410Z
2022-01-31T07:58:04.410Z
Michael Gmeiner
<p>So let us begin with a real case where the variance has to be infinite, the stock market.</p>
<p>I am not doing a complete derivation here as it would run as multiple chapters.  I know this because I am doing web presentations on this.  But I can do enough to show you that there cannot be a first moment for the stock market.  From that, I will explain the consequence for regressors.</p>
<p>Let us begin with a couple of simple observations about a return or a reward for investing.</p>
<p>If we define a return on a single pair of cash flows as <span class="math-container">$$r_t=\frac{p_{t+1}}{p_t}\times\frac{q_{t+1}}{q_t}-1$$</span> and <span class="math-container">$$R_t=r_t+1$$</span> we can note a few things.</p>
<p>First, <span class="math-container">$$R_t=R(p_t,p_{t+1},q_t,q_{t+1}).$$</span>  By definition, prices are data.  Volumes are data. Returns are a function of data.  Then, by definition, returns are a statistic.  As such, it is mathematically improper to assume their distribution into existence.  You must not assume that <span class="math-container">$R_t$</span> is lognormally distributed as it must be derived.</p>
<p>Second, the <span class="math-container">$-1$</span> part of the initial formula is irrelevant to the stochastic one, so <span class="math-container">$R_t$</span> is a product distribution of two ratio distributions.  I do apologize, but I won’t complete the math here as it is too long, but I will cover it enough in general to get a good solution for you.</p>
<p>For simplicity, we will ignore liquidity as Markowitz did, though that is very improper because it will violate the Dutch Book Theorem.  Nonetheless, if we do not care that our formulas are not useable, we will pretend that <span class="math-container">$P=P$</span> instead of <span class="math-container">$P=P(Q)$</span>.  We will also abolish dividends, again, the same problem as above.</p>
<p>Suppose we ignore short selling for this discussion as it is nearly symmetric, “nearly” being a big deal, in reality. In that case, we can focus on <span class="math-container">$q_{t+1},$</span> <span class="math-container">$p_t$</span>, and <span class="math-container">$p_{t+1}$</span> to get to the crux of the matter.  The problem with <span class="math-container">$q_{t+1}$</span>, even if we ignore liquidity costs is that a bankruptcy court may set it to <span class="math-container">$q_{t+1}=0$</span> , another firm may substitute its shares in a merger, or cash may take its place in a cash for stock merger.  So our distribution for the volume ratio will be multinomial even without considering liquidity costs.</p>
<p>As such, the probability of a return will be the weighted sum over the possible future states.  We will consider only the case where the firm remains a going concern at the end of the period.  There can be no mergers, dividends or bankruptcy and we live in a sea of infinite liquidity.</p>
<p>Relaxing those constraints does not alter the problem but does make it very long.</p>
<p>So we can restrict our case to <span class="math-container">$$R_t=\frac{p_{t+1}}{p_t}.$$</span>  The distribution of prices will depend on the auction rules.  For example, in an English style auction, there is a winner’s curse.  The high bidder wins, so the distribution of winning bids should be the Gumbel distribution.  Only winning bids get recorded as the market price.  The others represent transactions that did not happen.</p>
<p>In a NYSE style double auction, buyers bid against buyers and sellers bid against sellers.  In equilibrium, there is no winner’s curse.  The rational behavior is for each bidder to bid their expectation.  As such, the limit book as time goes to infinity should be a distribution of expectations around an equilibrium price.  From the central limit theorem, once scaled, it should be normally distributed.</p>
<p>There are two ways to handle this distribution.  The first and most direct way is simply to solve <span class="math-container">$$R_t=\frac{p_{t+1}}{p_t}.$$</span>  The problem with this is that for most of history, we only have end of day values and intraday trades are not recorded in the order they happen in due to larger orders being taken off the tape and reinserted later.  An alternative exists by converting to polar coordinates around the equilibrium.  In that case, we end up with one less parameter to estimate and a workable solution.</p>
<p>It is well known in the statistical literature that the distribution of two normal distributions has no first or higher moment, so the even moments turn out to be infinite and the odd moments do not exist.  The entire distribution is a weighted sum over the states, but as long as one of the components of the sum has infinite variance, then the entire distribution has infinite variance.</p>
<p>A proof for a simple case exists on the mathematical dictionary at Wolfram research.  The link is <a href="https://mathworld.wolfram.com/NormalRatioDistribution.html" rel="nofollow noreferrer">here</a>.</p>
<p>The distribution must be truncated at 0.  As an empirical note, once you have factored in all the other things and structural breaks, you end up with an excellent model of the real world.</p>
<p>A broader, more general discussion can be found at</p>
<blockquote>
<p>Marsaglia, G. (2006). Ratios of Normal Variables. Journal of Statistical Software, 16(4), 1–10 or Marsaglia G (1965). Ratios of Normal Variables and Ratios of Sums of Uniform Variables. Journal of the American Statistical Association, 60, 193–204.</p>
</blockquote>
<p>Now let us talk about what happens when you map variables without a mean onto variables without a mean.</p>
<p>The method of ordinary least squares does not have any distributional assumptions.  That does not imply that it works as intended for all possible distributions, hence the restriction to finite variance.</p>
<p>Ordinary least squares is a projection and an algorithm.  Indeed, all Frequentist decision rules are algorithms.  If the assumptions are violated, no matter how badly, they will still pop out a number.  The number can be meaningless, but it will exist.  It will minimize the squared loss that would have happened at the time the data was collected had the data been known in advance of collecting it.  The regression line will be the best fit line in terms of minimizing squared loss in the observed sample.</p>
<p>In 1851, the mathematician Augustin Cauchy entered into a battle with the mathematician Irenee-Jules Bienayme.  You probably have never heard of Bienayme, but his work is everywhere in statistics, but is always named for someone else.  Chebychev’s theorem was solved by Bienayme, for example.</p>
<p>What Bienayme basically showed in 1851 was that OLS was BLUE.  As entire fields of mathematics were discovered by Cauchy, he took this as a personal insult as he had just published a median based regression method.  Augustin Cauchy was probably in the top ten mathematicians of all time.</p>
<p>What Cauchy discovered was that if there was infinite variance, then OLS would always fail to produce a useful answer.  In fact, there is no mathematical difference between having two pairs of coordinates and one million pairs of coordinates.  Adding data does not improve the quality of the regression.  Any form of least squares regression has been known since 1851 to produce spurious results with variables that lack a finite second moment.</p>
<p>I have found a contracted and simplified proof of that for the univariate case that I will link at the end.</p>
<p>To understand why this is happening, think about what squared loss is trying to do, it is trying to minimize the variance of an estimator that has infinite population variance.  How precisely do you minimize infinity?</p>
<p>A proof of what is happening is too long to present here but is present in standard statistical texts that are not for applied use.  Some intuition can be provided, however.</p>
<p>A peculiar property of distributions with infinite variance is that the sample variance grows with the sample size.  That makes sense because as the sample size goes to infinity, the observed variance converges to the true variance, which is infinite.  For the least-squares estimator, the denominator goes to infinity as the sample size grows.  A discussion of the numerator is more complex because a covariance cannot exist, but an analogous concept exists.</p>
<p>What ends up happening is that the sampling distribution of the estimators eventually map out to the population distribution of the set of all possible slopes.  That makes inference meaningless and the sample estimates pointless.</p>
<p>What you can do is one of three things.</p>
<p>If you are doing academic work, then you can use either Thiel’s regression or Quantile regression because all distributions have a median.  Thiel’s regression is the better of the two in terms of efficiency.  It is related to bootstrapping methods but is very slow.  Quantile regression can have more problems with outliers.  Although the breakpoint is very high, it is not infinite.</p>
<p>If you are doing applied work, then you must use Bayesian regression.  There does not exist a sufficient point statistic for this type of regression, but the Bayesian likelihood function is always minimally sufficient.  There is simply too large of a qualitative difference between the Bayesian and Frequentist methods in that case.  Plus, Frequentist methods violate the Dutch Book Theorem.  As such, it is possible to force banks to take losses if users use Bayesian methods and banks use Frequentist methods.</p>
<p>A rather bizarre element of mapping distributions lacking second moments onto distributions lacking second moments, the OLS estimator will meet all standard Frequentist criteria used in econometrics for validity.  It will have the bizarre property, however, of having zero precision.  The estimator will be perfectly imprecise at estimating the population quantity as the sample size goes to infinity.</p>
<p>Los Alamos National Labs produced a detailed discussion of this issue at</p>
<blockquote>
<p>Hanson K.M., Wolf D.R. (1996) Estimators for the Cauchy Distribution. In: Heidbreder G.R. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics (An International Book Series on The Fundamental Theories of Physics: Their Clarification, Development and Application), vol 62. Springer, Dordrecht. <a href="https://doi.org/10.1007/978-94-015-8729-7_20" rel="nofollow noreferrer">https://doi.org/10.1007/978-94-015-8729-7_20</a></p>
</blockquote>
<p>This type of problem also appears in particle physics and in the physics of rolling objects.</p>
<p>A univariate case, where there is no regression, is Gull’s Lighthouse problem.  If you imagine a rotation of the regression line and a collapsing of the errors onto a single variable, then the projection of the rotated variables ends up as this univariate case.  The point of projection of the light of a lighthouse has this property.</p>
<p>A slide presentation of it is at Imperial college.  It is mostly 13 pages of equations <a href="https://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/astrophysics/public/icic/data-analysis-workshop/2016/CLT-and-Lighthouse-Problem.pdf" rel="nofollow noreferrer">here</a>.</p>
<p>The original statement of the problem is at</p>
<blockquote>
<p>Gull, Stephen (1988). Bayesian Inductive Inference and Maximum Entropy. Maximum Entropy and Bayesian Methods in Science and Engineering. Vol 1. 53-74.</p>
</blockquote>

CC BY-SA 4.0
2021-12-21T08:38:28.167Z
Thank you for this very detailed discussion. 
My takeaway from it is that OLS will be unbiased but inconsistent if there is an infinite variance regressor.  Is that correct?
Michael Gmeiner
CC BY-SA 4.0
2021-12-21T13:54:36.737Z
*So our distribution for the volume ratio will be multinomial*: multinomial? Also, very interesting historical notes! What is their source (not only in this, but in your other posts, too)?
Richard Hardy
CC BY-SA 4.0
2021-12-23T03:15:19.113Z
@MichaelGmeiner yes.  With an infinite amount of data the closest solution that you will get is $\theta\in\Re$. It was first discovered by Poisson. It rarely gets discussed because Poisson noted, that we will doubtlessly not encounter that case. That translated quote still appears every now and then. I am sure most people do not notice it. He recommends a footnote as it is irrelevant to reality. There are exceptions, but not to OLS. Theil's regression, logit or Bayesian methods do not necessarily have this issue.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:20:56.303Z
@RichardHardy Multinomial *only* if $P=P$ and not if $P=P(Q)$. Even then, you could alter that model by transforming a logistic regression so that you have that $\hat{p}_i$ being modeled, which I have done. It works wonderfully. I have a wonderful bankruptcy model.  I have a so-so merger model. It still has too much sampling variance. It may be nature is the issue, however, and not the model.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:25:13.723Z
@RichardHardy this is not the place for chat, but I will try and give a brief background. I used to manage money and someone noticed that my portfolio was about a six month leading signal for Berkshire Hathaway's purchases. I had some deaths and other things and I decided that I would figure out why Modern Portfolio Theory doesn't work. I am not generally bright enough to really notice how stupid such a quest was. So I read the bibliography section of every article that was relevant and traced them back to articles as far back as Maria Agnesi in the 18th century.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:28:11.817Z
@RichardHardy I ended up with around 900 relevant articles and read them in their entirety. I don't do the abstract/conclusion thing. I read the whole thing. For a long time I sat in the math library with a nest of books around me because I had to learn the fundamental mathematics to understand the discussions mathematicians were having with each other. How was I to know what made a function Greenian and why should I care? Why should I know what a Borel Tribe is? The articles ranged from spectral analysis by ranks in geology to planar Brownian motion.  I read everything.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:30:57.620Z
@RichardHardy I am trying to propose a new class of integrals and derivatives and an options pricing model for seven years now but I am unpublishable. Bayesian methods are generative. You cannot get away with a model like the CAPM, which assumes markets are in equilibrium, because Bayesian methods require you to cover all mutually exclusive and exhaustive sets of states. What happens when the assumption is false? What happens if ships sit off the California coast and knocks the markets out of equilibrium? What if there is a difference between short and long run?
Dave Harris
CC BY-SA 4.0
2021-12-23T03:34:41.320Z
@RichardHardy I am slowly working on getting my Italian citizenship recognized. I am thinking about trying again in Europe. Maybe the cultural difference would help. I don't know. I know that I can replace the rules for applied finance and macroeconomics from the ground up. I can also explain how to split academic econometrics from applied. The Dutch Book Theorem can be violated by academics, it cannot be by banks.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:38:06.527Z
@RichardHardy in 1953, John von Neumann wrote a brief warning on the mean-variance proofs going around at the time. He felt they may end up being proved false because that branch of math had yet to be solved by mathematicians. It was later solved and it doesn't match what economists do. Nobody told the mathematicians we don't do what they do, they didn't realize that we didn't know.  At least that has been my impression from discussions with mathematicians.
Dave Harris
CC BY-SA 4.0
2021-12-23T03:39:36.823Z
@RichardHardy I asked a mathematician about how to explain why the CAPM doesn't work. His response is just to say "it violates the laws of general summation." I told him I cannot tell economists they do not know how to add. His response was, "I don't know how to then, because that is the actual problem."
Dave Harris
CC BY-SA 4.0
2021-12-24T07:50:45.853Z
@DaveHarris, utterly fascinating! Thank you for your detailed answer. I hope I get to meet and talk to you at some point, as there is so much interesting you know and are able to tell in an exciting way. But for now: Merry Christmas!
Richard Hardy
CC BY-SA 4.0
2021-12-24T22:40:02.140Z
@RichardHardy Merry Christmas to you too!
Dave Harris
CC BY-SA 4.0
2022-01-12T13:12:52.077Z
I'm thinking more about this, and I am again not convinced.
It is well known that in time series econometrics if $y_t$ and $x_t$ are cointegrated that OLS is superconsistent, but if $x_t$ is a random walk with drift it has infinite variance. Thus, there is an example when OLS is definitely consistent in the case of an infinite-variance regressor.
Michael Gmeiner
CC BY-SA 4.0
2022-01-14T04:21:32.153Z
@MichaelGmeiner I agree with you on consistency. My language was sloppy. However, that consistency is a bit misleading.  As the sample size goes to infinity, the consistency is due to symmetry.  I am heavily involved in the local COVID response, so I do not have time now, but I will extend the answer.
Dave Harris
CC BY-SA 4.0
2021-12-21T06:15:32.530Z
2021-12-21T06:22:01.890Z
2021-12-21T06:22:01.890Z
Dave Harris
<p>Many presentations of OLS have a condition, <span class="math-container">$E(X'X)&lt;\infty$</span> and is invertible. My question is, why is <span class="math-container">$&lt; \infty$</span> critical?</p>
<p>Consider,</p>
<p><span class="math-container">$$y_i =\beta_0 +\beta_1 x_{1i} +\varepsilon_i $$</span></p>
<p>The condition, <span class="math-container">$E(X'X)&lt;\infty$</span>  in this setting is equivalent to <span class="math-container">$$Var(x_1)&lt;\infty$$</span> and <span class="math-container">$E[x_1]&lt;\infty$</span>.</p>
<p>The OLS estimator of <span class="math-container">$\beta_1$</span> is <span class="math-container">$$\hat{\beta_1}=\frac{Cov(x,y)}{Var(x)} = \frac{Cov(x,\beta_0 +\beta_1 x_{1i} +\varepsilon_i)}{Var(x)} =\beta_1 \frac{Var(x)}{Var(x)} + \frac{Cov(x,u)}{Var(x)}$$</span></p>
<p>Even if <span class="math-container">$Var(x)=\infty$</span>, we have <span class="math-container">$Var(x)/Var(x)=1 $</span>
<span class="math-container">$$\hat{\beta_1}=\beta_1  + \frac{Cov(x,u)}{Var(x)}$$</span></p>
<p>The way I see it, if <span class="math-container">$Cov(x,u)=0$</span>, then <span class="math-container">$0/\infty$</span> is typically defined as 0 and there are no issues.</p>
<p>Thanks in advance!</p>

CC BY-SA 4.0
2021-12-20T15:14:54.967Z
This would be squarely on topic at [Cross Validated](https://stats.stackexchange.com/questions) and a duplicate question might have already been answered there.
Richard Hardy
CC BY-SA 4.0
2021-12-20T15:52:59.483Z
As far as I know $\infty/\infty$ is not well defined and not necessarily equal to 1: e.g. $\lim_{x \to \infty} \frac{x^2}{x}$.
tdm
CC BY-SA 4.0
2021-12-20T16:27:26.480Z
@tdm sure, but at what point does $\infty/\infty$ become relevant in this problem? $0/\infty$ is typically defined as 0 right?
Michael Gmeiner
CC BY-SA 4.0
2021-12-20T16:39:21.283Z
I hope we can agree    $lim_{x\rightarrow \infty} \frac{x}{x} =1$
Michael Gmeiner
CC BY-SA 4.0
2021-12-20T23:35:53.860Z
Perhaps https://stats.stackexchange.com/questions/243452/the-least-squares-assumptions can help.
Walrasian Auctioneer
<regression>
Infinite Variance Regressors
Michael Gmeiner
# Infinite Variance Regressors

(Asked by: username_0 on 2021-12-20)

Many presentations of OLS have a condition, $E(X'X)<\infty$ and is invertible. My question is, why is $< \infty$ critical?

Consider,

$$y\_i =\beta\_0 +\beta\_1 x\_{1i} +\varepsilon\_i $$

The condition, $E(X'X)<\infty$ in this setting is equivalent to $$Var(x\_1)<\infty$$ and $E[x\_1]<\infty$.

The OLS estimator of $\beta\_1$ is $$\hat{\beta\_1}=\frac{Cov(x,y)}{Var(x)} = \frac{Cov(x,\beta\_0 +\beta\_1 x\_{1i} +\varepsilon\_i)}{Var(x)} =\beta\_1 \frac{Var(x)}{Var(x)} + \frac{Cov(x,u)}{Var(x)}$$

Even if $Var(x)=\infty$, we have $Var(x)/Var(x)=1 $
$$\hat{\beta\_1}=\beta\_1 + \frac{Cov(x,u)}{Var(x)}$$

The way I see it, if $Cov(x,u)=0$, then $0/\infty$ is typically defined as 0 and there are no issues.

Thanks in advance!

**Tags:** regression
> **Comments:**
> **username_1**: This would be squarely on topic at [Cross Validated](https://stats.stackexchange.com/questions) and a duplicate question might have already been answered there.  
> **username_2**: As far as I know $\infty/\infty$ is not well defined and not necessarily equal to 1: e.g. $\lim_{x \to \infty} \frac{x^2}{x}$.  
> **username_0**: @tdm sure, but at what point does $\infty/\infty$ become relevant in this problem? $0/\infty$ is typically defined as 0 right?  
> **username_0**: I hope we can agree    $lim_{x\rightarrow \infty} \frac{x}{x} =1$  
> **username_3**: Perhaps https://stats.stackexchange.com/questions/243452/the-least-squares-assumptions can help.  

---

## 2 answers

### Answer by username_0 on 2022-01-30. Score: 1 | Accepted answer

There is no problem with infinite variance regressors. The assumption is made because it results in consistency at the rate of the square root of the sample size. This is standard and common, thus it is presented in this way in most textbooks.

In the case of an infinite variance regressor, the OLS estimate converges to the truth at a rate faster than $\sqrt{N}$, and is "superconsistent".

Consider the model: $y\_i =\beta\_0 +\beta\_1 x\_{1i}+u\_i$.

The OLS estimator of $\beta\_1$ is $$\hat{\beta\_1} =\beta\_1 \frac{Var(x)}{Var(x)} + \frac{Cov(x,u)}{Var(x)}$$

Consider the case of a mean-zero regressor for convenience: $$\hat{\beta\_1} =\beta\_1 +\frac{\frac{1}{N}\sum\_{i=1}^Nx\_iu\_i}{\frac{1}{N}\sum\_{i=1}^Nx\_i^2} $$

$$\sqrt{N}(\hat{\beta\_1} -\beta\_1)=\frac{\frac{1}{\sqrt{N}}\sum\_{i=1}^Nx\_iu\_i}{\frac{1}{N}\sum\_{i=1}^Nx\_i^2} $$

If a CLT applies to the numerator and the LLN applies to the denominator (i.e., the regressor has finite variance), then the OLS estimate converges at the rate of $\sqrt{N}$. If there is an infinite variance regressor, then the denominator in the above expression does not converge. Suppose however, that $\frac{1}{N^2}\sum\_{i=1}^Nx\_i^2$ converges to a non-zero constant, then,

$$N^{3/2}(\hat{\beta\_1} -\beta\_1)=\frac{\frac{1}{\sqrt{N}}\sum\_{i=1}^Nx\_iu\_i}{\frac{1}{N^2}\sum\_{i=1}^Nx\_i^2} $$

Again, a CLT applies to the numerator, and the denominator converges. Thus, the OLS estimate converges at the rate $N^{3/2}$. This is "superconsistency" and a case that most basic textbooks rule out for ease of exposition.

> **Comments:**
> **username_4**: Do you have any textbooks or extended reading on this topic? I note that this answer has much less divergence - I am not interested in finance per se so it is quite clear.  
> **username_0**: Hamilton's Time Series Analysis, section “Asymptotic Properties of a First-Order Autoregression when the True Coefficient is Unity” (17.4).  
> **username_0**: This is most prominent in time series econometrics in which regressors have unit roots or are otherwise non-stationary, giving them infinite variance.  
> **username_4**: Ah, I should have known better than to think it would be anywhere else!  

---

### Answer by username_5 on 2021-12-21. Score: 4

So let us begin with a real case where the variance has to be infinite, the stock market.

I am not doing a complete derivation here as it would run as multiple chapters. I know this because I am doing web presentations on this. But I can do enough to show you that there cannot be a first moment for the stock market. From that, I will explain the consequence for regressors.

Let us begin with a couple of simple observations about a return or a reward for investing.

If we define a return on a single pair of cash flows as $$r\_t=\frac{p\_{t+1}}{p\_t}\times\frac{q\_{t+1}}{q\_t}-1$$ and $$R\_t=r\_t+1$$ we can note a few things.

First, $$R\_t=R(p\_t,p\_{t+1},q\_t,q\_{t+1}).$$ By definition, prices are data. Volumes are data. Returns are a function of data. Then, by definition, returns are a statistic. As such, it is mathematically improper to assume their distribution into existence. You must not assume that $R\_t$ is lognormally distributed as it must be derived.

Second, the $-1$ part of the initial formula is irrelevant to the stochastic one, so $R\_t$ is a product distribution of two ratio distributions. I do apologize, but I won’t complete the math here as it is too long, but I will cover it enough in general to get a good solution for you.

For simplicity, we will ignore liquidity as Markowitz did, though that is very improper because it will violate the Dutch Book Theorem. Nonetheless, if we do not care that our formulas are not useable, we will pretend that $P=P$ instead of $P=P(Q)$. We will also abolish dividends, again, the same problem as above.

Suppose we ignore short selling for this discussion as it is nearly symmetric, “nearly” being a big deal, in reality. In that case, we can focus on $q\_{t+1},$ $p\_t$, and $p\_{t+1}$ to get to the crux of the matter. The problem with $q\_{t+1}$, even if we ignore liquidity costs is that a bankruptcy court may set it to $q\_{t+1}=0$ , another firm may substitute its shares in a merger, or cash may take its place in a cash for stock merger. So our distribution for the volume ratio will be multinomial even without considering liquidity costs.

As such, the probability of a return will be the weighted sum over the possible future states. We will consider only the case where the firm remains a going concern at the end of the period. There can be no mergers, dividends or bankruptcy and we live in a sea of infinite liquidity.

Relaxing those constraints does not alter the problem but does make it very long.

So we can restrict our case to $$R\_t=\frac{p\_{t+1}}{p\_t}.$$ The distribution of prices will depend on the auction rules. For example, in an English style auction, there is a winner’s curse. The high bidder wins, so the distribution of winning bids should be the Gumbel distribution. Only winning bids get recorded as the market price. The others represent transactions that did not happen.

In a NYSE style double auction, buyers bid against buyers and sellers bid against sellers. In equilibrium, there is no winner’s curse. The rational behavior is for each bidder to bid their expectation. As such, the limit book as time goes to infinity should be a distribution of expectations around an equilibrium price. From the central limit theorem, once scaled, it should be normally distributed.

There are two ways to handle this distribution. The first and most direct way is simply to solve $$R\_t=\frac{p\_{t+1}}{p\_t}.$$ The problem with this is that for most of history, we only have end of day values and intraday trades are not recorded in the order they happen in due to larger orders being taken off the tape and reinserted later. An alternative exists by converting to polar coordinates around the equilibrium. In that case, we end up with one less parameter to estimate and a workable solution.

It is well known in the statistical literature that the distribution of two normal distributions has no first or higher moment, so the even moments turn out to be infinite and the odd moments do not exist. The entire distribution is a weighted sum over the states, but as long as one of the components of the sum has infinite variance, then the entire distribution has infinite variance.

A proof for a simple case exists on the mathematical dictionary at Wolfram research. The link is [here](https://mathworld.wolfram.com/NormalRatioDistribution.html).

The distribution must be truncated at 0. As an empirical note, once you have factored in all the other things and structural breaks, you end up with an excellent model of the real world.

A broader, more general discussion can be found at

> Marsaglia, G. (2006). Ratios of Normal Variables. Journal of Statistical Software, 16(4), 1–10 or Marsaglia G (1965). Ratios of Normal Variables and Ratios of Sums of Uniform Variables. Journal of the American Statistical Association, 60, 193–204.

Now let us talk about what happens when you map variables without a mean onto variables without a mean.

The method of ordinary least squares does not have any distributional assumptions. That does not imply that it works as intended for all possible distributions, hence the restriction to finite variance.

Ordinary least squares is a projection and an algorithm. Indeed, all Frequentist decision rules are algorithms. If the assumptions are violated, no matter how badly, they will still pop out a number. The number can be meaningless, but it will exist. It will minimize the squared loss that would have happened at the time the data was collected had the data been known in advance of collecting it. The regression line will be the best fit line in terms of minimizing squared loss in the observed sample.

In 1851, the mathematician Augustin Cauchy entered into a battle with the mathematician Irenee-Jules Bienayme. You probably have never heard of Bienayme, but his work is everywhere in statistics, but is always named for someone else. Chebychev’s theorem was solved by Bienayme, for example.

What Bienayme basically showed in 1851 was that OLS was BLUE. As entire fields of mathematics were discovered by Cauchy, he took this as a personal insult as he had just published a median based regression method. Augustin Cauchy was probably in the top ten mathematicians of all time.

What Cauchy discovered was that if there was infinite variance, then OLS would always fail to produce a useful answer. In fact, there is no mathematical difference between having two pairs of coordinates and one million pairs of coordinates. Adding data does not improve the quality of the regression. Any form of least squares regression has been known since 1851 to produce spurious results with variables that lack a finite second moment.

I have found a contracted and simplified proof of that for the univariate case that I will link at the end.

To understand why this is happening, think about what squared loss is trying to do, it is trying to minimize the variance of an estimator that has infinite population variance. How precisely do you minimize infinity?

A proof of what is happening is too long to present here but is present in standard statistical texts that are not for applied use. Some intuition can be provided, however.

A peculiar property of distributions with infinite variance is that the sample variance grows with the sample size. That makes sense because as the sample size goes to infinity, the observed variance converges to the true variance, which is infinite. For the least-squares estimator, the denominator goes to infinity as the sample size grows. A discussion of the numerator is more complex because a covariance cannot exist, but an analogous concept exists.

What ends up happening is that the sampling distribution of the estimators eventually map out to the population distribution of the set of all possible slopes. That makes inference meaningless and the sample estimates pointless.

What you can do is one of three things.

If you are doing academic work, then you can use either Thiel’s regression or Quantile regression because all distributions have a median. Thiel’s regression is the better of the two in terms of efficiency. It is related to bootstrapping methods but is very slow. Quantile regression can have more problems with outliers. Although the breakpoint is very high, it is not infinite.

If you are doing applied work, then you must use Bayesian regression. There does not exist a sufficient point statistic for this type of regression, but the Bayesian likelihood function is always minimally sufficient. There is simply too large of a qualitative difference between the Bayesian and Frequentist methods in that case. Plus, Frequentist methods violate the Dutch Book Theorem. As such, it is possible to force banks to take losses if users use Bayesian methods and banks use Frequentist methods.

A rather bizarre element of mapping distributions lacking second moments onto distributions lacking second moments, the OLS estimator will meet all standard Frequentist criteria used in econometrics for validity. It will have the bizarre property, however, of having zero precision. The estimator will be perfectly imprecise at estimating the population quantity as the sample size goes to infinity.

Los Alamos National Labs produced a detailed discussion of this issue at

> Hanson K.M., Wolf D.R. (1996) Estimators for the Cauchy Distribution. In: Heidbreder G.R. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics (An International Book Series on The Fundamental Theories of Physics: Their Clarification, Development and Application), vol 62. Springer, Dordrecht. <https://doi.org/10.1007/978-94-015-8729-7_20>

This type of problem also appears in particle physics and in the physics of rolling objects.

A univariate case, where there is no regression, is Gull’s Lighthouse problem. If you imagine a rotation of the regression line and a collapsing of the errors onto a single variable, then the projection of the rotated variables ends up as this univariate case. The point of projection of the light of a lighthouse has this property.

A slide presentation of it is at Imperial college. It is mostly 13 pages of equations [here](https://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/astrophysics/public/icic/data-analysis-workshop/2016/CLT-and-Lighthouse-Problem.pdf).

The original statement of the problem is at

> Gull, Stephen (1988). Bayesian Inductive Inference and Maximum Entropy. Maximum Entropy and Bayesian Methods in Science and Engineering. Vol 1. 53-74.

> **Comments:**
> **username_0**: Thank you for this very detailed discussion.  My takeaway from it is that OLS will be unbiased but inconsistent if there is an infinite variance regressor.  Is that correct?  
> **username_1**: *So our distribution for the volume ratio will be multinomial*: multinomial? Also, very interesting historical notes! What is their source (not only in this, but in your other posts, too)?  
> **username_5**: @MichaelGmeiner yes.  With an infinite amount of data the closest solution that you will get is $\theta\in\Re$. It was first discovered by Poisson. It rarely gets discussed because Poisson noted, that we will doubtlessly not encounter that case. That translated quote still appears every now and then. I am sure most people do not notice it. He recommends a footnote as it is irrelevant to reality. There are exceptions, but not to OLS. Theil's regression, logit or Bayesian methods do not necessarily have this issue.  
> **username_5**: @RichardHardy Multinomial *only* if $P=P$ and not if $P=P(Q)$. Even then, you could alter that model by transforming a logistic regression so that you have that $\hat{p}_i$ being modeled, which I have done. It works wonderfully. I have a wonderful bankruptcy model.  I have a so-so merger model. It still has too much sampling variance. It may be nature is the issue, however, and not the model.  
> **username_5**: @RichardHardy this is not the place for chat, but I will try and give a brief background. I used to manage money and someone noticed that my portfolio was about a six month leading signal for Berkshire Hathaway's purchases. I had some deaths and other things and I decided that I would figure out why Modern Portfolio Theory doesn't work. I am not generally bright enough to really notice how stupid such a quest was. So I read the bibliography section of every article that was relevant and traced them back to articles as far back as Maria Agnesi in the 18th century.  
> **username_5**: @RichardHardy I ended up with around 900 relevant articles and read them in their entirety. I don't do the abstract/conclusion thing. I read the whole thing. For a long time I sat in the math library with a nest of books around me because I had to learn the fundamental mathematics to understand the discussions mathematicians were having with each other. How was I to know what made a function Greenian and why should I care? Why should I know what a Borel Tribe is? The articles ranged from spectral analysis by ranks in geology to planar Brownian motion.  I read everything.  
> **username_5**: @RichardHardy I am trying to propose a new class of integrals and derivatives and an options pricing model for seven years now but I am unpublishable. Bayesian methods are generative. You cannot get away with a model like the CAPM, which assumes markets are in equilibrium, because Bayesian methods require you to cover all mutually exclusive and exhaustive sets of states. What happens when the assumption is false? What happens if ships sit off the California coast and knocks the markets out of equilibrium? What if there is a difference between short and long run?  
> **username_5**: @RichardHardy I am slowly working on getting my Italian citizenship recognized. I am thinking about trying again in Europe. Maybe the cultural difference would help. I don't know. I know that I can replace the rules for applied finance and macroeconomics from the ground up. I can also explain how to split academic econometrics from applied. The Dutch Book Theorem can be violated by academics, it cannot be by banks.  
> **username_5**: @RichardHardy in 1953, John von Neumann wrote a brief warning on the mean-variance proofs going around at the time. He felt they may end up being proved false because that branch of math had yet to be solved by mathematicians. It was later solved and it doesn't match what economists do. Nobody told the mathematicians we don't do what they do, they didn't realize that we didn't know.  At least that has been my impression from discussions with mathematicians.  
> **username_5**: @RichardHardy I asked a mathematician about how to explain why the CAPM doesn't work. His response is just to say "it violates the laws of general summation." I told him I cannot tell economists they do not know how to add. His response was, "I don't know how to then, because that is the actual problem."  
> **username_1**: @DaveHarris, utterly fascinating! Thank you for your detailed answer. I hope I get to meet and talk to you at some point, as there is so much interesting you know and are able to tell in an exciting way. But for now: Merry Christmas!  
> **username_5**: @RichardHardy Merry Christmas to you too!  
> **username_0**: I'm thinking more about this, and I am again not convinced. It is well known that in time series econometrics if $y_t$ and $x_t$ are cointegrated that OLS is superconsistent, but if $x_t$ is a random walk with drift it has infinite variance. Thus, there is an example when OLS is definitely consistent in the case of an infinite-variance regressor.  
> **username_5**: @MichaelGmeiner I agree with you on consistency. My language was sloppy. However, that consistency is a bit misleading.  As the sample size goes to infinity, the consistency is due to symmetry.  I am heavily involved in the local COVID response, so I do not have time now, but I will extend the answer.  

---

