Yule's "Nonsense Correlation" Solved!

• Published in 2016
• Added on 2021-08-29

In this paper, we resolve a longstanding open statistical problem. The problem is to mathematically confirm Yule's 1926 empirical finding of "nonsense correlation" (\cite{Yule}). We do so by analytically determining the second moment of the empirical correlation coefficient \beqn \theta := \frac{\int_0^1W_1(t)W_2(t) dt - \int_0^1W_1(t) dt \int_0^1 W_2(t) dt}{\sqrt{\int_0^1 W^2_1(t) dt - \parens{\int_0^1W_1(t) dt}^2} \sqrt{\int_0^1 W^2_2(t) dt - \parens{\int_0^1W_2(t) dt}^2}}, \eeqn of two {\em independent} Wiener processes, $W_1,W_2$. Using tools from Fred- holm integral equation theory, we successfully calculate the second moment of $\theta$ to obtain a value for the standard deviation of $\theta$ of nearly .5. The "nonsense" correlation, which we call "volatile" correlation, is volatile in the sense that its distribution is heavily dispersed and is frequently large in absolute value. It is induced because each Wiener process is "self-correlated" in time. This is because a Wiener process is an integral of pure noise and thus its values at different time points are correlated. In addition to providing an explicit formula for the second moment of $\theta$, we offer implicit formulas for higher moments of $\theta$.

Links

• http://arxiv.org/abs/1608.04120v2
• http://arxiv.org/pdf/1608.04120v2

Other information

BibTeX entry

@article{YulesNonsenseCorrelationSolved,
	key = {YulesNonsenseCorrelationSolved},
	type = {article},
	title = {Yule's "Nonsense Correlation" Solved!},
	author = {Philip Ernst and Larry Shepp and Abraham Wyner},
	abstract = {In this paper, we resolve a longstanding open statistical problem. The
problem is to mathematically confirm Yule's 1926 empirical finding of "nonsense
correlation" (\cite{\{}Yule{\}}). We do so by analytically determining the second
moment of the empirical correlation coefficient
  \beqn \theta := \frac{\{}\int{\_}0^1W{\_}1(t)W{\_}2(t) dt - \int{\_}0^1W{\_}1(t) dt \int{\_}0^1
W{\_}2(t) dt{\}}{\{}\sqrt{\{}\int{\_}0^1 W^2{\_}1(t) dt - \parens{\{}\int{\_}0^1W{\_}1(t) dt{\}}^2{\}}
\sqrt{\{}\int{\_}0^1 W^2{\_}2(t) dt - \parens{\{}\int{\_}0^1W{\_}2(t) dt{\}}^2{\}}{\}}, \eeqn of two {\{}\em
independent{\}} Wiener processes, {\$}W{\_}1,W{\_}2{\$}. Using tools from Fred- holm integral
equation theory, we successfully calculate the second moment of {\$}\theta{\$} to
obtain a value for the standard deviation of {\$}\theta{\$} of nearly .5. The
"nonsense" correlation, which we call "volatile" correlation, is volatile in
the sense that its distribution is heavily dispersed and is frequently large in
absolute value. It is induced because each Wiener process is "self-correlated"
in time. This is because a Wiener process is an integral of pure noise and thus
its values at different time points are correlated. In addition to providing an
explicit formula for the second moment of {\$}\theta{\$}, we offer implicit formulas
for higher moments of {\$}\theta{\$}.},
	comment = {},
	date_added = {2021-08-29},
	date_published = {2016-04-10},
	urls = {http://arxiv.org/abs/1608.04120v2,http://arxiv.org/pdf/1608.04120v2},
	collections = {drama,probability-and-statistics},
	url = {http://arxiv.org/abs/1608.04120v2 http://arxiv.org/pdf/1608.04120v2},
	year = 2016,
	urldate = {2021-08-29},
	archivePrefix = {arXiv},
	eprint = {1608.04120},
	primaryClass = {math.ST}
}