This project has built the mathematical foundations for designing the first demonstrably correct statistical tests for testing independence for pairs of paths of Gaussian processes: Wiener processes, Ornstein-Uhlenbeck (OU) processes, fractional Ornstein-Uhlenbeck (fOU), and fractional Brownian motion (fBm). The importance of constructing such tests is motivated by our 2017 paper on Yule's so called "nonsense" correlation, in which we provide the precise scale of the apparent correlation between two independent continuous series of data, such as what one encounters in economics, climate science, finance, and many other fields.

We embarked on this proposal by developing theory and methodology for calculating all moments (up to order 16) of Yule’s “nonsense” correlation for two independent Wiener processes. This allows us to provide the first density approximation to Yule's “nonsense” correlation. The methodology we develop is broad in spirit, allowing us to work conclusively in all settings where the Gaussian process arises as the solution of a linear stochastic differential equation (SDE). We then employ these methods to explicitly calculate the moments of the empirical correlation for two correlated Brownian motions (with correlation coefficient), two independent Ornstein-Uhlenbeck processes, and two independent Brownian bridges. Establishing unequivocal mathematical facts which are simple to explain, such as providing calculations of second moments of empirical correlation for the processes considered above, should bear enormous potential in  popularizing the risks associated with widespread misinterpretation of Pearson correlation coefficients.

We then explore the following question: what is the distribution of the empirical correlation for two independent Gaussian random walks? This question is of interest not least because discrete stochastic process data (for example, time series data) occur most frequently and extensively in the real world. A test statistic for discrete processes is thus easier for practitioners to apply than that for continuous stochastic processes. Studying the discrete data test statistic directly is also a means of minimizing the risk of using the continuous statistic abusively when the discrete-data situation is not sufficiently well approximated by a continuous-data one. In this vein, we succeed in providing an exact formula for the second moment of the empirical correlation of two independent Gaussian random walks (as well as implicit formulas for higher moments). We also provide rates of convergence of the empirical correlation of two independent Gaussian random walks to the empirical correlation of two independent Wiener processes, and explicit upper bounds (in terms of the Wasserstein distance) are given.

We proceed to work in statistical inference for discrete-time second chaos processes, as well as for Gaussian (discrete-time) processes. We compute the quadratic variations of all AR(1) stationary time series in the second chaos, and estimate their normal speeds of convergence in total variation. In addition to working with discrete-time second chaos processes, we consider basic objects in the fourth and second Wiener chaos which are directly relevant to their statistical properties, and moreover, these objects are statistics of the processes' entire paths. Understanding how these objects behave asymptotically will be a necessary first step in achieving quantitative estimates for asymptotic normality of the Pearson correlation for stationary processes like the Ornstein-Uhlenbeck process, as well as determining when the graininess of the time scale affects the correlation's distribution, and when it does not. In fact, the second-chaos AR(1) processes will all converge, under proper scaling, to the Gaussian Ornstein-Uhlenbeck process, but we are interested in mesoscopic scales where the fluctuations' normality is too distant to be relied upon.

The relevant notions of tests of independence of pairs of paths of stochastic processes we have considered, and which we will continue to consider, are manifold: from short-range to long-range correlation for individual paths, to whether single pairs or finite sets of paths are statistically related, and to applied consequences when considering questions of attribution of factors for real-world phenomena, particularly relating to weather and climate. Important applied aims include an investigation of climate-related risks, such as sea-level rise and extreme weather events, particularly in the North Atlantic Ocean, how they correlate dynamically over medium and long terms, and how heavy-tailed they are. We would be remiss if we did not highlight the recent misuse of Pearson correlation in the area of late-Holocene paleoclimatology, an area whose importance for projecting our planet's climate in the next 200 or 1000 years cannot be overstated.

Finally, our proposal has also significantly contributed to both graduate and undergraduate training. The support of the NSF has enabled us to train four Ph.D. students, all of which are now active contributors to the STEM research community. We have also engaged with undergraduate and masters students on aspects of this proposal relevant to climate science, to agricultural economics, and to development economics in least-developed countries.

Last Modified: 12/14/2022
Modified by: Philip Ernst