Does the price of butter predict the S&P 500? The reason you need cointegration.

Two unrelated series can appear highly correlated, even when there is no real relationship between them.

Computing the correlation between butter prices and the stock market may give you the impression that they are related. They are not. Two unrelated series that trend upward can still appear highly correlated.

This post builds two series with no relation at all, shows how correlation and a naive regression report a strong relationship, and then uses cointegration to show there is none. Cointegration is another way of saying that two time series may wander, but not away from each other forever.

Butter and the S&P 500

Both series are simulated. We build a “butter price” and an “S&P 500”: each drifts upward with random month-to-month moves, drawn independently so they share nothing. Simulating is the point, because it lets us know the truth up front: the two are unrelated, so any relationship the tests report is spurious.

%matplotlib inline
import numpy as np, pandas as pd, matplotlib.pyplot as plt
SEED, N = 6, 600                                          # 600 months, one fixed draw so the post reproduces
rng = np.random.default_rng(SEED)
butter = 50  + np.cumsum(0.10 + rng.normal(0, 1.0, N))    # butter price: small upward drift, its own noise
spx    = 100 + np.cumsum(0.30 + rng.normal(0, 2.0, N))    # S&P 500: upward drift, noise unrelated to butter
print(f"Correlation of the two levels: {np.corrcoef(butter, spx)[0, 1]:.2f}")

fig, ax1 = plt.subplots(figsize=(9, 5))
ax1.plot(butter, color="#E1A100", lw=1.5); ax1.set_ylabel("butter price", color="#E1A100"); ax1.set_xlabel("month")
ax2 = ax1.twinx()
ax2.plot(spx, color="#1F77B4", lw=1.5); ax2.set_ylabel("S&P 500", color="#1F77B4")
plt.title("Butter and the S&P 500, both drifting up (simulated, unrelated)")
fig.tight_layout(); plt.savefig("nr44_butter_spx.png", dpi=120, bbox_inches="tight"); plt.show()

Correlation of the two levels: 0.93

Correlation 0.93. On a chart the two climb together like cause and effect. Stop here and you would call butter a leading indicator for the S&P.

The naive regression agrees

A regression makes the claim even stronger: put butter on the right, the S&P on the left, and read how much of the S&P butter explains.

import statsmodels.api as sm
ols = sm.OLS(spx, sm.add_constant(butter)).fit()          # the naive move: regress the S&P on butter
print(f"R-squared:    {ols.rsquared:.2f}    (butter 'explains' {ols.rsquared:.0%} of the S&P)")
print(f"t-statistic:  {ols.tvalues[1]:.0f}     (past about 2 counts as significant)")
print(f"p-value:      {ols.pvalues[1]:.0e}   (this says certain)")

R-squared:    0.87    (butter 'explains' 87% of the S&P)
t-statistic:  63     (past about 2 counts as significant)
p-value:      9e-267   (this says certain)

R-squared 0.87, t-statistic 63, a p-value at zero. By the textbook, butter explains the S&P beyond doubt, and you might be tempted to trade on it. Do not: the result is spurious.

The t-statistic is only meaningful when the regression error behaves like noise around a stable average. Here it does not. The error still has a trend, so the regression is comparing two drifting series rather than measuring a stable relationship. Granger and Newbold named this the spurious regression in 1974.

What cointegration requires

The regression misled us because the series are non-stationary. Cointegration is built for that case, and it asks three things: both series non-stationary, both integrated of the same order, and a long enough sample to measure.

A stationary series has a constant mean and variance. It moves around a fixed level, and the size of those moves does not grow over time. A non-stationary series has a mean or variance that shifts over time. A price level is a good example. It can drift upward for long periods, so its average level today can be far above where it stood years ago. That does not make every rising price series economically related to every other rising price series.

A price is non-stationary, but its returns, the change from one period to the next, move around a stable mean and are stationary. A series that is non-stationary in levels and stationary after one difference is integrated of order one, written I(1), which is what we check next.

from statsmodels.tsa.stattools import adfuller
def adf(series):                                          # p-value of the Augmented Dickey-Fuller unit-root test
    return adfuller(series, autolag="AIC")[1]             # low (<0.05) = stationary; high = a unit root
print("                  levels    first difference")
print(f"butter  ADF p:    {adf(butter):.2f}      {adf(np.diff(butter)):.0e}")
print(f"S&P 500 ADF p:    {adf(spx):.2f}      {adf(np.diff(spx)):.0e}")
print("\nHigh in levels, about zero in differences, so both series are I(1).")

                  levels    first difference
butter  ADF p:    0.57      1e-17
S&P 500 ADF p:    0.94      0e+00

High in levels, about zero in differences, so both series are I(1).

Both series are I(1). In levels they are non-stationary, and the test says so: p-values of 0.57 and 0.94, too high to call either one stationary. Their month-to-month changes are stationary, with p-values near zero. That is the setup cointegration needs, so the test comes next.

The cointegration test

Now the test itself. Cointegration asks whether a fixed combination of the two series, one minus a multiple of the other, is stationary even though each series alone is not. Call that combination the spread. A stationary spread tends to stay near its average. When the two series move apart, the gap has historically closed again. A spread that drifts off and does not come back means there is no stable long-run relationship.

We use the Engle-Granger test: regress the S&P on butter, then test whether the spread is stationary. The null is that the spread is non-stationary, suggesting no cointegration. A low p-value (below 0.05) rejects it: the spread looks stationary, which is evidence of cointegration.

from statsmodels.tsa.stattools import coint
p_coint = coint(spx, butter)[1]                           # Engle-Granger: is the spread (the leftover) stationary?
print(f"Cointegration p-value: {p_coint:.2f}    (low = stationary spread; high = none)")

Cointegration p-value: 0.45    (low = stationary spread; high = none)

The cointegration test returns p = 0.45, far too high to reject the null. So we do not find evidence of a stationary spread. The two are not cointegrated, and butter tells you nothing about the S&P.

What it means

Correlation is a fine measure when both series are stationary. The concern is when the two series trend together. So a stationarity test is the right first step. If both series are stationary, correlation is at least measuring co-movement rather than a shared trend. If they are not, as prices often are, you need a test that accounts for that.

That test is cointegration, and butter and the S&P stand in for any two series you might compare. When two series are cointegrated, the spread tends to stay around a stable mean instead of drifting away indefinitely. That is what pairs trading uses: when the spread opens up, short the one that has risen and buy the one that has fallen, then close both when they come back together. The same idea can be used for longer-run relationships across markets, but the economic story still has to come first. The test is not a machine for finding meaning in random price charts.

Do not blindly trust correlation in price levels. First check stationarity. If the series are non-stationary, check whether they are cointegrated.

Disclaimer: This post is a teaching example. Not investment advice.