A sweep runs the same backtest once for every value in a list and puts the results in one table. The list here is the SMA window, and each run gives back a total return, a Sharpe ratio and a turnover.
In this lesson I sweep five windows on AAPL, put buy and hold in the same table, then run the sweep on a second ticker and on each half of the sample.
prices.csv sits next to this lesson and holds simulated data: six years of daily closes for four tickers. close_of pulls one ticker out as a Series indexed by date.
import numpy as np
import pandas as pd
prices = pd.read_csv("prices.csv", parse_dates=["Date"])
def close_of(ticker):
rows = prices[prices["Ticker"] == ticker]
return rows.set_index("Date")["Close"].sort_index()
aapl = close_of("AAPL")
print(len(aapl)) # -> 1566
print(aapl.head(2))
# -> Date
# -> 2020-01-01 75.08
# -> 2020-01-02 76.94
# -> Name: Close, dtype: float641566
Date
2020-01-01 75.08
2020-01-02 76.94
Name: Close, dtype: float64Here is the function from Lesson 34 again. It takes a price Series and a window, buys when the close is above its own moving average, lags the signal by a day, and returns three numbers.
def backtest(close, window, cost=0.0):
ret = close.pct_change()
sma = close.rolling(window).mean()
signal = (close > sma).astype(float)
position = signal.shift(1)
trades = position.diff().abs()
strat = (position * ret - cost * trades).dropna()
return {
"total": (1 + strat).prod() - 1,
"sharpe": strat.mean() / strat.std() * np.sqrt(252),
"turnover": trades.sum(),
}
one = backtest(aapl, 50)
print(f"total {one['total']:.2%} sharpe {one['sharpe']:.2f} turnover {one['turnover']:.0f}")
# -> total 58.61% sharpe 0.50 turnover 113total 58.61% sharpe 0.50 turnover 113I leave cost at zero for the whole lesson so the only thing changing between runs is the window.
The sweep is a loop over the list. Each pass calls backtest, tags the result with the window it came from, and appends the dictionary to rows. pd.DataFrame turns a list of dictionaries into a table with one row per dictionary.
windows = [20, 50, 100, 150, 200]
rows = []
for w in windows:
result = backtest(aapl, w)
result["rule"] = f"sma {w}"
rows.append(result)
sweep = pd.DataFrame(rows).set_index("rule")
print(sweep.round(4))
# -> total sharpe turnover
# -> rule
# -> sma 20 1.0932 0.7576 173.0
# -> sma 50 0.5861 0.5040 113.0
# -> sma 100 0.3079 0.3312 95.0
# -> sma 150 0.3359 0.3530 63.0
# -> sma 200 0.1826 0.2408 69.0 total sharpe turnover
rule
sma 20 1.0932 0.7576 173.0
sma 50 0.5861 0.5040 113.0
sma 100 0.3079 0.3312 95.0
sma 150 0.3359 0.3530 63.0
sma 200 0.1826 0.2408 69.0total is a fraction, so 1.0932 is a gain of 109.32%. The short window returns most and trades most: 173 switches in and out against 69 for the 200 day window.
A sweep with nothing to compare against only ranks the tries against each other. Buy and hold is a position of 1.0 every day, so its total is the compounded daily return and its turnover is zero.
ret = aapl.pct_change().dropna()
sweep.loc["buy and hold"] = {
"total": (1 + ret).prod() - 1,
"sharpe": ret.mean() / ret.std() * np.sqrt(252),
"turnover": 0.0,
}
print(sweep.sort_values("total", ascending=False).round(4))
# -> total sharpe turnover
# -> rule
# -> buy and hold 1.6545 0.7865 0.0
# -> sma 20 1.0932 0.7576 173.0
# -> sma 50 0.5861 0.5040 113.0
# -> sma 150 0.3359 0.3530 63.0
# -> sma 100 0.3079 0.3312 95.0
# -> sma 200 0.1826 0.2408 69.0 total sharpe turnover
rule
buy and hold 1.6545 0.7865 0.0
sma 20 1.0932 0.7576 173.0
sma 50 0.5861 0.5040 113.0
sma 150 0.3359 0.3530 63.0
sma 100 0.3079 0.3312 95.0
sma 200 0.1826 0.2408 69.0All five windows return less than buy and hold on this sample, and all five have a lower Sharpe. The best of the five, 20 days, earns 109.32% against 165.45% for holding the stock.
Now the same loop over both AAPL and NVDA, with a 10 day window added to the list. I keep only the total so the two columns sit side by side.
nvda = close_of("NVDA")
wide = [10, 20, 50, 100, 200]
both = pd.DataFrame(index=wide, columns=["AAPL", "NVDA"], dtype=float)
for w in wide:
both.loc[w, "AAPL"] = backtest(aapl, w)["total"]
both.loc[w, "NVDA"] = backtest(nvda, w)["total"]
both.index.name = "window"
print(both.round(4))
# -> AAPL NVDA
# -> window
# -> 10 0.9631 4.0776
# -> 20 1.0932 2.7945
# -> 50 0.5861 1.0602
# -> 100 0.3079 0.9117
# -> 200 0.1826 0.8039 AAPL NVDA
window
10 0.9631 4.0776
20 1.0932 2.7945
50 0.5861 1.0602
100 0.3079 0.9117
200 0.1826 0.8039idxmax() returns the index label of the largest value in each column.
The winner on AAPL is 20 days and the winner on NVDA is 10 days. On AAPL the 10 day window comes second, and on NVDA it beats the 20 day window by 128 percentage points.
aapl.index[len(aapl) // 2] is the date halfway down the sample. Slicing the price Series at that date gives two stretches of roughly equal length, and I sweep each one on its own.
2023-01-02 784 783halves = pd.DataFrame(index=windows, columns=["first", "second"], dtype=float)
for w in windows:
halves.loc[w, "first"] = backtest(first, w)["total"]
halves.loc[w, "second"] = backtest(second, w)["total"]
halves.index.name = "window"
print(halves.round(4))
# -> first second
# -> window
# -> 20 0.5117 0.4056
# -> 50 0.2763 0.3098
# -> 100 0.1151 0.1942
# -> 150 -0.0119 0.4593
# -> 200 -0.1202 0.4306 first second
window
20 0.5117 0.4056
50 0.2763 0.3098
100 0.1151 0.1942
150 -0.0119 0.4593
200 -0.1202 0.4306first 20
second 150
dtype: int64The 20 day window wins the first half with 51.17% and the 150 day window wins the second with 45.93%. The 150 day window lost 1.19% in the first half, and the 200 day window lost 12.02% in the first half and made 43.06% in the second.
A sweep tells you how each parameter did on the sample you ran it on. It does not tell you which one will do best next. Picking the highest number from a list of tries measures the list as well as the rule, because the more values you try the higher that top number goes on its own.
Further reading: Why a model with no predictive power can score 90% accuracy.
Run the Step 2 sweep on MSFT over the same five windows, add buy and hold, and sort by total. Which window returns most, and does any window beat holding MSFT?
msft = close_of("MSFT")
rows = []
for w in [20, 50, 100, 150, 200]:
result = backtest(msft, w)
result["rule"] = f"sma {w}"
rows.append(result)
sweep = pd.DataFrame(rows).set_index("rule")
ret = msft.pct_change().dropna()
sweep.loc["buy and hold"] = {
"total": (1 + ret).prod() - 1,
"sharpe": ret.mean() / ret.std() * np.sqrt(252),
"turnover": 0.0,
}
print(sweep.sort_values("total", ascending=False).round(4))
# -> total sharpe turnover
# -> rule
# -> buy and hold 4.1698 1.3756 0.0
# -> sma 20 3.9881 1.6637 155.0
# -> sma 50 3.9022 1.5744 75.0
# -> sma 200 3.0213 1.3704 19.0
# -> sma 100 2.8959 1.3392 49.0
# -> sma 150 2.5776 1.2506 45.0The 20 day window returns most at 398.81% and none of the five beats holding MSFT at 416.98%.