import numpy as np
import pandas as pd
def backtest(close, signal, cost_bps=0.0):
ret = close.pct_change().fillna(0.0)
position = signal.shift(1).fillna(0.0) # yesterday's signal, held today
turnover = position.diff().abs().fillna(0.0) # 1 on a day the position changes
cost = turnover * cost_bps / 10000
strategy_return = position * ret - cost
equity = (1 + strategy_return).cumprod()
return pd.DataFrame({"ret": ret,
"position": position,
"turnover": turnover,
"strategy_return": strategy_return,
"equity": equity})How do I put a backtest in one function?
Wrap the lag, the costs, the compounding and the metrics into one function that takes prices and a signal and hands back a table.
A backtest takes a price series and a signal, lags the signal by a day, charges costs on turnover, compounds what is left, and reports the result. That is five lines, so it fits in one function.
In this lesson I write backtest(close, signal, cost_bps=0), check it on five days I can do by hand, run it on AAPL with a price-above-50-day-SMA signal, and then put two signals and two tickers in one table next to buy and hold.
Step 1. The function
Every line here comes from an earlier lesson. pct_change() is Lesson 17, shift(1) is Lesson 30, diff().abs() is the turnover of Lesson 31, and cumprod() is the equity curve of Lesson 21. The function returns them all as one DataFrame so nothing is thrown away.
cost_bps is in basis points, so 5 means 5 hundredths of a percent charged each time the position moves by one unit. It defaults to 0.
Step 2. Five days I can check by hand
The price goes 100, 110, 121, 108.9, 108.9. The signal is on for the middle two days.
days = pd.to_datetime(["2024-01-02", "2024-01-03", "2024-01-04", "2024-01-05", "2024-01-08"])
close = pd.Series([100.0, 110.0, 121.0, 108.9, 108.9], index=days)
signal = pd.Series([0.0, 1.0, 1.0, 0.0, 0.0], index=days)
print(backtest(close, signal))
# -> ret position turnover strategy_return equity
# -> 2024-01-02 0.0 0.0 0.0 0.0 1.00
# -> 2024-01-03 0.1 0.0 0.0 0.0 1.00
# -> 2024-01-04 0.1 1.0 1.0 0.1 1.10
# -> 2024-01-05 -0.1 1.0 0.0 -0.1 0.99
# -> 2024-01-08 0.0 0.0 1.0 0.0 0.99 ret position turnover strategy_return equity
2024-01-02 0.0 0.0 0.0 0.0 1.00
2024-01-03 0.1 0.0 0.0 0.0 1.00
2024-01-04 0.1 1.0 1.0 0.1 1.10
2024-01-05 -0.1 1.0 0.0 -0.1 0.99
2024-01-08 0.0 0.0 1.0 0.0 0.99The signal turns on for the 3rd, and the position is on for the 4th. The +10% on the 3rd is missed, the +10% on the 4th is earned, the -10% on the 5th is taken, and one unit of money ends at 0.99. Turnover is 1 on the day the position goes on and 1 again on the day it comes off, so getting out is charged as well as getting in.
Step 3. AAPL and the 50-day rule
prices.csv sits next to this lesson and holds simulated daily closes for four tickers from 2020 to 2025. Two small functions, one to pull a ticker’s closes and one to build the signal, keep the call to backtest on a single line.
prices = pd.read_csv("prices.csv", parse_dates=["Date"])
def close_of(ticker):
px = prices[prices["Ticker"] == ticker].sort_values("Date")
return px.set_index("Date")["Close"]
def above_sma(close, window):
return (close > close.rolling(window).mean()).astype(float)
aapl = close_of("AAPL")
bt = backtest(aapl, above_sma(aapl, 50))
print(bt.head())
# -> ret position turnover strategy_return equity
# -> Date
# -> 2020-01-01 0.000000 0.0 0.0 0.0 1.0
# -> 2020-01-02 0.024774 0.0 0.0 0.0 1.0
# -> 2020-01-03 0.005199 0.0 0.0 0.0 1.0
# -> 2020-01-06 0.012671 0.0 0.0 0.0 1.0
# -> 2020-01-07 -0.010470 0.0 0.0 -0.0 1.0 ret position turnover strategy_return equity
Date
2020-01-01 0.000000 0.0 0.0 0.0 1.0
2020-01-02 0.024774 0.0 0.0 0.0 1.0
2020-01-03 0.005199 0.0 0.0 0.0 1.0
2020-01-06 0.012671 0.0 0.0 0.0 1.0
2020-01-07 -0.010470 0.0 0.0 -0.0 1.0The first 49 days have no 50-day mean to compare against, so the signal is off, the position is 0 and the curve sits at 1.0. Here is the stretch where it first switches.
print(bt.loc["2020-03-10":"2020-03-16"])
# -> ret position turnover strategy_return equity
# -> Date
# -> 2020-03-10 0.031592 0.0 0.0 0.000000 1.000000
# -> 2020-03-11 -0.016424 1.0 1.0 -0.016424 0.983576
# -> 2020-03-12 -0.012680 1.0 0.0 -0.012680 0.971104
# -> 2020-03-13 -0.035732 1.0 0.0 -0.035732 0.936404
# -> 2020-03-16 -0.001451 0.0 1.0 -0.000000 0.936404 ret position turnover strategy_return equity
Date
2020-03-10 0.031592 0.0 0.0 0.000000 1.000000
2020-03-11 -0.016424 1.0 1.0 -0.016424 0.983576
2020-03-12 -0.012680 1.0 0.0 -0.012680 0.971104
2020-03-13 -0.035732 1.0 0.0 -0.035732 0.936404
2020-03-16 -0.001451 0.0 1.0 -0.000000 0.936404The columns are there to be counted. The mean of position is the share of days in the market, and the sum of turnover is the number of position changes.
print(f"{bt['equity'].iloc[-1] - 1:.2%}") # -> 58.61%
print(f"{bt['position'].mean():.1%}") # -> 57.8%
print(int(bt["turnover"].sum())) # -> 11358.61%
57.8%
113113 changes over six years, each one charged at cost_bps:
for bps in (0, 5, 20):
b = backtest(aapl, above_sma(aapl, 50), cost_bps=bps)
print(bps, f"{b['equity'].iloc[-1] - 1:.2%}")
# -> 0 58.61%
# -> 5 49.89%
# -> 20 26.49%0 58.61%
5 49.89%
20 26.49%Step 4. Two signals, two tickers, one table
backtest returns a frame, and the metrics function from Lesson 33 turns a frame into five numbers. Together they take a signal to a row of a table. Buy and hold is the signal that is always 1.
def metrics(bt):
r = bt["strategy_return"]
equity = bt["equity"]
years = (bt.index[-1] - bt.index[0]).days / 365.25
return {"total": equity.iloc[-1] - 1,
"CAGR": equity.iloc[-1] ** (1 / years) - 1,
"vol": r.std() * np.sqrt(252),
"Sharpe": r.mean() / r.std() * np.sqrt(252),
"max_dd": (equity / equity.cummax() - 1).min()}
def ema_cross(close, fast, slow):
return (close.ewm(span=fast).mean() > close.ewm(span=slow).mean()).astype(float)
def hold(close):
return pd.Series(1.0, index=close.index)Now one dictionary of runs, one dictionary comprehension, one table.
msft = close_of("MSFT")
runs = {"AAPL buy and hold": backtest(aapl, hold(aapl)),
"AAPL SMA 50": backtest(aapl, above_sma(aapl, 50)),
"AAPL EMA 20/100": backtest(aapl, ema_cross(aapl, 20, 100)),
"MSFT buy and hold": backtest(msft, hold(msft)),
"MSFT SMA 50": backtest(msft, above_sma(msft, 50))}
summary = pd.DataFrame({name: metrics(bt) for name, bt in runs.items()}).T
print(summary.round(4))
# -> total CAGR vol Sharpe max_dd
# -> AAPL buy and hold 1.6545 0.1767 0.2349 0.7863 -0.2982
# -> AAPL SMA 50 0.5861 0.0799 0.1793 0.5036 -0.2780
# -> AAPL EMA 20/100 0.1852 0.0287 0.1843 0.2405 -0.4034
# -> MSFT buy and hold 4.1698 0.3150 0.2081 1.3752 -0.3295
# -> MSFT SMA 50 3.9022 0.3034 0.1721 1.5734 -0.1523 total CAGR vol Sharpe max_dd
AAPL buy and hold 1.6545 0.1767 0.2349 0.7863 -0.2982
AAPL SMA 50 0.5861 0.0799 0.1793 0.5036 -0.2780
AAPL EMA 20/100 0.1852 0.0287 0.1843 0.2405 -0.4034
MSFT buy and hold 4.1698 0.3150 0.2081 1.3752 -0.3295
MSFT SMA 50 3.9022 0.3034 0.1721 1.5734 -0.1523On this sample buy and hold has the higher total return on both tickers: 165.45% against 58.61% and 18.52% on AAPL, and 416.98% against 390.22% on MSFT. The rules hold the position about 58% of the time, which shows up in the lower volatility, and on MSFT that gives the rule the higher Sharpe ratio at a lower total.
Adding a sixth row now costs one line, which is the point of writing it as a function. Lesson 35 runs the window from 20 to 200 through the same call.
Your turn
Run backtest on JNJ with a 100-day SMA signal and compare it against buy and hold on total return, Sharpe and max drawdown.
TipShow answer
jnj = close_of("JNJ")
for name, sig in [("buy and hold", hold(jnj)), ("SMA 100", above_sma(jnj, 100))]:
m = metrics(backtest(jnj, sig))
print(f"{name:14} total {m['total']:7.2%} Sharpe {m['Sharpe']:.2f} max_dd {m['max_dd']:.2%}")
# -> buy and hold total 107.03% Sharpe 0.88 max_dd -20.17%
# -> SMA 100 total 44.89% Sharpe 0.57 max_dd -16.22%