How do I backtest with a package?

Run the Lesson 34 rule through the bt package and check its numbers against the hand built ones.

The bt package runs a backtest from a price table and a signal in a strategy of three algos and one run call. Lesson 34 did the same job with a function I wrote myself.

In this lesson I put the price-above-50-day-SMA rule from Lesson 29 through bt, pull five numbers out of the result, and print them next to the hand built ones.

Step 1. The prices, in the wide shape

bt wants a DataFrame indexed by date with one column per ticker. That is the wide shape from Lesson 25, and pivot gets there in one call. prices.csv sits next to this lesson and holds simulated daily closes for four tickers from 2020 to 2025.

import numpy as np
import pandas as pd
import bt

prices = pd.read_csv("prices.csv", parse_dates=["Date"])                # long frame, one row per ticker per day
wide   = prices.pivot(index="Date", columns="Ticker", values="Close")   # dates down, tickers across

print(wide.shape)                                                       # -> (1566, 4)

print(wide.head())
# -> Ticker        AAA     BBB     CCC    DDD
# -> Date
# -> 2020-01-01  75.08  144.61  160.26  58.98
# -> 2020-01-02  76.94  144.85  160.44  59.55
# -> 2020-01-03  77.34  145.17  159.84  61.45
# -> 2020-01-06  78.32  145.80  161.90  62.74
# -> 2020-01-07  77.50  146.50  162.89  63.62
(1566, 4)
Ticker        AAA     BBB     CCC    DDD
Date                                    
2020-01-01  75.08  144.61  160.26  58.98
2020-01-02  76.94  144.85  160.44  59.55
2020-01-03  77.34  145.17  159.84  61.45
2020-01-06  78.32  145.80  161.90  62.74
2020-01-07  77.50  146.50  162.89  63.62

The signal is the same comparison as Lesson 29, one column wide because the run is on AAA alone.

aapl   = wide[["AAA"]]                          # double brackets keep it a DataFrame, not a Series
signal = aapl > aapl.rolling(50).mean()          # True while the close leads its own 50-day mean

print(int(signal["AAA"].sum()), "days on out of", len(signal))
# -> 906 days on out of 1566
906 days on out of 1566

Step 2. The strategy

A bt.Strategy is a name and a list of algos, read top to bottom on every bar. Three of them cover this rule: which names to hold, how much of each, then trade to those weights.

s = bt.Strategy("above sma50", [bt.algos.SelectWhere(signal),   # hold AAA on the days signal is True
                                bt.algos.WeighEqually(),        # split the money evenly across what was picked
                                bt.algos.Rebalance()])          # place the trades that reach those weights

res = bt.run(bt.Backtest(s, aapl))                              # a Backtest is a strategy plus its prices

SelectWhere picks one name here, so WeighEqually puts 100% in it on the days the rule is on and nothing on the days it is off. Rebalance fills at the close of the bar it runs on, so the position earns from the next bar forward. That is the one-day lag of Lesson 30, applied by the package.

Step 3. The result

res.stats is a DataFrame with 46 rows, one per measure, and one column per backtest.

print(res.stats.shape)                     # -> (46, 1)

keep = ["total_return", "cagr", "daily_sharpe", "daily_vol", "max_drawdown"]

print(res.stats.loc[keep])
# ->              above sma50
# -> total_return    0.586086
# -> cagr            0.079892
# -> daily_sharpe    0.503652
# -> daily_vol       0.179307
# -> max_drawdown   -0.278002
(46, 1)
             above sma50
total_return    0.586086
cagr            0.079892
daily_sharpe    0.503652
daily_vol       0.179307
max_drawdown   -0.278002

Step 4. The same rule by hand

Those five numbers came from four lines of setup. Here is the Lesson 34 version of the same rule: returns, the signal lagged a day, the product, the compounded curve.

close    = wide["AAA"]                          # a plain Series this time
ret      = close.pct_change().fillna(0.0)        # the asset's own daily return
position = (close > close.rolling(50).mean()).astype(float).shift(1).fillna(0.0)
strategy_return = position * ret                 # earned only while the position is on
equity   = (1 + strategy_return).cumprod()       # one unit of money compounded

hand = pd.Series({"total_return": equity.iloc[-1] - 1,
                  "sharpe":       strategy_return.mean() / strategy_return.std() * np.sqrt(252),
                  "max_drawdown": (equity / equity.cummax() - 1).min()})

pkg = pd.Series({"total_return": res.stats.loc["total_return", "above sma50"],   # same three, from bt
                 "sharpe":       res.stats.loc["daily_sharpe", "above sma50"],
                 "max_drawdown": res.stats.loc["max_drawdown", "above sma50"]})

print(pd.DataFrame({"bt": pkg, "hand": hand, "gap": pkg - hand}).round(6))
# ->                     bt      hand       gap
# -> total_return  0.586086  0.586106 -0.000020
# -> sharpe        0.503652  0.503649  0.000003
# -> max_drawdown -0.278002 -0.278009  0.000007
                    bt      hand       gap
total_return  0.586086  0.586106 -0.000020
sharpe        0.503652  0.503649  0.000003
max_drawdown -0.278002 -0.278009  0.000007

The two agree to about four decimal places. 0.586086 against 0.586106 is 58.61% either way, which is the figure Lesson 34 printed and the figure in the sweep of Lesson 35.

The remaining gap is bookkeeping. bt models the trade on the bar Rebalance runs on and carries a cash balance through the first bar, while the hand built run is a position column multiplied by a return column with a zero in the first row. Two ways of writing down the same rule, differing in the fifth decimal.

Step 5. Buy and hold through bt

Buy and hold is SelectAll instead of SelectWhere, wrapped in RunOnce so the list fires on the first bar and never again. Passing two Backtest objects to one bt.run puts both in one table.

hold = bt.Strategy("buy and hold", [bt.algos.RunOnce(),      # run the rest of the list once, on bar one
                                    bt.algos.SelectAll(),    # every column in the price frame
                                    bt.algos.WeighEqually(),
                                    bt.algos.Rebalance()])

both = bt.run(bt.Backtest(s, aapl), bt.Backtest(hold, aapl))  # two backtests, one call

print(both.stats.loc[keep])
# ->              above sma50 buy and hold
# -> total_return    0.586086     1.654486
# -> cagr            0.079892      0.17665
# -> daily_sharpe    0.503652     0.786288
# -> daily_vol       0.179307      0.23494
# -> max_drawdown   -0.278002    -0.298204
             above sma50 buy and hold
total_return    0.586086     1.654486
cagr            0.079892      0.17665
daily_sharpe    0.503652     0.786288
daily_vol       0.179307      0.23494
max_drawdown   -0.278002    -0.298204

1.654486 is the 165.45% quoted for AAA buy and hold throughout the track, and 0.786288 is the 0.7865 Sharpe of Lesson 33. On this sample the rule ends below buy and hold on total return, at lower volatility, exactly as the hand built run reported.

Your turn

Run the 50-day rule on CCC through bt and compare its total_return with the 390.22% the hand built backtester gave in Lesson 34.

msft = wide[["CCC"]]                                            # one column, DataFrame shape
sig  = msft > msft.rolling(50).mean()                            # the same rule, new ticker

strat = bt.Strategy("CCC sma50", [bt.algos.SelectWhere(sig),
                                   bt.algos.WeighEqually(),
                                   bt.algos.Rebalance()])

out = bt.run(bt.Backtest(strat, msft))

print(out.stats.loc[["total_return", "daily_sharpe", "max_drawdown"]])
# ->               CCC sma50
# -> total_return    3.901776
# -> daily_sharpe    1.573369
# -> max_drawdown   -0.152341

390.18% against 390.22%, and 1.5734 against 1.5734 on Sharpe.