Does your backtested Sharpe ratio show a real edge?

Python
Backtesting
Tutorial
Code
In a simulated market with no edge, the best of 216 strategies has a Sharpe ratio of 0.72, which searching alone matches or beats 41% of the time.
Published

July 13, 2026

A backtest gives one Sharpe ratio, the average return divided by the volatility, the standard deviation of the returns, scaled to a year (here without deducting the risk-free rate, the return on cash). That Sharpe ratio is measured with error, from the data and from choices such as which strategy to keep after trying many. Bootstrapping, described below, measures how large that error is.

I build a market in which each stock’s expected return for the next day is zero, whatever happened before, so no strategy has an edge, an expected profit. The best of 216 strategies has a Sharpe ratio of 0.72. When the whole search is repeated on resampled returns with no edge, the best Sharpe ratio is at least 0.72 in 41% of the repetitions, so 0.72 is within what searching alone produces.

What bootstrapping does

A backtested Sharpe ratio comes from one historical path, and that path is only one of many the market could have taken. Under a different sequence of returns, the Sharpe ratio would also have been different.

Bootstrapping uses the data I already have to simulate those alternative paths. I resample the observed returns, drawing days at random with replacement, so the same day can be drawn more than once. The days are drawn in blocks of consecutive days, so that runs of calm or volatile days stay together. I then recompute the Sharpe ratio for each resampled series.

Repeating the resampling thousands of times turns one estimate into a distribution. The standard deviation of that distribution measures the uncertainty that the original Sharpe ratio alone does not show.

Simulating the data

To test the approach, I simulate 40 stocks over 3,024 trading days, about twelve years, in a market with no edge, so any pattern in the strategy returns is noise. Each stock’s daily return is its exposure to a common market factor, a return shared by all stocks, plus its own noise, and every part has an expected value of zero. The market factor’s volatility changes over time, so calm and volatile days come in runs, which is why the bootstrap below resamples blocks of consecutive days. The box below gives the formula and the code.

Stock \(i\)’s return on day \(t\) is its exposure to a common market factor plus its own noise,

\[r_{i,t} = \beta_i\, m_t + \sigma_i\, \varepsilon_{i,t}, \qquad m_t = \sqrt{h_t}\, z_t, \qquad h_t = 0.05 \times 0.01^2 + 0.90\, h_{t-1} + 0.05\, m_{t-1}^2,\]

where the exposure \(\beta_i\) is drawn with equal chance anywhere between 0.6 and 1.4 and the stock’s own daily volatility \(\sigma_i\) between 1.2% and 2.2%, once per stock. The \(\varepsilon_{i,t}\) and \(z_t\) are draws from the standard normal distribution, which is bell-shaped with a mean of 0 and a standard deviation of 1, independent across stocks and days and of each other. The market factor’s variance \(h_t\), the square of its standard deviation, keeps 90% of yesterday’s variance and adds 5% of yesterday’s squared market return, so calm and volatile days come in runs. The variance starts at \(0.01^2\), and the first term keeps its long-run level there, a daily market volatility of 1%. The two ranges, the weights of 0.90 and 0.05 and the volatility of 1% are assumptions. Every term has a mean of zero, so no stock’s expected return depends on the past, and the simulation contains no momentum, reversal, or low-volatility effect.

import numpy as np
import pandas as pd

def simulate_panel(n_stocks=40, n_days=3024, seed=7):
    """Daily returns for a panel of stocks, with no edge in them.

    Each stock is its own beta times a common market factor, plus noise that belongs to
    that stock alone. The factor has a mean of zero, so there is no premium to earn by
    holding the market and no strategy built on these returns has an expected edge.
    """
    random_generator = np.random.default_rng(seed)
    dates = pd.bdate_range("2014-01-02", periods=n_days)
    beta = random_generator.uniform(0.6, 1.4, n_stocks)       # exposure to the market factor
    idio_vol = random_generator.uniform(0.012, 0.022, n_stocks)   # stock-specific daily vol

    # The market factor is GARCH-like: today's variance is built out of yesterday's
    # variance and yesterday's squared return, so low-variance and high-variance periods arrive
    # in runs rather than at random. That clustering is the reason the bootstrap further
    # down resamples blocks of consecutive days instead of single days.
    market_variance = np.empty(n_days)
    market_return = np.empty(n_days)
    market_shock = random_generator.standard_normal(n_days)
    market_variance[0] = 0.01 ** 2
    for t in range(n_days):
        if t > 0:
            market_variance[t] = (0.01**2 * 0.05 + 0.90 * market_variance[t-1]
                                  + 0.05 * market_return[t-1]**2)
        market_return[t] = np.sqrt(market_variance[t]) * market_shock[t]

    stock_returns = np.zeros((n_days, n_stocks))
    for t in range(n_days):
        stock_noise = random_generator.standard_normal(n_stocks)
        stock_returns[t] = beta * market_return[t] + idio_vol * stock_noise

    tickers = [f"S{i:02d}" for i in range(n_stocks)]   # :02d pads to two digits: S00, S01
    return pd.DataFrame(stock_returns, index=dates, columns=tickers)

returns = simulate_panel()

simulated_stocks = returns.shape[1]              # shape is (rows, columns)
simulated_days = returns.shape[0]
print(f"{simulated_stocks} stocks x {simulated_days} trading days, "
      f"{returns.index[0].date()} to {returns.index[-1].date()}")
40 stocks x 3024 trading days, 2014-01-02 to 2025-08-05

Searching for the best strategy

I run 216 strategies on the same simulated market: three signal families, momentum (buy recent winners), reversal (buy recent losers) and low volatility (buy the calmest stocks), with six lookback windows, four portfolio sizes and three trading lags, 3 × 6 × 4 × 3 = 216. Each strategy buys one dollar of the stocks with the highest signal and sells one dollar of the stocks with the lowest, split equally across the stocks on each side. The strategy updates its choice of stocks monthly and restores the equal weights daily. So a strategy’s daily return is the average return of the stocks it buys minus the average return of the stocks it sells,

\[R_t = \overline{r}_{\text{long},t} - \overline{r}_{\text{short},t},\]

and if the stocks bought gain 2% on a day and the stocks sold gain 1%, the strategy earns 1%. The strategy’s annualised Sharpe ratio is

\[\widehat{S} = \sqrt{252}\,\frac{\overline{R}}{s_R},\]

where \(\overline{R}\) is the average daily strategy return and \(s_R\) the sample standard deviation of the daily strategy returns. Multiplying by \(\sqrt{252}\), the square root of the number of trading days in a year, turns the daily ratio into an annual one as if the days were independent. The strategy returns include no trading costs. The box below defines the signals and builds each strategy’s daily returns, and the block after it computes the Sharpe ratios and picks the winner.

For stock \(i\) on day \(t\), the momentum signal is the stock’s log return over the \(L\) days that end \(s\) days earlier,

\[\sum_{k=s}^{s+L-1} \ln(1 + r_{i,t-k}),\]

where \(L\) is the lookback window, 21 to 252 days, \(s\) is the trading lag of 1, 5 or 21 days, and \(\ln(1 + r)\) turns a daily return into a log return, which adds across days. The reversal signal is minus the same sum over \(L/3\) days, so it buys recent losers, and the low-volatility signal is minus the standard deviation of the stock’s daily returns over the \(L\) days, so it buys the calmest stocks. Between the monthly updates, the code keeps each stock’s weight fixed, which is the same as trading every day to restore the weights after prices move. Because the stocks have different exposures to the market factor, the strategies keep some exposure to it. The factor has a mean of zero, so no strategy has an expected edge by construction.

# Three signal families times these three lists is 216 strategies.
LOOKBACKS = [21, 42, 63, 126, 189, 252]    # days the signal reads; reversal reads a third
HOLDINGS  = [3, 5, 8, 12]                  # stocks held on each side
SKIPS     = [1, 5, 21]                     # days between the signal and the trade


def long_short_weights(signal, n_hold, rebalance_day):
    """Buy the n_hold stocks with the highest signal, sell the n_hold lowest.

    Each side is equally weighted and each side sums to one, so the portfolio holds a
    dollar long for every dollar short.
    """
    # rank(axis=1) ranks the stocks against each other within each day, rather than
    # ranking each stock down its own column. A stock with no signal yet is left
    # unranked, and the notna() check keeps it out of the portfolio.
    rank_from_top = signal.rank(axis=1, ascending=False)
    rank_from_bottom = signal.rank(axis=1, ascending=True)
    long_flag = ((rank_from_top <= n_hold) & signal.notna()).astype(float)
    short_flag = ((rank_from_bottom <= n_hold) & signal.notna()).astype(float)

    # Divide by the number of names held that day, so the weights on each side
    # add to one. replace(0, np.nan) leaves a day with nothing to hold missing rather
    # than dividing by zero.
    long_count = long_flag.sum(axis=1).replace(0, np.nan)
    short_count = short_flag.sum(axis=1).replace(0, np.nan)
    long_weight = long_flag.div(long_count, axis=0)      # axis=0 divides each row by its own count
    short_weight = short_flag.div(short_count, axis=0)
    weight = long_weight - short_weight

    # where() keeps the weights on rebalance days and blanks every other day, and ffill
    # then copies the last set forward, so the same weights apply every day until the next
    # rebalance. Holding the shares instead would let the weights change with prices. Days
    # before the first rebalance stay missing, because the strategy is not active yet.
    return weight.where(rebalance_day).ffill()


def build_grid(returns):
    """One column per strategy, holding that strategy's daily portfolio return."""
    # log1p turns a simple return into a log return, which can be added across days.
    log_return = np.log1p(returns)

    # A rebalance happens on the first trading day of each calendar month. Grouping the
    # dates by year and month and asking for the first date in each group finds them.
    dates = returns.index.to_series()
    month_groups = dates.groupby([returns.index.year, returns.index.month])
    is_rebalance_day = month_groups.transform("first") == returns.index

    # The weights have one column per stock, so the day flag is repeated across the
    # columns to give a True or False for every stock on every day.
    rebalance_day = pd.DataFrame(
        np.repeat(is_rebalance_day.values[:, None], returns.shape[1], axis=1),
        index=returns.index, columns=returns.columns)

    strategy_columns = {}
    for lookback in LOOKBACKS:
        trailing_return = log_return.rolling(lookback).sum()
        trailing_vol = returns.rolling(lookback).std()
        for skip in SKIPS:
            for n_hold in HOLDINGS:
                # Reversal reads a shorter window than momentum: a third of the lookback,
                # and never fewer than five days.
                reversal_window = max(lookback // 3, 5)
                reversal_return = log_return.rolling(reversal_window).sum()
                signals = {
                    "mom": trailing_return.shift(skip),      # buy the winners
                    "rev": -reversal_return.shift(skip),     # buy the losers
                    "lvol": -trailing_vol.shift(skip),       # buy the low-volatility stocks
                }
                for signal_name, signal in signals.items():
                    weight = long_short_weights(signal, n_hold, rebalance_day)
                    # shift(1) applies today's return to yesterday's weights.
                    position_return = weight.shift(1) * returns
                    # min_count=1 keeps a day missing while no position is held yet,
                    # instead of adding nothing up and calling the answer zero.
                    daily_return = position_return.sum(axis=1, min_count=1)
                    column_name = f"{signal_name}_lb{lookback}_n{n_hold}_sk{skip}"
                    strategy_columns[column_name] = daily_return
    # Keep only days on which every strategy is active, so all 216 are judged on the
    # same sample.
    return pd.DataFrame(strategy_columns).dropna(how="any")



strategy_returns = build_grid(returns)

Every signal is lagged by at least one day, and .shift(1) applies today’s return to yesterday’s weights. No strategy uses information that was unavailable when the position was formed. All 216 are compared on the 2,741 days on which every one of them holds positions. The first 283 days are lost, because the longest signal needs 252 days of history, a 21-day lag and then a first rebalance.

TRADING_DAYS_PER_YEAR = 252                # turns a daily Sharpe ratio into an annual one


def sharpe(returns_array, axis=0):
    """Annualised Sharpe ratio, with no risk-free rate deducted.

    ddof=1 asks for the sample standard deviation, which divides by n - 1 rather than n.
    """
    values = np.asarray(returns_array)
    average = values.mean(axis=axis)
    volatility = values.std(axis=axis, ddof=1)
    daily_sharpe = average / volatility
    return daily_sharpe * np.sqrt(TRADING_DAYS_PER_YEAR)

observed = pd.Series(sharpe(strategy_returns.values), index=strategy_returns.columns)
winner = observed.idxmax()                       # idxmax gives the label of the highest value
winner_sharpe = observed.loc[winner]             # .loc looks up the value with that label

print(f"{strategy_returns.shape[1]} candidates over {strategy_returns.shape[0]} common days")
print(f"grid Sharpe   min {observed.min():.2f} | median {observed.median():.2f} | "
      f"90th {observed.quantile(0.90):.2f} | max {observed.max():.2f}")
print(f"winner        {winner}   Sharpe {winner_sharpe:.2f}")
216 candidates over 2741 common days
grid Sharpe   min -0.68 | median 0.08 | 90th 0.45 | max 0.72
winner        rev_lb21_n3_sk1   Sharpe 0.72

Across the 216 strategies, the Sharpe ratios run from −0.68 to 0.72, with a median of 0.08, the middle value when they are sorted, and 90% of them are below 0.45. The search selects a reversal strategy that ranks stocks on seven days of past returns, a third of its 21-day lookback, holds three stocks on each side and trades one day after the signal, which its printed name, rev_lb21_n3_sk1, abbreviates. Its Sharpe ratio is 0.72 over nearly eleven years. Past returns here say nothing about future expected returns, so that Sharpe ratio comes from chance.

Method 1: Measuring the precision of the 0.72

With the winning strategy fixed, I resample the winner’s daily returns in blocks and recompute the Sharpe ratio thousands of times.

I use the stationary bootstrap of Politis and Romano (1994). A resample is built day by day: with probability 1/21 a new block starts at a day drawn at random, and otherwise the day after the previously drawn day follows, wrapping from the last day back to the first. So the blocks have random lengths averaging 21 days, about a trading month, which keeps runs of calm or volatile days together while giving every day the same chance of being selected. Each resample has 2,741 days, as many as the original series. From the 5,000 resampled Sharpe ratios, I take the 2.5th and 97.5th percentiles as a 95% interval and their standard deviation as the bootstrap standard error.

def block_indices(n_days, n_draws, mean_block, seed=101):
    """One row per resample, holding the day numbers that resample is built from.

    This is the stationary bootstrap. Each day either continues the block that is
    running or starts a new block at a randomly chosen day, and the chance of starting
    a new one is 1 / mean_block. That makes the block lengths random with an average of
    mean_block days, and blocks of consecutive days keep the day-to-day dependence partly intact.
    """
    random_generator = np.random.default_rng(seed)
    new_block_chance = 1.0 / mean_block

    indices = np.empty((n_draws, n_days), dtype=np.int32)
    indices[:, 0] = random_generator.integers(0, n_days, n_draws)
    for t in range(1, n_days):
        starts_new_block = random_generator.random(n_draws) < new_block_chance
        fresh_day = random_generator.integers(0, n_days, n_draws)
        next_day = (indices[:, t-1] + 1) % n_days    # % wraps the last day back to the first
        # np.where makes the choice separately for every draw: the fresh day where a new
        # block starts, and otherwise the day after the one already drawn.
        indices[:, t] = np.where(starts_new_block, fresh_day, next_day)
    return indices


def bootstrap_sharpes(daily_returns, resample_indices):
    """Sharpe ratio of every column of daily_returns, for every resample.

    A resample is a list of day numbers, and a Sharpe ratio does not depend on the order
    those days come in. Only how often each day was drawn matters, so counting the draws
    once and then multiplying the counts by the returns gives the same answer as building
    thousands of resampled return series one at a time.
    """
    n_draws, n_days = resample_indices.shape

    draw_counts = np.zeros((n_draws, n_days))
    for draw in range(n_draws):
        # bincount counts how often each day number appears. minlength keeps the row the
        # full length of the series, so a day never drawn is recorded as zero.
        draw_counts[draw] = np.bincount(resample_indices[draw], minlength=n_days)

    # @ is matrix multiplication. It multiplies each row of counts through the returns
    # and adds the products, giving one total per draw per strategy.
    sum_of_returns = draw_counts @ daily_returns
    sum_of_squares = draw_counts @ daily_returns**2

    mean_return = sum_of_returns / n_days
    # The sample variance rebuilt from the two sums, dividing by n - 1 as ddof=1 does.
    variance = (sum_of_squares - n_days * mean_return**2) / (n_days - 1)

    daily_sharpe = mean_return / np.sqrt(variance)
    return daily_sharpe * np.sqrt(TRADING_DAYS_PER_YEAR)

BOOTSTRAP_DRAWS = 5_000
MEAN_BLOCK = 21                # about one trading month of consecutive days per block

n_common_days = strategy_returns.shape[0]
resample_indices = block_indices(n_common_days, BOOTSTRAP_DRAWS, MEAN_BLOCK)

# The double brackets ask for a one-column frame rather than a plain series, which is
# the shape bootstrap_sharpes works on. ravel() then flattens its result, which has one
# row per draw and one column per strategy, into one Sharpe ratio per draw.
winner_returns = strategy_returns[[winner]].values
winner_sharpes = bootstrap_sharpes(winner_returns, resample_indices).ravel()

interval_low, interval_high = np.percentile(winner_sharpes, [2.5, 97.5])
bootstrap_se = winner_sharpes.std()
share_at_or_below_zero = (winner_sharpes <= 0).mean()

print(f"observed Sharpe   {winner_sharpe:.2f}")
print(f"bootstrap se      {bootstrap_se:.2f}")
print(f"95% interval      [{interval_low:.2f}, {interval_high:.2f}]")
print(f"share of bootstrap Sharpe ratios at or below 0: {share_at_or_below_zero:.1%}")
observed Sharpe   0.72
bootstrap se      0.30
95% interval      [0.14, 1.31]
share of bootstrap Sharpe ratios at or below 0: 0.7%

The distribution of the 5,000 resampled Sharpe ratios is centred near 0.72 and wide. The 95% interval runs from 0.14 to 1.31, and the bootstrap standard error is 0.30.

Read on its own, the interval misleads. The interval is above zero, and only 0.7% of the bootstrapped Sharpe ratios are at or below zero, so a conventional test would call the strategy significant, meaning unlikely to look this good by chance. The simulation contains no predictable expected returns, so the result is still produced by chance. Method 1 measures how precisely the winner’s Sharpe ratio is estimated, and it treats the winner as if it had been chosen in advance.

If the resampling is new to you, Lesson 36 builds the same idea on a single return series, and Lesson 33 covers the Sharpe ratio itself.

Method 2: Testing the winner against searches with no edge

The 0.72 is the highest Sharpe ratio among 216 strategies, and the best result of a search across many strategies can have a high Sharpe ratio even when none of the strategies has an edge. Method 2 repeats the whole search on returns with no edge. To remove any edge, the method first subtracts each strategy’s average return from its daily returns,

\[R^0_{j,t} = R_{j,t} - \overline{R}_j,\]

where \(R_{j,t}\) is strategy \(j\)’s return on day \(t\) and \(\overline{R}_j\) its average. With daily returns of 1% and 3%, for example, the average is 2%, and subtracting it gives −1% and 1%. Every strategy then has an average return of zero and keeps its volatility and its co-movement with the others.

The bootstrap resamples these daily returns of the 216 strategies already built, without rebuilding the portfolios, and uses the same days for all 216, which keeps the days that are good or bad for many strategies at once. Each resample records the highest Sharpe ratio, the rule that picked the winner. The 5,000 resamples give the null distribution, the distribution of the winning Sharpe ratio when no strategy has an edge. The p-value is the share of resampled searches whose best Sharpe ratio is at least the observed 0.72, computed as \((1 + c)/(5{,}000 + 1)\), where \(c\) is the number of such searches and the added ones count the observed search as one more.

# Taking each strategy's own average return away leaves its volatility, and its
# co-movement with the other 215, exactly as they were.
null_panel = strategy_returns - strategy_returns.mean()

# One Sharpe ratio per draw per strategy: 5000 rows by 216 columns.
null_grid = bootstrap_sharpes(null_panel.values, resample_indices)

# axis=1 takes the maximum across the strategies, so each row is the winner of one
# simulated search over the whole grid.
best_of_search = null_grid.max(axis=1)

# Counting the observed search itself, by adding one above and below, keeps the p-value
# away from an exact zero that the number of draws cannot support.
searches_at_least_as_good = (best_of_search >= winner_sharpe).sum()
p_value = (1 + searches_at_least_as_good) / (BOOTSTRAP_DRAWS + 1)

null_median = np.median(best_of_search)
null_90th = np.percentile(best_of_search, 90)
null_95th = np.percentile(best_of_search, 95)
share_beaten = (winner_sharpe > best_of_search).mean()

print("best Sharpe from a search with no edge:")
print(f"  median {null_median:.2f} | 90th {null_90th:.2f} "
      f"| 95th {null_95th:.2f} | max {best_of_search.max():.2f}")
print(f"our winner   {winner_sharpe:.2f}")
print(f"p-value      {p_value:.3f}   (beats {share_beaten:.0%} of searches with no edge)")
best Sharpe from a search with no edge:
  median 0.69 | 90th 0.91 | 95th 0.98 | max 1.65
our winner   0.72
p-value      0.407   (beats 59% of searches with no edge)

A search across 216 no-edge strategies produces a median winning Sharpe ratio of 0.69. In 90% of the searches the best Sharpe ratio is below 0.91, in 95% below 0.98, and the highest of the 5,000 is 1.65, so a winner would need a Sharpe ratio near 1 to be unusual for a search with no edge. In 41% of the resampled searches, the highest Sharpe ratio is at least the observed 0.72, a p-value of 0.41, and the observed 0.72 beats the other 59%. So, once the full search is accounted for, the 0.72 is ordinary.

The two methods answer different questions. Method 1 asks how precisely the winner’s Sharpe ratio is estimated. The answer is imprecisely, although the 95% interval is above zero. Method 2 asks whether 0.72 is unusual after searching across 216 strategies. The answer is no. The chart below shows both results.

import matplotlib.pyplot as plt

RED, TEAL, ORANGE, GREY = "#C0392B", "#17868A", "#D2822B", "#888888"
# The site's chart colours: a cream background, a light grid and a pale frame, so every chart
# on the site has the same look.
BG, INK, GRID, SPINE, AXIS_TEXT = "#FCEFE3", "#1f1f1f", "#EADCCC", "#D5C6B4", "#4a4a4a"


def style_chart(figure, axis):
    """Give a finished chart the site's cream background, light grid and pale frame."""
    figure.patch.set_facecolor(BG)
    axis.set_facecolor(BG)
    axis.grid(True, axis="y", color=GRID, lw=1.0)
    axis.set_axisbelow(True)
    for side in ("top", "right"):
        axis.spines[side].set_visible(False)
    for side in ("left", "bottom"):
        axis.spines[side].set_color(SPINE)
    axis.tick_params(axis="both", length=0, colors=INK, pad=6)
    axis.xaxis.label.set_color(AXIS_TEXT)
    axis.yaxis.label.set_color(AXIS_TEXT)
    axis.title.set_color(INK)
    # Each legend entry in the colour of what it labels: a line's colour or a bar's fill.
    # get_legend() returns None when the chart has no legend.
    legend = axis.get_legend()
    if legend is not None:
        for text, handle in zip(legend.get_texts(), legend.legend_handles):
            if hasattr(handle, "get_color"):
                text.set_color(handle.get_color())
            else:
                text.set_color(handle.get_facecolor())


# plt.subplots(1, 2) returns the figure and the two panels side by side, and unpacking
# them here gives each panel a name instead of a position.
fig, (method1_panel, method2_panel) = plt.subplots(1, 2, figsize=(11, 4))

method1_panel.hist(winner_sharpes, bins=60, color=TEAL, alpha=.85)
method1_panel.axvline(winner_sharpe, color=INK, lw=1.5,
                      label=f"observed {winner_sharpe:.2f}")
method1_panel.axvline(interval_low, color=INK, ls="--", lw=1)
method1_panel.axvline(interval_high, color=INK, ls="--", lw=1)
method1_panel.axvline(0, color=RED, lw=1.2)
method1_panel.set_title("Method 1: how precisely is the winner measured?")
method1_panel.set_xlabel("bootstrapped Sharpe ratio")
method1_panel.legend(frameon=False)
style_chart(fig, method1_panel)

method2_panel.hist(best_of_search, bins=60, color=GREY, alpha=.85,
                   label="best Sharpe in a search with no edge")
method2_panel.axvline(winner_sharpe, color=INK, lw=1.5,
                      label=f"observed winner {winner_sharpe:.2f}")
method2_panel.axvline(null_95th, color=RED, ls="--", lw=1.2,
                      label="95th percentile of the null")
method2_panel.set_title("Method 2: best Sharpe ratio in a search with no edge")
method2_panel.set_xlabel("best Sharpe ratio in the search")
method2_panel.legend(frameon=False, fontsize=8)
style_chart(fig, method2_panel)

plt.tight_layout()
plt.savefig("sharpe_uncertainty.png", dpi=120, bbox_inches="tight", facecolor=BG)
plt.show()

On the left, the teal histogram is the bootstrap distribution of the winner’s Sharpe ratio. The histogram is centred near 0.72, the solid line, and the 95% interval between the dashed lines is above zero, the red line. On the right, the grey histogram holds the best Sharpe ratios of the searches in which every strategy has a mean return of zero, and the same 0.72 is near its centre.

Limitations

Method 2 gives a p-value and does not correct the reported Sharpe ratio for the selection. The winner’s Sharpe ratio is the highest of 216, so it is likely to overstate the strategy’s performance outside this sample, even when the strategy has a real edge.

Method 2’s result also depends on including the full search. Every strategy tested and later discarded still counts in the search. Leaving discarded strategies out lowers the null distribution and makes the winner appear more unusual.

Finally, both methods resample the history already observed, and the strategy returns here include no trading costs. So the two methods do not show how the strategy would perform after trading costs or under a future market that behaves differently. Deduct costs before running the test.

Conclusion

The takeaway is that the best of many backtests has to be compared with the best that the same search finds without any edge: here that search reaches 0.72 or more in 41% of the repetitions, so the winner’s 0.72 shows no edge.