survivors +2.2% a year, all stocks +0.3%, 199 of 1000 stopped trading

July 26, 2026

I read Rolf Dobelli’s book The Art of Thinking Clearly, and it made me think of how the same ideas could apply to people who conduct quantitative research. There are hundreds of these biases, and I only mention a few of them here, the ones that I think are significant.
In real data, you rarely know what the truth is, so I simulate six small markets where I fix the truth in advance, put one bias into each of them, and measure how far the result ends up from the truth. The numbers in these markets are arbitrary.
import numpy as np
import matplotlib.pyplot as plt
# One colour for the misleading number, one for the truth, one for the individual paths
RED = "#C0392B"
TEAL = "#17868A"
GREY = "#888888"
# The site's chart colours: a cream background, a light grid and a pale frame, so every chart
# on the site has the same look.
BG, INK, GRID, SPINE, AXIS_TEXT = "#FCEFE3", "#1f1f1f", "#EADCCC", "#D5C6B4", "#4a4a4a"
D = 252 # trading days in a year
def style_chart(figure, axis):
"""Give a finished chart the site's cream background, light grid and pale frame."""
figure.patch.set_facecolor(BG)
axis.set_facecolor(BG)
axis.grid(True, axis="y", color=GRID, lw=1.0)
axis.set_axisbelow(True)
for side in ("top", "right"):
axis.spines[side].set_visible(False)
for side in ("left", "bottom"):
axis.spines[side].set_color(SPINE)
axis.tick_params(axis="both", length=0, colors=INK, pad=6)
axis.xaxis.label.set_color(AXIS_TEXT)
axis.yaxis.label.set_color(AXIS_TEXT)
axis.title.set_color(INK)
# Each legend entry in the colour of what it labels: a line's colour or a bar's fill.
# get_legend() returns None when the chart has no legend.
legend = axis.get_legend()
if legend is not None:
for text, handle in zip(legend.get_texts(), legend.legend_handles):
if hasattr(handle, "get_color"):
text.set_color(handle.get_color())
else:
text.set_color(handle.get_facecolor())
def sharpe(daily_returns):
"""The annual Sharpe ratio of a series of daily returns, with no risk-free rate.
The average daily return is divided by the daily standard deviation, and the result is
multiplied by the square root of the trading days in a year, which is how a daily ratio
is converted to an annual ratio.
"""
returns = np.asarray(daily_returns) # so a list is accepted as well as an array
average_return = returns.mean()
return_volatility = returns.std(ddof=1) # ddof=1 is the sample standard deviation
return average_return / return_volatility * np.sqrt(D)The first bias is that you exclude delisted companies. Say you have a thousand stocks whose daily returns have a mean of zero, and over ten years, about two hundred of them fall so far that they stop trading. If you look only at the stocks still listed, their average return is 2.2% a year.
\[V_{i,t} = \prod_{s=1}^{t} (1 + r_{i,s}), \qquad r_{i,s} = 0.02\, z_{i,s},\]
where \(V_{i,t}\) is the value of stock \(i\) after day \(t\), \(\prod\) multiplies the factors for days 1 to \(t\), and \(z_{i,s}\) is a draw from the standard normal distribution, independent across stocks and days.
So, by not accounting for the stocks that stopped trading (survivorship bias), you falsely believe that the average return of the stocks is 2.2% a year, while the average return of all 1,000 stocks is 0.3% a year.
rng = np.random.default_rng(1)
NUMBER_OF_STOCKS = 1000
NUMBER_OF_DAYS = 10 * D # ten years of trading days
DAILY_VOLATILITY = 0.02 # the standard deviation of one day's return
DELISTING_LEVEL = 0.20 # the level at which a stock stops trading
# Every stock gets one random move per day, drawn around an average of zero, so in truth
# no stock has an edge. cumprod multiplies each column of daily moves together, which
# turns them into the value of 1 invested in that stock at the start.
daily_moves = rng.normal(0, DAILY_VOLATILITY, (NUMBER_OF_DAYS, NUMBER_OF_STOCKS))
value_paths = np.cumprod(1 + daily_moves, axis=0) # one column per stock
# True on every day a stock's value is below that level
below_level = value_paths < DELISTING_LEVEL
stopped_trading = below_level.any(axis=0) # one True or False for each stock
# A stock that falls that far never trades again, so its value is held at the level from
# the first day it reached it. argmax on a column of True and False returns the position
# of the first True, because True counts as 1 and False as 0.
for stock in np.where(stopped_trading)[0]:
first_day_below = below_level[:, stock].argmax()
value_paths[first_day_below:, stock] = DELISTING_LEVEL
# ~ flips True and False, so this average covers only the stocks still listed at the end
survivor_average = value_paths[:, ~stopped_trading].mean(axis=1)
all_stocks_average = value_paths.mean(axis=1) # every stock, delisted ones included
# Ten years of growth becomes an average yearly return by taking the tenth root, which is
# the power 0.1, and then removing the 1 that was invested at the start.
survivor_yearly = (survivor_average[-1] ** 0.1 - 1) * 100
all_stocks_yearly = (all_stocks_average[-1] ** 0.1 - 1) * 100
print(f"survivors {survivor_yearly:+.1f}% a year, "
f"all stocks {all_stocks_yearly:+.1f}%, "
f"{stopped_trading.sum()} of {NUMBER_OF_STOCKS} stopped trading")
years = np.arange(NUMBER_OF_DAYS) / D # the x-axis in years rather than days
fig, ax = plt.subplots(figsize=(9, 5))
for stock in np.where(stopped_trading)[0][:80]: # eighty of them, to keep the chart clear
ax.plot(years, value_paths[:, stock], color=GREY, lw=0.5, alpha=0.4)
ax.plot(years, survivor_average, color=RED, lw=2.4, label="survivors")
ax.plot(years, all_stocks_average, color=TEAL, lw=2.4, label="all stocks")
ax.axhline(1.0, color=GREY, lw=0.8)
ax.set_xlabel("years")
ax.set_ylabel("value of 1")
ax.set_title("Survivors vs all stocks", fontsize=13)
ax.legend(frameon=False)
style_chart(fig, ax)
plt.show()survivors +2.2% a year, all stocks +0.3%, 199 of 1000 stopped trading

In the chart, the red line is the stocks that are still listed, and the teal line is all 1,000 stocks.
The second bias is mistaking a correlation for a true relationship. For example, I build two price series that have nothing to do with each other, but both drift upward over time, and their price levels correlate at 0.86. But when I look at their changes from one period to the next, the correlation is only -0.02.
So the 0.86 only tells you that both series rise over time, and nothing about whether they are related.
rng = np.random.default_rng(2)
NUMBER_OF_PERIODS = 600
# Each series is a random walk with drift. Every period it takes a step, and the step is a
# fixed drift plus that period's own noise. cumsum adds the steps up, so the level wanders
# upward. The two sets of steps are drawn separately, so nothing connects them.
steps_a = 0.10 + rng.normal(0, 1.0, NUMBER_OF_PERIODS) # its own drift and noise
steps_b = 0.30 + rng.normal(0, 2.0, NUMBER_OF_PERIODS) # drawn separately from series A
series_a = 50 + np.cumsum(steps_a) # the level of series A, starting at 50
series_b = 100 + np.cumsum(steps_b) # the level of series B, starting at 100
# np.diff subtracts each value from the next, leaving the change from one period to the
# next. A change does not trend upward the way a level does, so a correlation of the changes has
# no shared trend to pick up.
changes_a = np.diff(series_a)
changes_b = np.diff(series_b)
# corrcoef returns a 2x2 table of every pair, so the number wanted is off the diagonal
levels_correlation = np.corrcoef(series_a, series_b)[0, 1]
changes_correlation = np.corrcoef(changes_a, changes_b)[0, 1]
print(f"levels r = {levels_correlation:.2f}, changes r = {changes_correlation:.2f}")
# subplots(1, 2) returns the two panels together, and unpacking them names each one
fig, (levels_panel, changes_panel) = plt.subplots(1, 2, figsize=(10, 4.6))
levels_panel.scatter(series_a, series_b, s=7, color=RED, alpha=0.5)
levels_panel.set_title(f"Levels, r = {levels_correlation:.2f}", fontsize=12)
changes_panel.scatter(changes_a, changes_b, s=7, color=TEAL, alpha=0.5)
changes_panel.set_title(f"Changes, r = {changes_correlation:.2f}", fontsize=12)
for panel in (levels_panel, changes_panel):
panel.set_xlabel("series A")
panel.set_ylabel("series B")
style_chart(fig, panel)
plt.show()levels r = 0.86, changes r = -0.02

The left panel shows levels, and the right panel shows changes.
The third bias is confirmation bias, where you keep changing things in the portfolio until you get a better Sharpe ratio.
In my example, a strategy holds 30 assets, and none of them are predictable. I change the weight of one asset at a time, test again on the same data, and keep the change only if the Sharpe ratio improves. After 1,500 such changes, the Sharpe ratio is 3.79, but on ten years of fresh data, the same strategy has a Sharpe ratio of -0.26.
rng = np.random.default_rng(3)
NUMBER_OF_ASSETS = 30
NUMBER_OF_ADJUSTMENTS = 1500
# Two separate draws from the same process, both centred on zero, so in truth none of the
# thirty assets can be predicted and no set of weights has an edge.
tuned_on_returns = rng.normal(0, 0.01, (3 * D, NUMBER_OF_ASSETS)) # the data we tune on
fresh_returns = rng.normal(0, 0.01, (10 * D, NUMBER_OF_ASSETS)) # data the tuning never sees
# The strategy starts with everything in one asset, chosen at random
weights = np.zeros(NUMBER_OF_ASSETS)
weights[rng.integers(NUMBER_OF_ASSETS)] = 1.0
# The @ operator multiplies the table of daily returns by the vector of weights, which
# leaves one portfolio return per day, and that is what the Sharpe ratio is measured on.
best_sharpe = sharpe(tuned_on_returns @ weights)
tuned_path = []
fresh_path = []
# The loop is the search itself, so it is written out step by step. Each pass adjusts one
# asset at random, keeps the adjustment only when the Sharpe ratio on the tuning data
# improves, and then records the Sharpe ratio of the strategy on both sets of data.
for _ in range(NUMBER_OF_ADJUSTMENTS):
asset_to_change = rng.integers(NUMBER_OF_ASSETS)
trial_weights = weights.copy() # a copy, so a rejected trial leaves nothing behind
trial_weights[asset_to_change] += rng.normal(0, 0.25)
trial_sharpe = sharpe(tuned_on_returns @ trial_weights)
if trial_sharpe > best_sharpe:
weights = trial_weights
best_sharpe = trial_sharpe
tuned_path.append(best_sharpe)
fresh_path.append(sharpe(fresh_returns @ weights))
print(f"tuned Sharpe ratio {tuned_path[-1]:.2f}, fresh Sharpe ratio {fresh_path[-1]:.2f}")
fig, ax = plt.subplots(figsize=(9, 5))
ax.plot(tuned_path, color=RED, lw=2.0, label="the data we tuned on")
ax.plot(fresh_path, color=TEAL, lw=2.0, label="fresh data")
ax.axhline(0, color=GREY, lw=0.8)
ax.set_xlabel("adjustments tried")
ax.set_ylabel("Sharpe ratio")
ax.set_title("Tuned data vs fresh data", fontsize=13)
ax.legend(frameon=False)
style_chart(fig, ax)
plt.show()tuned Sharpe ratio 3.79, fresh Sharpe ratio -0.26

The red line is the Sharpe ratio on the data I tuned on, and the teal line is the Sharpe ratio on fresh data.
Cherry picking is close to confirmation bias. Here you build many versions of a strategy and only report the best one.
I build two hundred versions of a strategy, and none of them has an edge. The best one has a Sharpe ratio of 1.53, while the average across all two hundred is 0.04, and the more versions you test, the higher the best one tends to be.
rng = np.random.default_rng(4)
NUMBER_OF_VERSIONS = 200
# One column is one version of the strategy and one row is one day. Every column comes from
# the same process, centred on zero, so no version has an edge.
version_returns = rng.normal(0, 0.01, (5 * D, NUMBER_OF_VERSIONS))
# Start with an empty list, work along the columns, and collect one Sharpe ratio per version
sharpe_of_each = []
for version in range(NUMBER_OF_VERSIONS):
one_version = version_returns[:, version] # every day of a single column
sharpe_of_each.append(sharpe(one_version))
sharpe_ratios = np.array(sharpe_of_each) # an array, because .max() and .mean() come next
best_sharpe_ratio = sharpe_ratios.max()
average_sharpe_ratio = sharpe_ratios.mean()
print(f"best {best_sharpe_ratio:.2f}, average {average_sharpe_ratio:.2f}")
fig, ax = plt.subplots(figsize=(9, 5))
ax.hist(sharpe_ratios, bins=25, color=TEAL, alpha=0.85)
ax.axvline(best_sharpe_ratio, color=RED, lw=2.4)
# get_ylim reports the bottom and the top of the y-axis, and the label is near the top
bottom_of_chart, top_of_chart = ax.get_ylim()
ax.text(best_sharpe_ratio - 0.04, top_of_chart * 0.9, "the version we report",
ha="right", color=RED, fontsize=11)
ax.axvline(0, color=GREY, lw=0.8)
ax.set_xlabel("Sharpe ratio")
ax.set_ylabel("versions")
ax.set_title("200 versions, no edge", fontsize=13)
style_chart(fig, ax)
plt.show()best 1.53, average 0.04

The histogram shows the two hundred Sharpe ratios, and the red line marks the one you would report.
With regression to the mean, you pick the ones that have performed well, such as mutual fund managers, and expect them to keep doing so.
Say you have a thousand fund managers, and none of them has any skill. The best tenth over the first five years returned 12.3% a year, but over the next five years, the same managers returned -1.5% a year. Their strong record came from luck, so it did not continue.
In real data, a record mixes skill and luck, but luck is still part of why a manager ends up at the top.
rng = np.random.default_rng(5)
NUMBER_OF_MANAGERS = 1000
DAYS_IN_PERIOD = 5 * D # five years of trading days
def yearly_return_over_five_years(daily_returns):
"""Turn five years of daily returns into one average yearly return, in percent.
prod multiplies the daily gross returns down each column, which gives the total growth
of one manager over the whole five years. Raising that total to the power 0.2 takes the
fifth root, which is the average year, and removing the 1 leaves the return.
"""
total_growth = (1 + daily_returns).prod(axis=0)
average_year = total_growth ** 0.2
return (average_year - 1) * 100
# In truth every manager has the same skill, which is none, so the two periods are drawn
# from the same process and the second one is independent of the first.
first_period_daily = rng.normal(0, 0.01, (DAYS_IN_PERIOD, NUMBER_OF_MANAGERS))
next_period_daily = rng.normal(0, 0.01, (DAYS_IN_PERIOD, NUMBER_OF_MANAGERS))
first_period_return = yearly_return_over_five_years(first_period_daily)
next_period_return = yearly_return_over_five_years(next_period_daily)
# percentile finds the level that nine tenths of the managers are below, so the comparison
# marks the top decile on the first period alone.
top_decile_cutoff = np.percentile(first_period_return, 90)
in_top_decile = first_period_return >= top_decile_cutoff
top_decile_first = first_period_return[in_top_decile].mean()
top_decile_next = next_period_return[in_top_decile].mean()
print(f"top decile {top_decile_first:+.1f}% then {top_decile_next:+.1f}%")
fig, ax = plt.subplots(figsize=(9, 5))
for manager in np.where(in_top_decile)[0]: # one grey line per manager in the top decile
ax.plot([0, 1], [first_period_return[manager], next_period_return[manager]],
color=GREY, lw=0.7, alpha=0.5)
ax.plot([0, 1], [top_decile_first, top_decile_next], color=RED, lw=3,
marker="o", ms=8, label="top decile")
ax.plot([0, 1], [first_period_return.mean(), next_period_return.mean()], color=TEAL, lw=2.4,
marker="o", ms=7, label="all managers")
ax.axhline(0, color=GREY, lw=0.8)
ax.set_xticks([0, 1])
ax.set_xticklabels(["first 5 years", "next 5 years"])
ax.set_ylabel("yearly return (%)")
ax.set_title("The top decile, five years on", fontsize=13)
ax.legend(frameon=False)
style_chart(fig, ax)
plt.show()top decile +12.3% then -1.5%

In the chart, each gray line represents one of the best managers, and they spread out across the full range over the next five years.
And the last one is outcome bias, where you only look at the outcome of a strategy and forget how you came to the portfolio or the trading signal.
Say you have a strategy with a true Sharpe ratio of 0.5, so it is worth keeping. If you simulate five thousand first years of this strategy, a third of them lose money, so if you judge it on its first year alone, you would often stop a good strategy.
rng = np.random.default_rng(6)
TRUE_SHARPE = 0.5
STRATEGY_VOLATILITY = 0.01
NUMBER_OF_FIRST_YEARS = 5000
# A Sharpe ratio is the drift divided by the volatility and scaled by the square root of
# the trading days in a year. Running that backwards gives the daily drift a strategy needs
# in order to have a true Sharpe ratio of 0.5.
daily_drift = TRUE_SHARPE * STRATEGY_VOLATILITY / np.sqrt(D)
# Five thousand separate first years, every one of them from this same strategy. One column
# is one year, and prod multiplies its daily gross returns together.
daily_returns_by_year = rng.normal(daily_drift, STRATEGY_VOLATILITY, (D, NUMBER_OF_FIRST_YEARS))
year_growth = (1 + daily_returns_by_year).prod(axis=0)
first_year_return = (year_growth - 1) * 100
ended_down = first_year_return < 0
print(f"{ended_down.mean() * 100:.0f}% of first years end down")
fig, ax = plt.subplots(figsize=(9, 5))
# One set of bin edges for both histograms, so the two colours are on the same scale
bins = np.linspace(first_year_return.min(), first_year_return.max(), 45)
kept_years = first_year_return[~ended_down]
cut_years = first_year_return[ended_down]
ax.hist(kept_years, bins=bins, color=TEAL, alpha=0.85, label="kept")
ax.hist(cut_years, bins=bins, color=RED, alpha=0.85, label="cut")
ax.axvline(0, color=GREY, lw=1.0)
ax.set_xlabel("first-year return (%)")
ax.set_ylabel("cases")
ax.set_title("One year from the same process", fontsize=13)
ax.legend(frameon=False)
style_chart(fig, ax)
plt.show()34% of first years end down

Every year in the chart comes from the same strategy, and the red years are the ones after which you would have stopped.
For each of these biases, there is something you can do. Check that your data include the delisted companies. Ask whether there is an economic reason for the two series to be related, and correlate their changes to remove the trend. Before you test, write down what result would make you drop the idea, count every change and every version you try, and keep a part of the data that none of them has touched. When a strong record turns weak, first ask what luck alone would have given. Keep a journal of what you expect a strategy to return and why, so you can judge it on more than its first year.
So, when you conduct quantitative research, be mindful of these biases, because they can affect your conclusion about whether you should trade on a signal that may be noise or a true edge.
Disclaimer: a teaching example on simulated data. Every number here comes from a market I built myself, where the truth was fixed before any test ran. Not investment advice.
Sources: Rolf Dobelli, The Art of Thinking Clearly, Harper, 2013, which prompted this post. Stephen Brown, William Goetzmann, Roger Ibbotson and Stephen Ross, Survivorship Bias in Performance Studies, Review of Financial Studies (1992). Clive Granger and Paul Newbold, Spurious Regressions in Econometrics, Journal of Econometrics (1974). Peter Wason, On the Failure to Eliminate Hypotheses in a Conceptual Task, Quarterly Journal of Experimental Psychology (1960). Tyler Shumway, The Delisting Bias in CRSP Data, Journal of Finance (1997). David Bailey and Marcos López de Prado, The Deflated Sharpe Ratio, Journal of Portfolio Management (2014). Jonathan Baron and John Hershey, Outcome Bias in Decision Evaluation (1988).