Firm Insider Date Shares Year Month
0 Company A Alexander 2015-03-15 1000 2015 3
1 Company A Alexander 2016-03-15 1000 2016 3
2 Company A Alexander 2017-03-15 1000 2017 3
3 Company A Alexander 2018-03-15 1000 2018 3
4 Company A Alexander 2019-03-15 1000 2019 3
5 Company A Alexander 2020-03-15 1000 2020 3
6 Company A Alexander 2021-03-15 1000 2021 3
7 Company A Alexander 2022-03-15 1000 2022 3
8 Company A Alexander 2023-03-15 1000 2023 3
9 Company A Alexander 2024-03-15 1000 2024 3
10 Company B Alexander 2015-05-10 200 2015 5
11 Company B Alexander 2016-08-10 200 2016 8Classify insiders as routine or opportunistic
Code
Python
Tutorial
How to label an insider routine or opportunistic in Python, from the calendar months of their trades over the three previous years.
Corporate insiders, such as officers and directors, can have material information about their firm, information that would move the share price and that the market does not have. Insiders also trade for reasons unrelated to the firm’s prospects: taxes, diversification, the need for cash and the vesting of share grants. Those reasons tend to repeat in the same calendar month every year.
Cohen, Malloy and Pomorski (2012), CMP from here, use that timing to classify insiders, assuming that trades with predictable timing are unlikely to be driven by information. An insider who traded in the same calendar month in each of three consecutive years is routine, and stays routine in every later year. Any other insider who traded in each of the three prior years is opportunistic. CMP find that all the ability of insider trades to predict returns comes from the opportunistic trades, while the routine trades predict an abnormal return, a return beyond what the market explains, of close to zero. CMP call the rule “a noisy proxy for actual routine trading”. The labels describe when insiders trade, not why a particular insider traded.
This post builds a simpler version of the classifier in Python on made-up trades: it labels each insider every year from the three prior years alone and does not carry a Routine label forward. So a routine insider becomes Opportunistic in a year whose three prior years share no month, and NonClassified when one of those years has no trade, where CMP would keep the Routine label. The four made-up insiders below get the same labels under both. The post does not replicate CMP’s sample, transaction filters or return tests.
Defining the rule
At the start of each calendar year Y, the code looks at the insider’s trades in years Y-3, Y-2, Y-1:
- Routine, traded in all three prior years, and at least one calendar month appears in all three.
- Opportunistic, traded in all three prior years, and no calendar month appears in all three.
- NonClassified, no trade in at least one of the three prior years.
With \(M_y\) the set of calendar months in which the insider traded in year \(y\), the months shared across the three prior years are
\[M_{Y-3} \cap M_{Y-2} \cap M_{Y-1},\]
where \(\cap\) keeps the months that are in every set, and the Routine label needs this set to hold at least one month.
Every trade the insider makes in year Y inherits that label. The classification is at the insider level, pooled across firms, as in the paper.
Step 1. Simulating the data
I build made-up trades by four insiders at three firms over ten years, so the label each insider should get is known in advance and I can check the classifier.
The first made-up insider is me, Alexander: an insider at Company A, buying 1,000 shares every March. I am also an insider at Company B, where I trade once a year in a different month each year. The rule pools my trades across firms, so March appears in each of my prior three years, and the rule labels me Routine. That label also applies to my Company B trades, which have no monthly pattern of their own.
Beatrice (Company B, different month every year) → Opportunistic. Cecilia (Company C, only odd years) → NonClassified. Diana (one trade in 2015, then no further trades) → NonClassified too.
NoteCode: the trades
import pandas as pd
# One row is one trade: the firm, the insider, the date, and the size in shares.
trade_rows = []
# Alexander, me, buys at Company A in March of every year. Same month every year, so the
# rule should call me Routine.
for year in range(2015, 2025):
trade_rows.append(('Company A', 'Alexander', f'{year}-03-15', 1000))
# I am an insider at Company B as well, where I trade once a year in a scattered month.
# These trades have no pattern of their own. zip walks the years and the months side by
# side, one pair per trade.
alexander_company_b_months = [5, 8, 11, 2, 6, 10, 1, 4, 7, 12]
for year, month in zip(range(2015, 2025), alexander_company_b_months):
trade_rows.append(('Company B', 'Alexander', f'{year}-{month:02d}-10', 200))
# Beatrice trades once a year at Company B and never repeats a calendar month, so no month
# can be shared across three years and the rule should label her Opportunistic.
beatrice_months = [6, 8, 11, 2, 9, 4, 7, 10, 1, 5]
for year, month in zip(range(2015, 2025), beatrice_months):
trade_rows.append(('Company B', 'Beatrice', f'{year}-{month:02d}-10', 800))
# Cecilia always trades in June, but only in odd years, so she never has three years of
# history in a row and the rule labels her NonClassified.
for year in range(2015, 2025, 2):
trade_rows.append(('Company C', 'Cecilia', f'{year}-06-20', 500))
# Diana trades once and then stops, so she never has three years of history.
trade_rows.append(('Company A', 'Diana', '2015-04-10', 300))
trades = pd.DataFrame(trade_rows, columns=['Firm', 'Insider', 'Date', 'Shares'])
# The dates arrive as text, and to_datetime turns them into real dates, so the .dt
# accessor below can read them. The rule reads calendar years and calendar months, so each
# one gets its own column here, and every later step reads them directly.
trades['Date'] = pd.to_datetime(trades['Date'])
trades['Year'] = trades['Date'].dt.year
trades['Month'] = trades['Date'].dt.month
# to_string() writes the whole table as text, so print shows it in a script as well
print(trades.head(12).to_string())Step 2. Classifying each insider-year
The rule needs two facts first: the calendar months an insider traded in during the three years before year Y, and the months among those that appear in every one of those years. For me in 2018, the three prior years are 2015 to 2017. Pooled across my two firms, I traded in March and May in 2015, March and August in 2016, and March and November in 2017. So March appears in every one of those years, and all my 2018 trades are Routine under the insider-level rule. Two small helpers compute these facts, so the rule below contains only the decision it makes.
NoteCode: the two helpers
def months_traded_by_year(insider, year):
"""The calendar months this insider traded in, one set of months per year, over the
three calendar years before the given year.
Asking about Alexander in 2018 looks at 2015, 2016 and 2017, and answers March and
May for 2015, March and August for 2016, and March and November for 2017.
"""
is_this_insider = trades['Insider'] == insider
is_in_lookback = trades['Year'].between(year - 3, year - 1) # both ends included
# The paper classifies at the insider level, so trades at every firm count together.
# .loc keeps the rows where the True/False condition is True.
lookback_trades = trades.loc[is_this_insider & is_in_lookback]
# groupby splits the trades into one group per year, and agg(set) collapses each group
# down to the months it contains. A month traded twice in one year still counts once.
return lookback_trades.groupby('Year')['Month'].agg(set)
def months_traded_in_every_year(months_by_year):
"""The calendar months that appear in every year of a months-by-year table.
Given March and May, then March and August, then March and November, the answer is
March on its own. An empty answer means no calendar month was repeated every year.
"""
one_set_per_year = list(months_by_year)
shared_months = one_set_per_year[0]
for months_of_one_year in one_set_per_year[1:]:
shared_months = shared_months & months_of_one_year # & keeps months in both sets
return shared_months
NoteCode: the label of each insider-year
def classify_insider_year(insider, year):
"""A simpler version of the paper's rule, applied to one insider in one year.
Routine if some calendar month appears in all three prior years, Opportunistic if
three years of history share no month, NonClassified if a year of history is missing.
"""
months_by_year = months_traded_by_year(insider, year)
# The rule returns NonClassified for an insider who skipped one of the three prior years.
if len(months_by_year) < 3:
return 'NonClassified'
shared_months = months_traded_in_every_year(months_by_year)
if shared_months:
return 'Routine'
return 'Opportunistic'
# The first three years of the sample cannot be classified, because each classification
# year needs three earlier years of history before it.
classification_years = range(trades['Year'].min() + 3, trades['Year'].max() + 1)
label_rows = []
for insider in trades['Insider'].unique():
for year in classification_years:
label = classify_insider_year(insider, year)
label_rows.append((insider, year, label))
insider_year_class = pd.DataFrame(label_rows, columns=['Insider', 'Year', 'Class'])
# pivot reshapes the long table into one row per insider and one column per year.
print(insider_year_class.pivot(index='Insider', columns='Year', values='Class').to_string())Year 2018 2019 2020 2021 2022 2023 2024
Insider
Alexander Routine Routine Routine Routine Routine Routine Routine
Beatrice Opportunistic Opportunistic Opportunistic Opportunistic Opportunistic Opportunistic Opportunistic
Cecilia NonClassified NonClassified NonClassified NonClassified NonClassified NonClassified NonClassified
Diana NonClassified NonClassified NonClassified NonClassified NonClassified NonClassified NonClassifiedThe table shows the labels I built the data to produce. My month history, pooled across firms, contains March in every year, so I am Routine from 2018. Beatrice has no repeat month, so she is Opportunistic. Cecilia and Diana never have three consecutive years of trading, so they are NonClassified.
The per-year groupby doing the work here is covered in Lesson 22, and the insider-year panel it builds on in Lesson 25.
Step 3. Labelling every trade
A merge attaches each insider’s label for the year to every trade that insider made in that year.
NoteCode: the label of every trade
# A left merge keeps every trade and attaches the label its insider had that year.
labelled_trades = trades.merge(insider_year_class, on=['Insider', 'Year'], how='left')
# Trades in the first three years matched no label, so the merge left Class empty for them.
# Empty here means too little history, and NonClassified is the label for that case.
labelled_trades = labelled_trades.fillna({'Class': 'NonClassified'})
is_alexander = labelled_trades['Insider'] == 'Alexander'
# .loc[rows, columns] keeps the rows marked True and the columns listed
print(labelled_trades.loc[is_alexander, ['Firm', 'Date', 'Shares', 'Class']].to_string()) Firm Date Shares Class
0 Company A 2015-03-15 1000 NonClassified
1 Company A 2016-03-15 1000 NonClassified
2 Company A 2017-03-15 1000 NonClassified
3 Company A 2018-03-15 1000 Routine
4 Company A 2019-03-15 1000 Routine
5 Company A 2020-03-15 1000 Routine
6 Company A 2021-03-15 1000 Routine
7 Company A 2022-03-15 1000 Routine
8 Company A 2023-03-15 1000 Routine
9 Company A 2024-03-15 1000 Routine
10 Company B 2015-05-10 200 NonClassified
11 Company B 2016-08-10 200 NonClassified
12 Company B 2017-11-10 200 NonClassified
13 Company B 2018-02-10 200 Routine
14 Company B 2019-06-10 200 Routine
15 Company B 2020-10-10 200 Routine
16 Company B 2021-01-10 200 Routine
17 Company B 2022-04-10 200 Routine
18 Company B 2023-07-10 200 Routine
19 Company B 2024-12-10 200 RoutineMy Company B trades from 2018 on are Routine, although none of them is in March, because each trade takes its insider’s label for that year. My trades from 2015 to 2017 are NonClassified, because the classifier needs three earlier years of history. The paper’s Table III describes a trade-level variant that labels my Company B trades from 2018 on by their own month. The chart below draws every trade of the four insiders at its year and calendar month, in the colour of its label.
NoteCode: the chart
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
RED, TEAL, GREY = '#C0392B', '#17868A', '#888888'
# The site's chart colours, in order: the cream background, the text, the light grid lines,
# the pale frame and the axis labels, the same on every chart of the site.
BG, INK, GRID, SPINE, AXIS_TEXT = '#FCEFE3', '#1f1f1f', '#EADCCC', '#D5C6B4', '#4a4a4a'
# One colour per label
CLASS_COLOURS = {'Routine': TEAL, 'Opportunistic': RED, 'NonClassified': GREY}
def style_chart(figure, axis):
"""Give a finished chart the site's cream background, light grid and pale frame."""
figure.patch.set_facecolor(BG)
axis.set_facecolor(BG)
axis.grid(True, axis='y', color=GRID, lw=1.0)
axis.set_axisbelow(True)
for side in ('top', 'right'):
axis.spines[side].set_visible(False)
for side in ('left', 'bottom'):
axis.spines[side].set_color(SPINE)
axis.tick_params(axis='both', length=0, colors=INK, pad=6)
axis.xaxis.label.set_color(AXIS_TEXT)
axis.yaxis.label.set_color(AXIS_TEXT)
axis.title.set_color(INK)
# plt.subplots(2, 2) creates the figure with four charts in two rows, one per insider. figsize
# is the figure's width and height in inches, and sharex and sharey give the four charts the
# same axes. axes.flat goes through the four charts row by row.
figure, axes = plt.subplots(2, 2, figsize=(10, 6.4), sharex=True, sharey=True)
for axis, insider in zip(axes.flat, trades['Insider'].unique()):
insider_trades = labelled_trades.loc[labelled_trades['Insider'] == insider]
# One scatter per label, so each trade takes the colour of its label; s is the dot's size
for label, colour in CLASS_COLOURS.items():
trades_with_label = insider_trades.loc[insider_trades['Class'] == label]
axis.scatter(trades_with_label['Year'], trades_with_label['Month'], s=55, color=colour)
axis.set_title(insider)
style_chart(figure, axis)
# The axes are shared, so setting the ticks on one chart sets them on all four: a month every
# quarter, and every third year from 2015
axes[0, 0].set_yticks([3, 6, 9, 12], ['Mar', 'Jun', 'Sep', 'Dec'])
axes[0, 0].set_xticks([2015, 2018, 2021, 2024])
for axis in axes[1]: # axes[1] is the bottom row
axis.set_xlabel('Year')
for axis in axes[:, 0]: # axes[:, 0] is the left column
axis.set_ylabel('Calendar month')
# One legend for the whole figure. Line2D([], []) draws nothing on a chart, so it only gives the
# legend a dot in each label's colour; ls='' leaves out the line through the dot.
legend_dots = []
for label, colour in CLASS_COLOURS.items():
legend_dots.append(Line2D([], [], marker='o', ls='', color=colour, label=label))
# frameon=False draws the legend without a box, ncol=3 puts the three labels in one row, and
# bbox_to_anchor places the legend's top centre at the middle of the figure's top edge
legend = figure.legend(handles=legend_dots, frameon=False, ncol=3, loc='upper center',
bbox_to_anchor=(0.5, 1.0))
for text, colour in zip(legend.get_texts(), CLASS_COLOURS.values()):
text.set_color(colour)
figure.suptitle('Every trade by year and calendar month', color=INK, y=1.04)
# tight_layout() fits the labels inside the figure. savefig writes the chart to a PNG file:
# dpi=140 sets its resolution, bbox_inches='tight' trims the empty margin, and facecolor=BG
# keeps the cream background. show() displays the chart.
plt.tight_layout()
plt.savefig('insider_trade_calendar.png', dpi=140, bbox_inches='tight', facecolor=BG)
plt.show()
Each dot is one trade, at its year across and its calendar month up, in the colour of its insider’s label that year. In my panel, the Company A trades form a row of dots at March in every year. From 2018 all my dots are teal, including the Company B trades in other months, because each of the three years before contains a March trade. Beatrice’s dots never repeat a month, so from 2018 her dots are red. Cecilia’s dots appear only in odd years, and Diana has a single dot, so all their dots are grey. So an insider is Routine in a year when one calendar month has a dot in each of the three years before it.
Using the classifier on real data
To apply this to a real dataset such as LSEG Workspace:
- Filter to open-market buys and sells only. Drop
Exercise/Conversion of Awards,Payment of Exercise Price or Tax Liability,Giftand private transactions. CMP study only open-market purchases and sales, so these transactions are outside their sample, even though the timing of some, such as a gift, can be a choice. - Reconcile insider identities across firms. LSEG sometimes reports the same person with different spellings at different firms. Without reconciliation, a multi-firm insider like me is split into separate records, the rule classifies each record on part of the history, and a record on its own may lack the three years the rule needs.
- Make sure you have at least three years of history before the first classification year. CMP’s 1989 start required data back to 1986. If your data starts in 2000, your first classification year is 2003.
Conclusion
The takeaway is that the routine label needs only the calendar months of an insider’s trades in the three previous years, pooled across firms: a month shared by all three years makes the insider routine. The label describes when an insider trades, not why.
Disclaimer: a teaching example on made-up trades. Not investment advice.
Reference
Cohen, L., Malloy, C., and Pomorski, L. (2012). Decoding Inside Information. Journal of Finance 67(3): 1009–1043.