How do I get market data into Python?
Price data is essential for testing a trading idea, and getting that data is a separate part of the whole process. In my case, I use three sources, and which one I reach for depends on what I am testing.
This page says what each source gives you and when it fits, and each links to a post with the code. If you do not have paid access, there is a free option at the end.
What are you testing?
The source follows from the question.
One stock, daily closes. Almost anything works, including the free route below.
A whole market, over many years. Now you need the companies that no longer exist. A universe of every stock in the index today, tested back to 2000, holds only the survivors, and survivors beat the market by construction. This is survivorship bias. Few sources give you the delisted names.
Trades and quotes, intraday. Now you need a tick vendor. The data is thousands of times larger, and the download has to survive being interrupted.
Interactive Brokers
If you have an IBKR account, you have a data feed. ib_async talks to the Trader Workstation app, and one call returns bars straight into pandas.
Connecting is not the same as having the data. What a request returns depends on the subscriptions on your account. You can connect and still get nothing back for an instrument you are not entitled to.
I use it for single names when the account is already open.
LSEG Workspace
Workspace is the route for a whole market. One call returns the full universe for a country, active and delisted. The delisted names are the point. They are what keep the sample free of survivorship bias.
I use it whenever the test covers more than a handful of stocks over more than a few years.
LSEG Tick History
Tick History gives trades and quotes rather than daily bars. The API is not the hard part. The volume is. A single month of one active stock runs to millions of rows, requests fail partway through, and a download that cannot resume wastes hours.
I use it for microstructure work, where the daily close is too coarse to answer the question.
If you do not have a paid feed
yfinance costs nothing, needs no account, and returns a DataFrame in one line. For learning the mechanics or checking a single index, it is enough.
It serves what exists now. Build a universe from it and test back twenty years, and every company that went bankrupt or was bought is missing. The result looks good and means nothing. Use it to learn, not to test a strategy across a market.
Once the prices are in
Getting the file is the smaller half. These lessons cover the rest, in order:
- Load a CSV file, and parse the dates, which is where most price files go wrong
- The Series and the DataFrame
- Returns with pct_change, and why you compound them rather than add them
- The same calculation per ticker, so it does not bleed across names
- Join two tables, to put prices next to fundamentals
- A panel, date by ticker, the shape most strategy code wants
Then lag the signal. A signal built from today’s close cannot be traded at today’s close, and getting that wrong is what makes a bad backtest look good.
So which one?
Start with yfinance and one ticker, so you are debugging your own code and not a login. Move to IBKR for broker data. Move to Workspace the moment the question covers a whole market. There, survivorship bias is not a footnote. It is the result.
Working in R? R for Investing covers the same ground, and fread() reads a large price file far faster than read.csv().