The three sources that matter: the official Betfair Exchange API for live and your own recorded data, Betfair’s free Historical Data archive for backtesting, and third-party feeds (Betfair Databowl-style services, odds aggregators) for convenience. Most serious work uses the API for live capture plus the historical archive for testing. Start with the API guide; you’ll need an app key.
This page contains affiliate links — if you open an account through them we may earn a commission at no cost to you. It never changes our verdict.
- Why Data Source Choice Matters
- 1. The Official Betfair Exchange API
- 2. Betfair Historical Data Archive
- 3. Third-Party Feeds and Aggregators
- Rolling Your Own: Capturing Live Data
- Data Formats and the Gotchas
- Sources Compared
- From the Desk: What 90 Days of Captured Data Told Me
- What It All Costs
- How to Start This Week
- Cleaning, Storing and Joining Your Data
- Three Data Pitfalls That Cost Real Money
This is a sub of our Betfair data and analytics pillar. Before you can build a model, backtest a system, or even honestly answer "does my strategy actually work?", you need clean price data — and where that data comes from shapes everything downstream. Get the source wrong and you backtest on numbers that never existed at the moment you'd have traded. So this is the map: every realistic source, ranked, with the trade-offs spelled out.
Why Data Source Choice Matters
There is a world of difference between "the odds were 3.4" and "at 14:29:58 the best available back price for £50 was 3.4 with £180 queued at 3.45". Strategy testing lives in the second world. The headline last-traded price hides the order book, the available liquidity, and the exact timing — and those are precisely what determine whether your hypothetical trade would have filled. Your data source decides which of those details you can see.
Get this wrong and you produce a beautiful backtest of a strategy that could never have been executed, then lose real money discovering the gap. So the question is never just "where's the cheapest data" — it's "which source preserves the detail my strategy depends on." For a scalping system that means tick-level order-book depth; for a spreadsheet review of your own results, last-traded prices are plenty.
1. The Official Betfair Exchange API
The official API is the source of truth and the one serious traders build on. It gives you live market catalogues, full order-book prices (back and lay at multiple depths), traded volumes, and the ability to place and manage bets programmatically. Everything Bet Angel and Geeks Toy show you comes through this same API; using it directly just means you control the pipeline.
To use it you register an application, generate an app key, and authenticate (typically with a certificate for non-interactive login). After that you call endpoints like listMarketCatalogue and listMarketBook for data, and the betting endpoints to trade. There's a free tier (a "delayed" app key for development) and a live key for real-money operation. The full walkthrough is in our Betfair API guide, and if you want to build automated tools, see building bots. This is the route I recommend for anyone capturing their own data — nothing else gives you the same fidelity at zero data cost.
2. Betfair Historical Data Archive
Betfair publishes its own Historical Data service — recorded market data going back years, sold as downloadable files at different detail tiers (a free "Basic" plan with limited granularity, and paid "Advanced/Pro" tiers with full tick-level stream data). This is the single most valuable resource for backtesting, because it is Betfair's own recording of what the market actually did, tick by tick, on real events you never traded.
The premium tiers contain the full stream — every price change and matched bet — which lets you reconstruct the exact order book at any millisecond of a past race or match. That's the gold standard for testing an in-play or pre-race system honestly. The detail covered in where to get historical data. The catch is volume: full tick data for a busy day is enormous, so you'll be working with compressed archives and writing parsers — which is where the format gotchas below bite.
3. Third-Party Feeds and Aggregators
A layer of commercial services repackages Betfair (and other-book) data into friendlier formats: cleaned CSVs, odds-comparison feeds, results databases, and ready-made datasets for specific sports. These trade money and some fidelity for convenience — you skip building parsers and get analysis-ready tables, but you inherit whatever the provider chose to record and however they timestamped it.
They're genuinely useful for two cases: quick research where you don't need millisecond precision, and sports where someone has already done the tedious work of joining odds to results. They're a poor choice if your edge depends on order-book microstructure the feed doesn't capture, or if you can't verify exactly how the provider sourced and timestamped the numbers. Use them for breadth and speed; fall back to the API and historical archive for anything your money depends on.
Rolling Your Own: Capturing Live Data
The most underrated source is the one you build: a small script that subscribes to the API's streaming endpoint and writes every price update for the markets you care about to your own store. Run it on a cheap always-on machine or a small cloud instance, and within weeks you have a bespoke dataset of exactly the markets, sports and times you trade — recorded the way you need it.
This is what I eventually did and it changed how I work. The Python tutorial shows the betfairlightweight library that makes streaming capture straightforward, and the data analysis guide covers turning the captured stream into research tables. The advantage over buying historical data is total relevance: you record your exact markets at your exact decision points, including the ones no commercial dataset bothers with.
Data Formats and the Gotchas
Three gotchas catch everyone the first time. Timestamps: know whether a price is stamped at publish time or capture time, and what timezone — an hour's drift silently invalidates any time-of-day analysis. Last-traded vs available: the last-traded price is not the price you could have got; for execution testing you need the available back/lay at your stake, not the headline number. Liquidity: a price with £8 available is not the same opportunity as the same price with £800 available, and many simplified feeds drop the available-volume detail entirely.
The Betfair historical files are JSON-stream based and large; you'll parse them with a streaming reader rather than loading whole files into memory. Plan storage early — full tick data accumulates into hundreds of gigabytes quickly. Decide upfront which markets and which depth you actually need, because "capture everything" becomes unmanageable faster than you expect.
Sources Compared
| Source | Fidelity | Cost | Best for |
|---|---|---|---|
| Exchange API (live) | Full order book, real time | Free (app key) | Live trading, self-capture |
| Betfair Historical (Pro) | Full tick stream, archived | Paid tiers | Honest backtesting |
| Betfair Historical (Basic) | Reduced granularity | Free | Light historical research |
| Third-party feeds | Cleaned, variable detail | Subscription | Convenience, breadth |
| Self-captured stream | Exactly what you record | Free + hosting | Bespoke, relevant datasets |
From the Desk: What 90 Days of Captured Data Told Me
The build: I ran a betfairlightweight streaming script on a £4/month cloud instance, recording the full order book for every UK and Irish win market from 10 minutes before the off through to the result, for 90 days — roughly 4,200 races into a compressed store.
What I tested: my long-held belief that the 2.0-ish second-favourite "always" firms in the last three minutes. The captured data said: of 4,180 usable races, the second favourite shortened in the final three minutes in 54% of cases — barely better than a coin flip, not the near-certainty my gut claimed.
The trade implication: a blind "back the second fav at T-3min, lay at the off" system on that sample returned −2.1% after 5% commission across 4,180 races. My intuition was a story I'd built from the races I remembered, not the ones I didn't. Without my own captured data I'd never have known.
The lesson: the value of owning your data isn't the data — it's killing your own bad ideas cheaply, before they cost you real money. That 90-day capture cost me about £12 in hosting and saved me a great deal more in stakes I now never placed.
What It All Costs
The headline is that the most powerful combination is nearly free. The Exchange API costs nothing beyond a one-off small charge some regions apply for the app key. Self-capture costs a few pounds a month in hosting. Betfair's Basic historical tier is free; the Pro tier with full tick data costs a monthly subscription that's modest if you genuinely backtest and wasteful if you don't. Third-party feeds range from cheap CSV packs to serious monthly subscriptions for live aggregated data.
My honest spend recommendation: start with the free API app key and self-capture, which costs almost nothing and teaches you the most. Add Betfair's Pro historical tier only once you have a specific strategy that needs more history than you can capture going forward. Pay for third-party feeds last, and only for breadth you can't reasonably build yourself. Don't buy data you won't analyse — an unused subscription is the most common waste in this whole area.
How to Start This Week
Concretely: register a Betfair developer application and generate a delayed app key today (free, no money at risk). Work through the API guide to authenticate and pull your first listMarketBook response. Then follow the Python tutorial to capture a single market's stream to a file. Within an evening you'll have real, full-fidelity Betfair data on your own disk — the foundation everything else builds on.
From there, scale capture to the markets you actually trade, pull a slice of Betfair's free historical archive to compare against, and only then decide whether any paid tier earns its place. Pair the data with a disciplined trading journal so your live results and your historical tests speak the same language, and you'll be researching on numbers you trust rather than odds someone else cleaned for reasons you can't see.
Cleaning, Storing and Joining Your Data
Raw data is never analysis-ready, and the work of cleaning it is where most projects quietly die. Three jobs always need doing. First, parsing — turning the API responses or historical stream files into tidy rows you can query, which for the Betfair historical archive means a streaming JSON reader rather than loading whole files. Second, normalising — making sure odds, volumes and timestamps are in consistent units and timezones across every source so they can be compared. Third, joining — linking price data to results, which is the step that lets you ask "did this signal actually predict anything?" and is often the hardest part because results live in a different place from prices.
For storage, start simpler than you think you need. A few months of self-captured markets fit comfortably in a local SQLite database or even well-organised CSV/Parquet files; you do not need a cloud data warehouse to test a strategy. Reach for a proper database only when query speed on a large dataset becomes the bottleneck. The common mistake is over-engineering the storage before you've proven there's an edge worth storing for. Get to a first answer cheaply, then scale the plumbing if the answer justifies it. The data visualisation piece covers turning a clean store into dashboards.
A realistic first project: capture one sport's win markets for a month, join the closing prices to the results, and measure one simple thing — say, the strike rate and ROI of blindly backing the favourite at the off. It's an unglamorous question with a known-ish answer, which is exactly why it's a good first project: you'll learn whether your pipeline produces numbers you can trust by checking them against reality. Only once you trust the pipeline should you point it at the question you actually care about. Build the boring thing first; it's the foundation every more interesting analysis stands on, and it's how you avoid the trap of analysing dirty data and believing the output.
Three Data Pitfalls That Cost Real Money
Beyond sourcing, three analysis pitfalls catch traders who do get clean data, and each can turn a losing idea into one that looks like a winner. The first is survivorship and selection in your own sample — testing a strategy only on the markets you happened to capture, or only on the sport you follow, then assuming the result generalises. A signal that works on liquid UK racing may collapse on thin evening cards, and a backtest that quietly excludes the hard markets flatters itself. Capture broadly and test on everything, not just the data that was convenient to record.
The second is look-ahead bias — accidentally using information in your test that wouldn't have been available at the moment you'd have traded. The classic version is testing against the last-traded price when you'd actually have had to take the available price, or using the closing line to grade a bet you'd have placed an hour earlier. Your data has to be timestamped precisely enough to reconstruct exactly what you could have seen and done at the decision point, which is why full-fidelity sources beat cleaned summaries for execution testing.
The third is overfitting — tuning a strategy until it fits your historical sample perfectly, at which point it has learned the noise rather than a real edge and falls apart live. The defences are familiar from any quantitative discipline: keep the rules simple, test on data the strategy was never tuned on, and be suspicious of any result that looks too good. Clean data from the right source removes the excuses; honest method removes the self-deception. Get both right and your research finally tells you the truth, which — as the captured-data example above showed — is often that your favourite idea doesn't work, saving you the cost of finding out with real stakes.
FAQ
Is the Betfair API free to use?
The API itself is free to call once you have an application key. Some regions apply a small one-off charge to activate a live app key, and a free delayed key is available for development. There is no per-call data fee — you only pay normal commission on bets you actually place.
Where can I get historical Betfair data?
Betfair sells its own Historical Data service with downloadable archived market files. There is a free Basic tier with reduced granularity and paid Advanced/Pro tiers with full tick-level stream data. The Pro tier is the gold standard for honest backtesting because it is Betfair’s own recording of real markets.
What is the best way to collect my own Betfair data?
Run a streaming-capture script using the API and a library like betfairlightweight on a cheap always-on machine, recording the exact markets you trade. Within weeks you build a bespoke dataset recorded at full fidelity for a few pounds a month in hosting — more relevant than any bought dataset.
Do I need full order-book data or are last-traded prices enough?
It depends on the strategy. For reviewing your own results or broad research, last-traded prices suffice. For testing whether a trade would actually have filled — especially scalping or in-play — you need the available back/lay prices at your stake and the queued liquidity, not just the headline price.
Are third-party Betfair data feeds worth paying for?
Sometimes. They save you building parsers and give analysis-ready tables, useful for breadth and quick research. But you inherit whatever the provider recorded and however they timestamped it. For anything your money depends on, verify against the official API or historical archive rather than trusting a feed blindly.
Related Reading
Stay in the cluster: data pillar, historical data, Python analysis, Excel templates. Build with it: API guide, building bots, data analysis guide, glossary.