Resumable Polymarket indexer: CLOB market metadata plus on-chain trades from Polygon, in one DuckDB file you can query with SQL
# PolyLedger
A resumable indexer for Polymarket market metadata and on-chain trade data, backed by DuckDB.
PolyLedger pulls every market from the Polymarket CLOB API, streams every OrderFilled event from Polygon via [Envio HyperSync](https://envio.dev), and writes both into a single DuckDB file you can query with SQL immediately.
## Table of contents
## Features
order_fills is keyed on (transaction_hash, log_index) and inserted with ON CONFLICT DO NOTHING. Re-running a range is a no-op.OrderFilled layout and the older V1 one, with separate checkpoints per stream.Retry-After support, and a shared token-bucket rate limiter across all REST sources.## Requirements
The CLOB and Gamma APIs are public and need no credentials.
## Installation
git clone https://github.com/nahrek/polyledger cd polyledger python -m venv .venv .venv\Scripts\activate.bat pip install -e .
Then configure the HyperSync token. Copy .env.example to .env, open it in any
editor, and set HYPERSYNC_BEARER_TOKEN. PolyLedger reads .env from the working
directory on startup, so there is no export step. Real environment variables still
take precedence if you prefer to set them that way.
## Quick start
Run the full pipeline:
polyledger sync
The first backfill is slow. On the free HyperSync tier it takes anywhere from a few hours to a couple of days depending on how much history you want. It is safe to interrupt at any point. Every subsequent run is incremental and finishes in seconds.
To verify the setup on a small slice first:
polyledger markets # metadata only, a few minutes polyledger chain --max-blocks 200000 # roughly five days of Polygon history polyledger stats
stats reports what is actually in the database:
markets 14231
tokens 28455
fills 1204773
unmatched_fills 392
first_block 75014820
last_block 75214820
first_time 2026-05-14 08:11:23
last_time 2026-05-19 03:44:02
checkpoints:
order_filled:v2: block 75214820, 1204773 rows, updated 2026-09-02 15:11:23Then query it:
polyledger query "SELECT question, sum(usd_size) AS volume FROM trades GROUP BY 1 ORDER BY 2 DESC LIMIT 10"
## CLI reference
| Command | Description |
|---|---|
polyledger markets [--backfill] | Sync market metadata from the CLOB. --backfill also resolves unmatched token ids via Gamma. |
polyledger chain [options] | Index OrderFilled logs from Polygon. |
polyledger trades [--drop-unmatched] | Materialise the joined trades_mat table. |
polyledger sync [options] | Run markets → chain → backfill → trades in order. |
polyledger stats | Show row counts, block range, and checkpoints. |
polyledger export [--out DIR] | Write every table to Parquet. |
polyledger query "SELECT ..." | Run one SQL statement and print the result. |
chain options:
| Flag | Description | ||
|---|---|---|---|
| `--contracts v2\ | v1\ | all` | Which exchange generation to index. Default v2. |
--from-block N | Override the start block. Ignored if a checkpoint exists. | ||
--to-block N | Stop at this block. | ||
--max-blocks N | Index at most this many blocks, then exit. |
Global flags: --db PATH to use a different database file, -v for verbose logging.
The polyledger executable is installed by pip install -e .. Without installing, python -m polyledger and python -m polyledger.cli are equivalent. The entry point is main() in polyledger/cli.py.
## Data model
### trades