An open source version of Stripe's Sigma: Analytics and archival of Stripe events. Backfills from existing events, and receives new updates through Stripe webhooks.
- Accept webhooks directly from Stripe, or relay thrm through your API
- Different storage providers for string data (many derivative indexes, materialized views, etc. provided) (prefer ClickHouse or Tinybird)
- Backfill specific data types from Stripe API
Stripe deletes most data from events after 30 days. This is not a lot of time, especially if you need to check something like a checkout session's metadata.
They go from looking like this:
To looking like this:
From 97 lines of data down to 3. Ouch.
They also have a product, Sigma, to query your Stripe data, but that has a per-transaction cost along with an increasing base cost, minimum $10. You also need to enable this before they collect data, so any objects >30 days old are also lost.
This solution allows you to get most of this functionality for free in a database you control.
The system is inherently limited to only backfilling for object types that can be listed without an ID. For example application fee refunds are not backfilled because they require an :id in the url path.
If you are already accepting all webhooks, then the /direct endpoint lets you send the JSON body from your existing service that already received a stripe webhook. This basically just avoids the signature check.
An optional DUCKDB_EXTENSIONS env var exists that allows you to install custom extensions.
By default, the json extension is installed. You can set DUCKDB_EXTENSIONS to a CSV of additional extensions to have them install on Init().