# Web Archive API — historical website snapshots from the Wayback Machine and Common Crawl: capture index, closest snapshot, first/last-seen history and domain-wide captures

> All Wayback captures for a URL (timestamp, status, mimetype, digest, archival size, snapshot_url), date/status/mime filtered, collapsible, RESUME-KEY PAGINATED.
> ReefAPI engine `web-archive` · 7 endpoints · clean JSON, no scraping or browsers to manage.

## How to call
- **Endpoint:** `POST https://api.reefapi.com/web-archive/v1/<action>` with a JSON body.
- **Auth:** header `x-api-key: <YOUR_REEFAPI_KEY>` — create one free (1,000 credits, no card): https://reefapi.com/signup
- **Response (every call):** `{ ok: boolean, data: ..., meta: { record_count, credits, ... }, error: { code, message } }` — branch on `ok`. Failed or blocked calls are free.
- **One key + one shared credit pool** across every ReefAPI API. Per-call credits are listed on each endpoint below.
- **Use it from an AI agent (MCP):** connect `https://api.reefapi.com/mcp` (remote streamable-http, `Authorization: Bearer <key>`) and your assistant can call these actions directly.

## Endpoints

### POST /web-archive/v1/snapshots — 1 credit
All Wayback captures for a URL (timestamp, status, mimetype, digest, archival size, snapshot_url), date/status/mime filtered, collapsible, RESUME-KEY PAGINATED.

**Parameters:**
- `url` (string, required) — The URL, host, or path to look up in the archive (e.g. 'github.com', 'github.com/torvalds', 'https://example.com/page'). Scheme optional. Use with match=prefix/domain/host to widen.
- `from` (string, optional) — Earliest capture to include. ISO date (YYYY-MM-DD) or a Wayback timestamp (YYYYMMDDhhmmss, any 4-14 digit prefix). Inclusive.
- `to` (string, optional) — Latest capture to include. ISO date or Wayback timestamp prefix. Inclusive.
- `limit` (integer, optional, default 100) — Max captures to return (1-1000, default 100). Page further with meta.resume_key on the snapshots action.
- `collapse` (enum, optional) — Deduplicate consecutive captures on a field (CDX 'collapse'). collapse=digest is the change-detection workhorse. [one of: digest, timestamp:4, timestamp:6, timestamp:8, timestamp:10, urlkey, original, statuscode, mimetype]
- `match` (enum, optional, default "exact") — How to match url: exact | prefix | host | domain. domain_captures forces 'domain'. [one of: exact, prefix, host, domain]
- `status` (string, optional) — Keep only captures with this HTTP status (e.g. 200, 404, 301). Prefix with '!' to exclude (e.g. '!200').
- `mime` (string, optional) — Keep only captures of this MIME type (e.g. text/html, application/pdf, image/png). Prefix with '!' to exclude.
- `filter` (string, optional) — Advanced raw CDX filter expression(s), comma-separated. Format [!]field:regex over urlkey/timestamp/original/mimetype/statuscode/digest/length (e.g. 'original:.*\.pdf$'). Power-user escape hatch.
- `resume_key` (string, optional) — Opaque pagination token from meta.resume_key of the previous page; returns the next page of captures.

**Returns:** snapshots[] (timestamp, datetime, original, mimetype, statuscode, digest, length, snapshot_url); meta.record_count + meta.resume_key/has_more

**Example request body:**
```json
{
  "url": "github.com",
  "limit": 10,
  "collapse": "digest"
}
```

### POST /web-archive/v1/available — 1 credit
Closest single Wayback snapshot to a given date (or the latest). Fast existence check.

**Parameters:**
- `url` (string, required) — The URL, host, or path to look up in the archive (e.g. 'github.com', 'github.com/torvalds', 'https://example.com/page'). Scheme optional. Use with match=prefix/domain/host to widen.
- `timestamp` (string, optional) — Target date for the closest snapshot. ISO date or Wayback timestamp prefix. Omitted → the most recent capture.

**Returns:** available (bool), closest{timestamp, datetime, status, snapshot_url}, url

**Example request body:**
```json
{
  "url": "github.com",
  "timestamp": "20120601"
}
```

### POST /web-archive/v1/history — 1 credit
Lifespan + capture cadence for a URL: EXACT first_seen/last_seen + span, plus per-year capture counts and status/mime breakdown — the SEO/due-diligence summary.

**Parameters:**
- `url` (string, required) — The URL, host, or path to look up in the archive (e.g. 'github.com', 'github.com/torvalds', 'https://example.com/page'). Scheme optional. Use with match=prefix/domain/host to widen.
- `from` (string, optional) — Earliest capture to include. ISO date (YYYY-MM-DD) or a Wayback timestamp (YYYYMMDDhhmmss, any 4-14 digit prefix). Inclusive.
- `to` (string, optional) — Latest capture to include. ISO date or Wayback timestamp prefix. Inclusive.
- `match` (enum, optional, default "exact") — How to match url: exact | prefix | host | domain. domain_captures forces 'domain'. [one of: exact, prefix, host, domain]

**Returns:** first_seen + last_seen (EXACT, pinpoint), first/last_snapshot_url, span_days, years{year:count}, status_codes{code:count}, mime_types{type:count}, sampled_capture_days + sample_capped (years/breakdown are a capture-per-day sample capped at 1000 — first/last_seen are NOT sampled)

**Example request body:**
```json
{
  "url": "github.com"
}
```

### POST /web-archive/v1/domain_captures — 1 credit
All archived URLs under a domain (the domain + its subdomains), one row per unique URL — the 'every page this site ever had' view. Resume-key paginated.

**Parameters:**
- `url` (string, required) — The URL, host, or path to look up in the archive (e.g. 'github.com', 'github.com/torvalds', 'https://example.com/page'). Scheme optional. Use with match=prefix/domain/host to widen.
- `from` (string, optional) — Earliest capture to include. ISO date (YYYY-MM-DD) or a Wayback timestamp (YYYYMMDDhhmmss, any 4-14 digit prefix). Inclusive.
- `to` (string, optional) — Latest capture to include. ISO date or Wayback timestamp prefix. Inclusive.
- `limit` (integer, optional, default 100) — Max captures to return (1-1000, default 100). Page further with meta.resume_key on the snapshots action.
- `status` (string, optional) — Keep only captures with this HTTP status (e.g. 200, 404, 301). Prefix with '!' to exclude (e.g. '!200').
- `mime` (string, optional) — Keep only captures of this MIME type (e.g. text/html, application/pdf, image/png). Prefix with '!' to exclude.
- `filter` (string, optional) — Advanced raw CDX filter expression(s), comma-separated. Format [!]field:regex over urlkey/timestamp/original/mimetype/statuscode/digest/length (e.g. 'original:.*\.pdf$'). Power-user escape hatch.
- `resume_key` (string, optional) — Pagination token from the previous page's meta.resume_key.

**Returns:** urls[] (original, urlkey, first/last timestamp+datetime, captures, statuscode, mimetype, snapshot_url); meta.record_count + resume_key/has_more

**Example request body:**
```json
{
  "url": "github.com",
  "limit": 20
}
```

### POST /web-archive/v1/batch — 1 credit
Closest-snapshot existence + capture-count for up to 20 URLs in one call (bulk archival presence — competitor/portfolio sweeps).

**Parameters:**
- `urls` (array, required) — Up to 20 URLs/hosts to look up at once (JSON array or comma/newline-separated string).
- `timestamp` (string, optional) — Target date for the closest snapshot. ISO date or Wayback timestamp prefix. Omitted → the most recent capture.

**Returns:** results[] (url, archived(bool), closest{timestamp,datetime,snapshot_url,status}); meta.record_count = URLs processed

**Example request body:**
```json
{
  "urls": [
    "github.com",
    "python.org"
  ]
}
```

### POST /web-archive/v1/cc_search — 1 credit
Common Crawl index lookup for a URL — alternate/broader coverage with richer fields (detected language, encoding, WARC offset/filename). Cross-source corroboration.

**Parameters:**
- `url` (string, required) — The URL, host, or path to look up in the archive (e.g. 'github.com', 'github.com/torvalds', 'https://example.com/page'). Scheme optional. Use with match=prefix/domain/host to widen.
- `collection` (string, optional) — Common Crawl index id (e.g. 'CC-MAIN-2026-21'). Omit to use the newest index. List ids via action='cc_indexes'.
- `match` (enum, optional, default "exact") — URL match mode for the Common Crawl index query. [one of: exact, prefix, host, domain]
- `limit` (integer, optional, default 100) — Max captures to return (1-1000, default 100). Page further with meta.resume_key on the snapshots action.
- `from` (string, optional) — Earliest capture to include. ISO date (YYYY-MM-DD) or a Wayback timestamp (YYYYMMDDhhmmss, any 4-14 digit prefix). Inclusive.
- `to` (string, optional) — Latest capture to include. ISO date or Wayback timestamp prefix. Inclusive.
- `filter` (string, optional) — Advanced raw CDX filter expression(s), comma-separated. Format [!]field:regex over urlkey/timestamp/original/mimetype/statuscode/digest/length (e.g. 'original:.*\.pdf$'). Power-user escape hatch.

**Returns:** records[] (timestamp, datetime, original, mimetype, statuscode, digest, length, languages, encoding, warc_filename, warc_offset); meta.collection

**Example request body:**
```json
{
  "url": "github.com",
  "limit": 5
}
```

### POST /web-archive/v1/cc_indexes — 1 credit
List the available Common Crawl monthly indexes (id, name, date range) — pick a collection for cc_search.

**Parameters:** none

**Returns:** indexes[] (id, name, from, to, cdx_api); meta.record_count = index count

## More
- Try it live, no code: https://reefapi.com/playground?engine=web-archive
- Human docs page: https://reefapi.com/docs/web-archive
- Every ReefAPI API in one file (for your AI): https://reefapi.com/llms-full.txt