web-extract API
The Web Extract API turns any URL into markdown, text, HTML or structured data as clean JSON.
🤖 Using an AI assistant? Copy this link into ChatGPT / Claude / Cursor — it reads every endpoint and parameter instantly and tells you if this API fits your use case.
The primary scrape endpoint returns a page's final URL, title, metadata (canonical, language, site name, author, publish date, Open Graph) and the extracted content with a confidence signal, and you can map a site's URLs, crawl it, run structured extraction and batch many URLs. It is built for RAG pipelines, AI agents and content tools that need clean, LLM-ready page content without running a headless browser. One ReefAPI key, one shared credit pool, the standard envelope.
Real request and response JSON
Captured from the indexed primary action, scrape, on .
{
"method": "POST",
"url": "https://api.reefapi.com/web-extract/v1/scrape",
"headers": {
"x-api-key": "$REEF_KEY",
"content-type": "application/json"
},
"body": {
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"formats": [
"markdown",
"metadata"
]
}
}{
"ok": true,
"meta": {
"api": "web-extract",
"endpoint": "scrape",
"mode": "live",
"latency_ms": 1076.4,
"record_count": 1,
"bytes": 231204,
"cache_hit": false,
"method": "bare_dc_curl_cffi+trafilatura",
"browserless": true,
"ssrf_guarded": true,
"final_url": "https://en.wikipedia.org/wiki/Web_scraping",
"formats": [
"markdown",
"metadata"
],
"extraction_method": "trafilatura",
"confidence": "high"
},
"data": {
"final_url": "https://en.wikipedia.org/wiki/Web_scraping",
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"title": "Web scraping - Wikipedia",
"metadata": {
"title": "Web scraping - Wikipedia",
"description": null,
"canonical": "https://en.wikipedia.org/wiki/Web_scraping",
"lang": "en",
"site_name": "Wikimedia Foundation, Inc.",
"author": "Contributors to Wikimedia projects",
"published_at": "[redacted-phone]T18:57:30Z",
"modified_at": null,
"section": null,
"keywords": [],
"favicon": "https://en.wikipedia.org/static/apple-touch/wikipedia.png",
"og": {
"title": "Web scraping - Wikipedia",
"type": "website"
},
"twitter": {},
"hreflang": [],
"feeds": [
"https://en.wikipedia.org/w/index.php?title=Special:RecentChanges&feed=atom"
],
"status_code": 200,
"content_type": "text/html; charset=UTF-8",
"robots_meta": "max-image-preview:standard"
},
"extraction": {
"method": "trafilatura",
"rendered": false,
"confidence": "high",
"content_chars": 25663
},
"markdown": "|  | \n\n**Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](https://en.wikipedia.org/wiki/Data_extraction) from [websites](https://en.wikipedia.org/wiki/Website).\n\n[World Wide Web](https://en.wikipedia.org/wiki/World_Wide_Web)using the\n\n[Hypertext Transfer Protocol](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol)or a web browser. While web scraping can be done manually "
}
}What the web-extract API does
| Action | Description | Concrete use case | Key params |
|---|---|---|---|
| scrape | Ops teams call scrape to pull structured data. | none | |
| map | Developer tools call map to pull structured data. | none | |
| crawl | Validation workflows call crawl to pull structured data. | none | |
| extract | Data-quality teams call extract to pull structured data. | none | |
| batch | Ops teams call batch to pull structured data. | none |
Call scrape from your stack
curl -X POST https://api.reefapi.com/web-extract/v1/scrape \
-H "x-api-key: $REEF_KEY" \
-H "content-type: application/json" \
-d '{"url":"https://en.wikipedia.org/wiki/Web_scraping","formats":["markdown","metadata"]}'import requests
r = requests.post(
"https://api.reefapi.com/web-extract/v1/scrape",
headers={"x-api-key": REEF_KEY},
json={
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"formats": [
"markdown",
"metadata"
]
},
)
print(r.json()["data"])const res = await fetch("https://api.reefapi.com/web-extract/v1/scrape", {
method: "POST",
headers: {
"x-api-key": process.env.REEF_KEY,
"content-type": "application/json",
},
body: JSON.stringify({
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"formats": [
"markdown",
"metadata"
]
}),
});
const { ok, data, meta, error } = await res.json();Ask your MCP-connected assistant: call reefapi.web-extract.scrape with {"url":"https://en.wikipedia.org/wiki/Web_scraping","formats":["markdown","metadata"]}.Who uses this API and why
- RAG pipelines call scrape to turn a URL into clean markdown for an AI knowledge base.
- AI agents use extract to pull structured fields from a page without custom parsers.
- Content tools use map and crawl to discover and ingest an entire site's pages.
Questions developers ask before integrating
What is the web-extract API?
web-extract API is a ReefAPI endpoint group for web-extract It returns live JSON through POST requests under /web-extract/v1.
Is the web-extract API free to try?
Yes. ReefAPI starts with 1,000 free credits, no card required. web-extract calls use the same shared credit balance as every other ReefAPI engine.
Do I need a web-extract login or account?
No login to web-extract is needed for the API response. You call ReefAPI with your x-api-key header, and the playground can run live examples before you create a production key.
How fresh is the web-extract data?
The page example is captured from a live scrape call, and production requests fetch live data through ReefAPI rather than a static sample.
How many credits does the web-extract API use?
web-extract actions currently cost 1-2 credits per successful call. Failed or blocked calls are free, and all APIs draw from one credit pool.
Can I call web-extract from an AI assistant or MCP client?
Yes. Connect ReefAPI once through MCP and your assistant can call web-extract actions with the same key, credit pool and JSON envelope used by normal REST requests.
Is the web-extract API a web-extract scraper?
It is the managed alternative to a DIY web-extract scraper. Instead of building and maintaining your own scraper — proxies, headless browsers, captcha and constant breakage — you call one ReefAPI endpoint and get the same web-extract back as clean JSON.
Why does my web-extract scraper keep getting blocked?
Most web-extract scrapers break on anti-bot defenses, rate limits and IP bans that need rotating residential proxies and browser fingerprinting to clear. ReefAPI handles all of that for you — no proxies, no captchas, no maintenance — and returns live JSON. Blocked or failed calls are free.