News Articles API & Scraper
The News Article API extracts a clean article from any news URL as clean JSON.
🤖 Using an AI assistant? Copy this link into ChatGPT / Claude / Cursor — it reads every endpoint and parameter instantly and tells you if this API fits your use case.
The primary extract endpoint returns the article's title, text, author, date, top image, images, language, site name and canonical URL, and you can also run a news_search. It is built for RAG pipelines, media monitoring and reader apps that need clean article content without a headless browser. One ReefAPI key, one shared credit pool, the standard envelope.
What each news_search source gives you, and what lands in the url field
news_search speaks to four different backends behind one schema, and the differences that bite live in two fields: what `url` points at, and what `source` is named after. Google News hands back its own redirect links unless you ask for them to be resolved. The rows below are from live calls made in one sitting.
| source | Requires | What `url` holds | What `source` is named after |
|---|---|---|---|
| query | A news.google.com/rss/articles/CBMi... redirect, unless resolve=true | The publisher's display name: "The New York Times", "CNBC" | |
| bing | query | The publisher URL directly | The hostname: "msn.com" |
| gdelt | query | The publisher URL directly | The hostname: "tickerreport.com", "e24.no" |
| rss | rss_url; query is ignored | The link from the feed, tracking params included | The feed's hostname: "bbc.co.uk" |
| all | query | Mixed, depending on which source contributed the row | Mixed; meta.sources reports the per-source counts |
published and date are ISO 8601 with an explicit offset ("2026-08-26T11:06:51+00:00"), while extract's article.date is a bare calendar date ("2005-09-17") with no time at all. limit is 1-100 and is clamped rather than rejected. An unknown source is rejected outright: source="yahoo" returned INVALID_PARAM with detail.allowed listing the five valid values.
Real request and response JSON
Captured from the indexed primary action, extract, on .
{
"method": "POST",
"url": "https://api.reefapi.com/news-article/v1/extract",
"headers": {
"x-api-key": "$REEF_KEY",
"content-type": "application/json"
},
"body": {
"url": "https://en.wikipedia.org/wiki/Web_scraping"
}
}{
"ok": true,
"meta": {
"api": "news-article",
"endpoint": "extract",
"mode": "live",
"latency_ms": 1161.2,
"record_count": 1,
"bytes": 231114,
"cache_hit": false,
"completeness_pct": 85.71,
"stop_reason": "bare",
"requests": 1,
"rendered": false,
"resolved_from": null
},
"data": {
"article": {
"url": "https://en.wikipedia.org/wiki/Web_scraping",
"canonical": "https://en.wikipedia.org/wiki/Web_scraping",
"title": "Web scraping - Wikipedia",
"author": "Contributors to Wikimedia projects",
"date": "[redacted-phone]",
"text": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. It is a form of copying in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis.\nScraping a web page involves fetching it and then e",
"excerpt": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. Web scraping software may directly access the World Wide Web using the Hypertext Transfer Protocol or a web browser. While web scraping can be done manually by a software…",
"top_image": null,
"images": [
"https://upload.wikimedia.org/wikipedia/en/thumb/9/99/Question_book-new.svg/60px-Question_book-new.svg.png",
"https://upload.wikimedia.org/wikipedia/en/thumb/b/b4/Ambox_important.svg/40px-Ambox_important.svg.png",
"https://upload.wikimedia.org/wikipedia/commons/thumb/b/bd/Ambox_globe_content.svg/60px-Ambox_globe_content.svg.png"
],
"language": "en",
"site_name": "Wikimedia Foundation, Inc.",
"favicon": "https://en.wikipedia.org/static/apple-touch/wikipedia.png",
"section": null,
"keywords": [],
"word_count": 4094,
"reading_time_min": 19,
"content_markdown": "|  | \n\n**Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](https://en.wikipedia.org/wiki/Data_scraping) used for [extracting data](https://en.wikipedia.org/wiki/Data_extraction) from [websites](https://en.wikipedia.org/wiki/Website).\n\n[World Wide Web](https://en.wikipedia.org/wiki/World_Wide_Web)using the\n\n[Hypertext Transfer Protocol](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol)or a web browser. While web scraping can be done manually "
}
}
}What the News Articles API does
| Action | Description | Concrete use case | Key params |
|---|---|---|---|
| extract | Extract a clean article from any news or blog URL: returns title, full text, author, publication date, top image, all images, language, site name, and canonical URL. Automatically follows Google News and Bing News redirect links to the original publisher page. | Content platforms call extract to extract a clean article from any news or blog URL. | url, render, include_markdown |
| news_search | recent-news list from source=google|bing|gdelt|rss|all (unified schema title/url/source/published/lang); resolve=true → Google-News URLs become real publisher URLs | Research tools call news_search to get recent-news list from source=google|bing|gdelt|rss|all (unified schema title/url/source/publi…. | query, source, lang, country, limit, ... |
Call extract from your stack
curl -X POST https://api.reefapi.com/news-article/v1/extract \
-H "x-api-key: $REEF_KEY" \
-H "content-type: application/json" \
-d '{"url":"https://en.wikipedia.org/wiki/Web_scraping"}'import requests
r = requests.post(
"https://api.reefapi.com/news-article/v1/extract",
headers={"x-api-key": REEF_KEY},
json={
"url": "https://en.wikipedia.org/wiki/Web_scraping"
},
)
print(r.json()["data"])const res = await fetch("https://api.reefapi.com/news-article/v1/extract", {
method: "POST",
headers: {
"x-api-key": process.env.REEF_KEY,
"content-type": "application/json",
},
body: JSON.stringify({
"url": "https://en.wikipedia.org/wiki/Web_scraping"
}),
});
const { ok, data, meta, error } = await res.json();Ask your MCP-connected assistant: call reefapi.news-article.extract with {"url":"https://en.wikipedia.org/wiki/Web_scraping"}.Who uses this API and why
- RAG pipelines call extract to turn a news URL into clean text for an LLM.
- Media-monitoring tools use news_search then extract to track coverage of a topic.
- Reader apps use extract to strip a page down to its article body.
Questions developers ask before integrating
Does extract return the whole article body or a summary?
The whole body. A measured extract on the Wikipedia "Web scraping" page returned article.text at 26,559 characters with word_count 4242 and reading_time_min 19, plus a separate short `excerpt` for previews. By default you also get content_markdown, which was 43,882 characters for the same page because it preserves the inline links. Set include_markdown=false and the content_markdown key is dropped from the response entirely rather than returned as null.
What happens on a paywalled article?
You get a clean failure, not a stub. A live extract against a WSJ article returned ok:false with error code UPSTREAM_HTTP and the message ending "http 401", after three attempts recorded in meta.requests. A dead URL behaves the same way with code NOT_FOUND and "http 404". In both cases every article field is absent rather than half-filled, so an ok:true response means real extracted text and not a login wall rendered as prose.
How reliable are the author and date fields?
They come from the page's own metadata, so they are as good as the publisher's markup and no better. Across measured calls an AP News page filled both author and top_image, a BBC section page returned author null and top_image null, and Wikipedia returned author "Contributors to Wikimedia projects". Section and index pages are the worst case, because the date you get back is whatever the template stamped rather than an article date. Treat both fields as optional and check for null.
Why do my Google News results have news.google.com URLs instead of the publisher's?
That is what the Google News RSS feed publishes, and the API returns it untouched by default. Pass resolve=true and the URLs are followed to the real publisher pages: a measured "nvidia earnings" search came back with investors.com, cnbc.com and bloomberg.com links, meta.resolved 3, and meta.requests climbing from 1 to 7. It costs extra upstream requests and time, so leave it off if you only need headlines. Passing a Google News link to extract resolves it too, and reports the original in meta.resolved_from.
Does lang actually filter the results?
For google it selects the edition, but for gdelt it behaves more like a hint. A measured gdelt search for "nvidia" with the default lang=en returned a Norwegian result from e24.no tagged lang "no" alongside the English ones. Every result carries its own lang field, so filter on that yourself if your pipeline is language-sensitive rather than trusting the request parameter to have done it.
Is site_name the publication or the domain?
Whichever the page declares, which means it is not consistent across publishers. Measured values in one sitting: "AP News", "bbc.com" and "Wikimedia Foundation, Inc." - a brand name, a bare hostname and a legal entity. It reads og:site_name where present and falls back to the host. If you need a stable publisher key, derive it from article.canonical rather than from site_name.
Does the render parameter do anything?
No. It is accepted for compatibility, but this build ships without a browser, so meta.rendered comes back false regardless of what you pass, including render="force". Every measured extract reported meta.stop_reason "bare". Recon found that rendering recovered close to nothing on real news pages, which is why it was left out. A JavaScript-only page will return thin text rather than silently pretending it rendered.
What does source=rss expect, and does query still apply?
It expects rss_url and ignores query. A measured call with rss_url pointing at the BBC news feed returned results in the same unified schema, with meta.query null and source set to "bbc.co.uk". Omitting query for a source that needs one is rejected up front: source=google with no query returned MISSING_PARAM with the message "`query` is required for source='google'".
What is the News Articles API?
News Articles API is a ReefAPI endpoint group for extract clean article text from any news url. It returns live JSON through POST requests under /news-article/v1.
Is the News Articles API free to try?
Yes. ReefAPI starts with 1,000 free credits, no card required. News Articles calls use the same shared credit balance as every other ReefAPI engine.
Do I need a News Articles login or account?
No login to News Articles is needed for the API response. You call ReefAPI with your x-api-key header, and the playground can run live examples before you create a production key.
How fresh is the News Articles data?
The page example is captured from a live extract call, and production requests fetch live data through ReefAPI rather than a static sample.
How many credits does the News Articles API use?
News Articles actions currently cost 1-2 credits per successful call. Failed or blocked calls are free, and all APIs draw from one credit pool.
Can I call News Articles from an AI assistant or MCP client?
Yes. Connect ReefAPI once through MCP and your assistant can call news-article actions with the same key, credit pool and JSON envelope used by normal REST requests.