News Firehose
Raw news data at scale
3M+ articles/day from 140,000+ sources, with 5–10 minute latency on priority sources. Live feed from day one, with historical backfill to 2019.
Delivery
How you receive the data
We push data into your existing infrastructure. Take the full feed or filter by country, language, source type, or topic before it arrives.
Data dumps
Written directly to your S3-compatible bucket — AWS, GCS, Azure Blob, or your own. No polling.
Streaming
Articles pushed directly into your message queue as they're crawled. Connect to whichever queue you already run.
REST API
Query the full index on demand. Boolean search, date ranges, language, country, and source filters. Paginate or poll on a schedule.
The data
What's included
Every article arrives with timestamps, source metadata, and NLP enrichment already applied. No additional processing needed on your end.
Every article arrives with timestamps, source metadata, and NLP enrichment already applied. No additional processing needed on your end.
Latency & SLAs
Predictable delivery, per source
Default
5–15 min
Standard crawl cadence across all indexed sources.
Priority tier
5–10 min
Contracted SLA on a defined source list. We verify current latency per source before you sign.
Custom sources
2–5 days
New sources onboarded within 2–5 business days. Crawl latency agreed upfront based on site complexity.
Use cases
What teams build with it
Common questions
Things we get asked on every enterprise call.
Things we get asked on every enterprise call.
What happens to the price if our usage grows?
Nothing. The firehose is a flat monthly fee with no per-article or per-call limits. Usage can grow 10x or 20x and the price doesn't move. That's the typical reason teams move from the News API to the Firehose: they outgrow API quotas and want unlimited, push-based delivery at a fixed cost.
Do I get a full firehose or search-filtered results?
Both are available. The firehose gives you every article across all 140,000+ sources as it's crawled. You can also filter by country, language, source type, or Boolean query to receive only the slice you need. Most enterprise customers take the full feed and apply their own filters downstream.
How much data volume should I expect?
Approximately 3 million articles per day across all sources. We can give you a precise estimate for your target source subset before you sign.
Can I start with a subset of countries and expand?
Yes. The schema is identical regardless of geographic scope. Start with a few countries to validate your pipeline, then expand to the full global feed. No integration changes required.
How do I get lower latency on specific sources?
Send us your priority source list and we'll run a check showing current crawl latency per domain. Many high-priority sources are already on 5–10 minute cycles. For sources that aren't, we agree a contracted SLA and add them to your priority tier.
Is historical data included?
The firehose is a live feed. It delivers articles as they're crawled from the point you go live. Historical backfill to 2019 (over 2 billion articles) is available on request as a separate delivery. NLP enrichment (entities, sentiment) is available from July 2023 onwards; earlier articles can be enriched on request.
Can I add sources that aren't in your index?
Yes. Standard source onboarding takes 2–5 business days. We analyse your list and quote based on source complexity and required crawl frequency.
Does filtering change the price?
No. You can filter the feed by country, language, source type, or topic to receive only what you need. The fee stays flat regardless of the volume you take. Most customers start narrow and expand as their pipeline matures.
Unlimited, push-based delivery at a fixed cost.
Flat monthly fee. Latency SLAs.
