Monitors automate data collection by running CatchAll jobs on a schedule. Create
a monitor from a successful job, define when to run it, and receive webhook
notifications when new results are available.
How it works Each scheduled execution:
Creates a new job with its own job_id.
Uses a rolling date window based on schedule frequency.
Applies the reference job’s validators, extractors, and schema.
Deduplicates records across all runs and merges the results.
Reference job records are not included in monitor results. Only records
from scheduled executions appear in aggregated results.
Before you start First, create a successful job following the
Quickstart guide .
Create a monitor Once your reference job completes with status: job_completed, use job_id to
create a monitor:
cURL
JSON
Python
TypeScript
Java
curl -X POST "https://catchall.newscatcherapi.com/catchAll/monitors/create" \ -H "x-api-key: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "reference_job_id": "295b95d8-6041-4f4b-b132-9f009fc6af70", "schedule": "every day at 12 PM UTC", "webhook": { "url": "https://your-endpoint.com/catchall", "method": "POST" } }'
Response:
{ "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" , "status" : "Monitor Created Successfully" }
The webhook parameter is optional. For webhook payload structure and
authentication options, see the Webhooks section. Retrieve results List all monitors Get all monitors for your API key:
curl "https://catchall.newscatcherapi.com/catchAll/monitors" \ -H "x-api-key: YOUR_API_KEY"
{ "total_monitors" : 2 , "monitors" : [ { "monitor_id" : "0bcbf554-1f38-460e-9f6d-4bb9338560a4" , "reference_job_id" : "2acada6a-3e55-423d-9406-00f5c6dc73da" , "reference_job_query" : "Cross-border mergers and acquisitions involving US companies" , "enabled" : false , "schedule" : "every day at 12 PM UTC" , "timezone" : "UTC" , "created_at" : "2025-11-04T21:04:22Z" }, { "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" , "reference_job_id" : "295b95d8-6041-4f4b-b132-9f009fc6af70" , "reference_job_query" : "AI company acquisitions and mergers" , "enabled" : true , "schedule" : "every 5 minutes" , "timezone" : "UTC" , "created_at" : "2025-11-07T10:54:42Z" } ] }
Key fields:
enabled (boolean): Whether the monitor is currently activereference_job_query (string): Original query text from the reference jobschedule (string): Natural language schedule descriptioncreated_at (string): Monitor creation timestamp List monitor jobs Get execution history for a specific monitor:
curl "https://catchall.newscatcherapi.com/catchAll/monitors/{monitor_id}/jobs" \ -H "x-api-key: YOUR_API_KEY"
{ "monitor_id" : "3fec5b07-8786-86d7-9486-d43ff67eccd4" , "sort_order" : "asc" , "total_jobs" : 2 , "jobs" : [ { "job_id" : "8a9763dd-3611-4b3b-a6cf-3e893a0c6746" , "start_date" : "2025-11-07T10:50:00Z" , "end_date" : "2025-11-07T10:55:00Z" }, { "job_id" : "288387df-7e05-4722-83cc-ecbebb6d8123" , "start_date" : "2025-11-07T10:55:00Z" , "end_date" : "2025-11-07T11:00:00Z" } ] }
Use the job_id to retrieve detailed results for a specific execution via
/catchAll/pull/{job_id}. Get aggregated results Retrieve all deduplicated records from all monitor jobs:
curl "https://catchall.newscatcherapi.com/catchAll/monitors/pull/{monitor_id}" \ -H "x-api-key: YOUR_API_KEY"
{ "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" , "cron_expression" : "*/5 * * * *" , "timezone" : "UTC" , "reference_job" : { "query" : "AI company acquisitions and mergers" , "context" : "Focus on technology companies, include deal size if mentioned" }, "run_info" : { "first_run" : "2025-11-07T10:50:00Z" , "last_run" : "2025-11-07T11:30:00Z" }, "records" : 2 , "status" : "Done" , "all_records" : [ { "record_id" : "6417909601438475967" , "record_title" : "Samsung Electronics Acquires Oxford Semantic Technologies" , "added_on" : "2025-11-07T10:55:00Z" , "enrichment" : { "acquiring_company" : "Samsung Electronics" , "acquired_company" : "Oxford Semantic Technologies" , "acquisition_date" : "2024" , "deal_type" : "acquisition" , "ai_technology_focus" : [ "knowledge graphs" , "personalized AI" ], "confidence" : "high" }, "citations" : [ { "title" : "Samsung invests in AI and M&A" , "link" : "https://n.news.naver.com/mnews/article/629/0000441563" , "published_date" : "2025-11-07T11:15:11Z" , "job_id" : "c4cb35e9-c8a5-46bc-87aa-5fdbf36f8e33" , "added_on" : "2025-11-07T10:55:00Z" } ], "added_on" : "2025-11-07T10:55:00Z" , "updated_on" : "2025-11-07T10:55:00Z" } ] }
Key fields:
cron_expression (string): Parsed cron format from your natural language
schedulereference_job (object): Original query and context from the reference jobrun_info (object): Execution timeframe showing first and last monitor runsall_records (array): Deduplicated records from all monitor executions Manage monitors Enable a monitor Resume execution of a disabled monitor:
curl -X POST "https://catchall.newscatcherapi.com/catchAll/monitors/{monitor_id}/enable" \ -H "x-api-key: YOUR_API_KEY"
Response:
{ "success" : true , "message" : "Monitor enabled successfully." , "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" }
Disable a monitor Pause execution without deleting the monitor:
curl -X POST "https://catchall.newscatcherapi.com/catchAll/monitors/{monitor_id}/disable" \ -H "x-api-key: YOUR_API_KEY"
Response:
{ "success" : true , "message" : "Monitor disabled successfully." , "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" }
Webhooks Receive HTTP POST notifications when monitor jobs complete. Add a webhook
object when creating the monitor (shown in
Create a monitor ).
Webhook payload Show Full payload structure
When a monitor job completes, your endpoint receives: { "monitor_id" : "3fec5b07-8786-46d7-9486-d43ff67eccd4" , "reference_job_id" : "295b95d8-6041-4f4b-b132-9f009fc6af70" , "latest_job_id" : "e2dd78b0-3189-48f3-9011-eef0ecd9aced" , "records_count" : 2 , "jobs_processed" : 3 , "updated_at" : "2025-11-07T11:25:33.877039" , "cron_expression" : "*/5 * * * *" , "timezone" : "UTC" , "records" : [ { "record_id" : "-39216465981207817" , "record_title" : "Homeplus Seeks Buyer, Receives LOI from AI Firm Harex Infotech" , "added_on" : "2025-11-07T11:25:33Z" , "enrichment" : { "schema_based_summary" : "Harex Infotech acquired Homeplus Co. for null on null" , "acquisition_date" : null , "deal_type" : "acquisition" , "deal_status" : "in talks" , "confidence" : "high" , "acquired_company" : "Homeplus Co." , "acquiring_company" : "Harex Infotech" }, "citations" : [ { "title" : "Court extends Homeplus' rehabilitation plan submission deadline" , "link" : "https://n.news.naver.com/mnews/article/009/0005586304" , "published_date" : "2025-11-07T11:21:21Z" , "job_id" : "e2dd78b0-3189-48f3-9011-eef0ecd9aced" , "added_on" : "2025-11-07T11:25:33Z" } ] } ] }
Key fields:
latest_job_id: Use to retrieve full job results via
/catchAll/pull/{job_id}records_count: Number of new records in this execution (after deduplication)jobs_processed: Total number of jobs executed by this monitorrecords: Array of new records from the latest jobrecords[].added_on: Timestamp when this record was first collectedrecords[].citations[].added_on: Timestamp when this citation was added Webhooks fire on every job completion, even when records_count is 0 (no new
events after deduplication).
Authentication Show Authentication examples
Bearer token: { "webhook" : { "url" : "https://api.example.com/catchall" , "method" : "POST" , "headers" : { "Authorization" : "Bearer YOUR_TOKEN" , "Content-Type" : "application/json" } } }
Basic authentication: { "webhook" : { "url" : "https://your-endpoint.com/webhook" , "method" : "POST" , "auth" : [ "username" , "password" ] } }
Custom headers: { "webhook" : { "url" : "https://your-endpoint.com/webhook" , "method" : "POST" , "headers" : { "X-Custom-Header" : "value" , "X-API-Key" : "your-key" , "Content-Type" : "application/json" } } }
See also 
