Jobs lifecycle
Initialize job
POST /catchAll/initialize
analyzes your query and returns LLM-generated suggestions for validators,
enrichments, and a date range. It does not create a job or start processing.
This endpoint is a preview — it shows you one example of what validators and
enrichments could look like for your query. Because the suggestions are
LLM-generated, they are not deterministic. If you call initialize and then
submit a job without passing any validators or enrichments, the system generates
them again from scratch and the results will differ from the initialize
response.
To use the initialize suggestions in your job, pass them explicitly in your
submit request. You can use them as-is, modify them, or write your own.
Create job
POST /catchAll/submit creates
the job and starts processing. Only query is required — all other parameters
are optional.
Validators and enrichments are independent. You can define one without the
other — for example, provide custom enrichments and let the system generate
validators, or define validators and let the system generate enrichments.
If you omit both, the system generates both from scratch based on your query.
If you submit without a
limit, the job processes all available data for the
scope, and there is nothing left for Continue job.Get job status
GET /catchAll/status/{job_id}
returns the current processing stage. Poll every 30-60 seconds until status is
completed. Jobs typically take 10-15 minutes.
Jobs progress through these stages in order:
| Stage | Description |
|---|---|
submitted | Job queued, waiting to start |
analyzing | Generating search queries, validators, and enrichments |
fetching | Retrieving web pages |
clustering | Grouping related web pages into event clusters |
enriching | Validating clusters and extracting structured data |
completed | Processing finished, results ready |
failed | Processing failed |
You can pull partial results once the job reaches the
enriching stage
without waiting for completion. See Get job results below.Get job results
GET /catchAll/pull/{job_id}
retrieves structured records. During the enriching stage, progress_validated
tracks how many candidate clusters have been processed so far.
Fields of type
company (like investee_company and investor_company in
the example above) return a structured object with confidence scores for entity
identification and domain resolution. See the Company enrichment data model.Continue job
To process more records after reviewing the initial results of a job submitted with alimit, use POST /catchAll/continue.
It extends the limit without restarting the job — all analysis, validation, and
enrichment logic from the original job is preserved.
Before continuing, check candidate_records and progress_validated in the pull
response to estimate how much data remains. candidate_records is the total number
of event clusters found, progress_validated is how many have been checked so far.
The difference tells you roughly how many candidates are still unprocessed — note
that hit rate is not linear, so the number of new valid records per continuation may vary.
List user jobs
GET /catchAll/jobs/user
returns all jobs created with your API key, sorted by creation date, most recent
first. Supports pagination via page and page_size parameters.