> ## Documentation Index
> Fetch the complete documentation index at: https://newscatcherinc-docs.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CatchAll API quickstart guide

> Get started with CatchAll API.

CatchAll is a web search API that generates unique datasets from 2+ billion web
pages. This guide walks you through your first API call.

## Before you start

Before you begin, make sure you meet these prerequisites:

* CatchAll API key (obtain from
  [platform.newscatcherapi.com](https://platform.newscatcherapi.com))
* Basic understanding of REST APIs
* Your preferred programming language and HTTP client
* Basic knowledge of JSON data format

## Steps

<Steps>
  <Step title="Set up environment">
    Install the CatchAll client library for your programming language:

    <CodeGroup>
      ```bash curl theme={null}
      # cURL is typically included in your system
      # To check, open the terminal and type the following:
      curl --version
      ```

      ```bash Python theme={null}
      pip install newscatcher-catchall-sdk
      ```

      ```bash TypeScript theme={null}
      npm install newscatcher-catchall-sdk
      ```

      ```bash Java (Gradle) theme={null}
      dependencies {
          implementation 'com.newscatcherapi:newscatcher-catchall-sdk:3.0.0'
      }
      ```

      ```bash Java (Maven) theme={null}
      <dependency>
          <groupId>com.newscatcherapi</groupId>
          <artifactId>newscatcher-catchall-sdk</artifactId>
          <version>3.0.0</version>
      </dependency>
      ```
    </CodeGroup>
  </Step>

  <Step title="Create job">
    Submit a query to create a new processing job:

    <CodeGroup>
      ```bash curl theme={null}
      curl -X POST "https://catchall.newscatcherapi.com/catchAll/submit" \
        -H "x-api-key: YOUR_API_KEY_HERE" \
        -H "Content-Type: application/json" \
        -d '{
          "query": "AI company acquisitions",
          "context": "Focus on deal size and acquiring company details",
          "start_date": "2025-12-03T19:00:40Z",
          "end_date": "2025-12-08T19:00:40Z",
          "limit": 10
        }'
      ```

      ```json JSON theme={null}
      {
        "query": "AI company acquisitions",
        "context": "Focus on deal size and acquiring company details",
        "start_date": "2025-12-03T19:00:40Z",
        "end_date": "2025-12-08T19:00:40Z",
        "limit": 10
      }
      ```

      ```python Python theme={null}
      from newscatcher_catchall import CatchAllApi

      # Initialize the client
      client = CatchAllApi(api_key="YOUR_API_KEY_HERE")

      # Create a job
      job = client.jobs.create_job(
          query="AI company acquisitions",
          context="Focus on deal size and acquiring company details",
          start_date="2025-12-03T19:00:40Z",
          end_date="2025-12-08T19:00:40Z",
          limit=10
      )

      job_id = job.job_id
      ```

      ```typescript TypeScript theme={null}
      import { CatchAllApiClient } from "newscatcher-catchall-sdk";

      // Initialize the client
      const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY_HERE" });

      // Create a job
      const job = await client.jobs.createJob({
        query: "AI company acquisitions",
        context: "Focus on deal size and acquiring company details",
        start_date: "2025-12-03T19:00:40Z",
        end_date: "2025-12-08T19:00:40Z",
        limit: 10
      });

      const jobId = job.job_id;
      ```

      ```java Java theme={null}
      import com.newscatcher.catchall.CatchAllApi;
      import com.newscatcher.catchall.resources.jobs.requests.SubmitRequestDto;

      // Initialize the client
      CatchAllApi client = CatchAllApi.builder()
          .apiKey("YOUR_API_KEY_HERE")
          .build();

      // Create a job
      var job = client.jobs().createJob(
          SubmitRequestDto.builder()
              .query("AI company acquisitions")
              .context("Focus on deal size and acquiring company details")
              .startDate("2025-12-03T19:00:40Z")
              .endDate("2025-12-08T19:00:40Z")
              .limit(10)
              .build()
      );

      String jobId = job.getJobId();
      ```
    </CodeGroup>

    <Note>
      Replace `YOUR_API_KEY_HERE` with your actual API key. Only `query` is
      required - all other parameters are optional. For details, see
      [`POST /catchAll/submit`](/web-search-api/api-reference/jobs/create-job).
    </Note>

    <Tip>
      Use `limit=10` for faster testing, then continue processing more with
      [`POST /catchAll/continue`](/web-search-api/api-reference/jobs/continue-job) without
      restarting.
    </Tip>

    You receive a response with a job ID:

    ```json theme={null}
    {
      "job_id": "af7a26d6-cf0b-458c-a6ed-4b6318c74da3"
    }
    ```

    Use this `job_id` to check the status and retrieve results.
  </Step>

  <Step title="Check job status">
    Wait 1-2 minutes, then check the processing status. Poll this endpoint every 30-60 seconds until the status is `completed`:

    <CodeGroup>
      ```bash curl theme={null}
      curl -X GET https://catchall.newscatcherapi.com/catchAll/status/af7a26d6-cf0b-458c-a6ed-4b6318c74da3 \
        -H "x-api-key: YOUR_API_KEY_HERE"
      ```

      ```python Python theme={null}
      from newscatcher_catchall import CatchAllApi
      import time

      client = CatchAllApi(api_key="YOUR_API_KEY_HERE")
      job_id = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3"

      # Poll until job completes
      while True:
          status = client.jobs.get_job_status(job_id)

          if status.status == "completed":
              break
          elif status.status == "failed":
              break

          time.sleep(60)  # Wait 60 seconds before next check
      ```

      ```typescript TypeScript theme={null}
      import { CatchAllApiClient } from "newscatcher-catchall-sdk";

      const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY_HERE" });
      const jobId = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3";

      // Poll until job completes
      while (true) {
        const status = await client.jobs.getJobStatus({ job_id: jobId });

        if (status.status === "completed") {
          break;
        } else if (status.status === "failed") {
          break;
        }

        await new Promise(resolve => setTimeout(resolve, 60000)); // Wait 60 seconds
      }
      ```

      ```java Java theme={null}
      import com.newscatcher.catchall.CatchAllApi;
      import com.newscatcher.catchall.types.StatusResponseDto;

      CatchAllApi client = CatchAllApi.builder()
          .apiKey("YOUR_API_KEY_HERE")
          .build();

      String jobId = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3";

      // Poll until job completes
      while (true) {
          StatusResponseDto status = client.jobs().getJobStatus(jobId);

          if ("completed".equals(status.getStatus().orElse(null))) {
              break;
          } else if ("failed".equals(status.getStatus().orElse(null))) {
              break;
          }

          Thread.sleep(60000); // Wait 60 seconds
      }
      ```
    </CodeGroup>

    The response shows the current processing stage:

    <Expandable title="Example response">
      ```json theme={null}
      {
        "job_id": "af7a26d6-cf0b-458c-a6ed-4b6318c74da3",
        "status": "fetching",
        "steps": [
          {
            "status": "submitted",
            "order": 1,
            "completed": true
          },
          {
            "status": "analyzing",
            "order": 2,
            "completed": true
          },
          {
            "status": "fetching",
            "order": 3,
            "completed": false
          },
          {
            "status": "clustering",
            "order": 4,
            "completed": false
          },
          {
            "status": "enriching",
            "order": 5,
            "completed": false
          },
          {
            "status": "completed",
            "order": 6,
            "completed": false
          },
          {
            "status": "failed",
            "order": 7,
            "completed": false
          }
        ]
      }
      ```
    </Expandable>

    Jobs advance through stages sequentially from `submitted` (order 1) to
    either `completed` (order 6) or `failed` (order 7). Processing typically
    takes 10-15 minutes.
  </Step>

  <Step title="Retrieve results">
    Results become available during the `enriching` stage. The `progress_validated`
    field shows how many candidate clusters have been validated. You can retrieve
    partial results as batches are completed, or wait until status is `completed`
    and retrieve all records:

    <CodeGroup>
      ```bash curl theme={null}
      curl -X GET https://catchall.newscatcherapi.com/catchAll/pull/af7a26d6-cf0b-458c-a6ed-4b6318c74da3 \
        -H "x-api-key: YOUR_API_KEY_HERE"
      ```

      ```python Python theme={null}
      from newscatcher_catchall import CatchAllApi

      client = CatchAllApi(api_key="YOUR_API_KEY_HERE")
      job_id = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3"

      # Retrieve results
      results = client.jobs.get_job_results(job_id)
      ```

      ```typescript TypeScript theme={null}
      import { CatchAllApiClient } from "newscatcher-catchall-sdk";

      const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY_HERE" });
      const jobId = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3";

      // Retrieve results
      const results = await client.jobs.getJobResults({ job_id: jobId });
      ```

      ```java Java theme={null}
      import com.newscatcher.catchall.CatchAllApi;
      import com.newscatcher.catchall.types.PullJobResponseDto;

      CatchAllApi client = CatchAllApi.builder()
          .apiKey("YOUR_API_KEY_HERE")
          .build();

      String jobId = "af7a26d6-cf0b-458c-a6ed-4b6318c74da3";

      // Retrieve results
      PullJobResponseDto results = client.jobs().getJobResults(jobId);
      ```
    </CodeGroup>
  </Step>

  <Step title="Review response">
    The API returns structured records according to LLM-generated enrichment schema:

    <Expandable title="Example response">
      ```json JSON {22-31} theme={null}
      {
        "job_id": "89e48e60-7131-411b-b5cd-0d4c85e1927b",
        "query": "AI company acquisitions",
        "context": "Focus on deal size and acquiring company details",
        "validators": [
          "has_deal_details",
          "involves_actual_acquisition",
          "is_about_ai_company_acquisition"
        ],
        "status": "completed",
        "duration": "23m",
        "candidate_records": 3068,
        "valid_records": 230,
        "progress_validated": 3068
        "date_range": {
          "start_date": "2025-12-03T19:00:40Z",
          "end_date": "2025-12-08T19:00:40Z"
        },
        "all_records": [
          {
            "record_id": "5685210581375815445",
            "record_title": "Marvell to Acquire Celestial AI for $3.25 Billion",
            "enrichment": {
              "confidence": "high",
              "deal_date": "2025-12-02",
              "acquiring_company": "Marvell Technology",
              "deal_value": "$3.25 billion",
              "strategic_rationale": "To strengthen Marvell's portfolio, gain access to Celestial AI's photonics technology, accelerate its roadmap in the interconnect space, strengthen Marvell's position as a leader in the fastest-growing opportunities in the AI data center infrastructure market, and develop a 'silicon photonics turnkey solution' for AI servers.",
              "acquiring_company_details": "American semiconductor design company, a fabless semiconductor company that designs, develops, and sells integrated circuits (ICs), and supplies semiconductor chips and hardware products to the data center, telecommunications company (carrier) 5G infrastructure, corporate networking, and storage markets. Helps big-tech clients like Amazon, Google, Microsoft, and Meta Platforms design their own AI chips. Market capitalization was $78.59 billion.",
              "ai_technology_focus": "photonics, silicon photonics, Photonic Fabric technology, optical interconnect, co-packaged optical (CPO) connections",
              "acquisition_status": "announced, pending regulatory approval",
              "acquired_company": "Celestial AI"
            },
            "citations": [
              {
                "title": "'빛' 기술에 꽂힌 구글·AWS…AI 통신 속도 획기적으로 높인다",
                "link": "https://n.news.naver.com/mnews/article/015/0005221540",
                "published_date": "2025-12-08 15:52:17"
              },
              {
                "title": "Miliardový obchod v AI čipech. Marvell jedná o koupi rychle rostoucího start-upu",
                "link": "https://www.zive.cz/clanky/miliardovy-obchod-v-ai-cipech-marvell-jedna-o-koupi-rychle-rostouciho-start-upu/sc-3-a-238455/default.aspx",
                "published_date": "2025-12-06 16:45:00"
              },
              {
                "title": "Marvell to buy chip startup Celestial AI for $3.25 billion",
                "link": "https://www.communicationstoday.co.in/marvell-to-buy-chip-startup-celestial-ai-for-3-25-billion",
                "published_date": "2025-12-04 03:01:56"
              }
            ]
          }
        ]
      }
      ```
    </Expandable>
  </Step>
</Steps>

<Warning>
  Field names in the `enrichment` object are LLM-generated and vary between jobs,
  even with identical queries. The example shows one possible structure for
  acquisition queries. Use [`POST /catchAll/initialize`](/web-search-api/api-reference/jobs/initialize-job)
  to preview suggested fields, or define custom enrichments for consistent schemas.
</Warning>

## What's next

Now that you've made your first calls to the CatchAll API:

1. Learn how to automate recurring queries with
   [Monitors](/web-search-api/concepts/monitors).
2. Read [Dynamic schemas](/web-search-api/concepts/dynamic-schemas)
   to understand variable response structures in your code.
3. Explore the [API Reference](/web-search-api/api-reference/jobs/create-job) for detailed
   endpoint documentation.

<Note>Need help? Contact our support team at [support@newscatcherapi.com](mailto:support@newscatcherapi.com)</Note>