> ## Documentation Index
> Fetch the complete documentation index at: https://newscatcherinc-docs.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Working with dynamic schemas

> Build integrations that handle variable field names and structures.

By default, CatchAll generates response schemas dynamically—field names can change
between jobs, even with identical inputs. This guide shows how to build integrations
that handle this variability and how to define custom enrichments to ensure
consistent field names.

## Guaranteed vs variable fields

Every response includes `record_id`, `record_title`, and `citations`.

The `enrichment` object always contains `enrichment_confidence`, added by the
system regardless of the enrichments generated or defined by you. All other field names
in `enrichment` are variable with automatic generation, or consistent when you
define custom enrichments.

## Why schemas vary

Submit the same query twice, get different field names:

<CodeGroup>
  ```json Job 1 result theme={null}
  {
    "record_id": "5262823697790152939",
    "record_title": "Oracle Q1 2026 Earnings Exceed Expectations",
    "enrichment": {
      "company_name": "Oracle",
      "revenue": "$14.9 billion",
      "profit_margin": "42%"
    }
  }
  ```

  ```json Job 2 result theme={null}
  {
    "record_id": "7930223600790151893",
    "record_title": "Oracle Q1 2026 Results",
    "enrichment": {
      "company": "Oracle",
      "total_revenue": "$14.9B",
      "operating_margin": "42%"
    }
  }
  ```
</CodeGroup>

**Why this happens:**

* LLMs generate extractors dynamically for each job
* Different keywords, validators, and extractors are created
* Field names are chosen semantically to match content

**This is expected behavior, not a bug.**

## Get enrichment suggestions

Use the initialize endpoint to see suggested enrichments for your query:

<CodeGroup>
  ```python Python theme={null}
  from newscatcher_catchall import CatchAllApi

  client = CatchAllApi(api_key="YOUR_API_KEY")

  suggestions = client.jobs.initialize_job(
      query="Executive departures at Fortune 500 technology companies",
      context="Include executive name, position, company name, departure date"
  )

  print(suggestions.enrichments)
  ```

  ```typescript TypeScript theme={null}
  import { CatchAllApiClient } from "newscatcher-catchall-sdk";

  const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY" });

  const suggestions = await client.jobs.initialize({
    query: "Executive departures at Fortune 500 technology companies",
    context: "Include executive name, position, company name, departure date"
  });

  console.log(suggestions.enrichments);
  ```
</CodeGroup>

Suggested enrichments:

```json theme={null}
{
  "enrichments": [
    {
      "name": "executive_name",
      "description": "Extract the name of the departing executive",
      "type": "text"
    },
    {
      "name": "executive_position",
      "description": "Extract the position or title of the departing executive",
      "type": "text"
    },
    {
      "name": "subject_company",
      "description": "Extract the name of the company from which the executive is departing",
      "type": "company"
    },
    {
      "name": "departure_date",
      "description": "Extract the date of the executive's departure",
      "type": "date"
    },
    {
      "name": "departure_reason",
      "description": "Extract the stated reason for the executive's departure if available",
      "type": "text"
    }
  ]
}
```

These suggestions help you understand possible enrichments before submitting.
You can use them as-is, modify them, add more, or ignore them and let the system
generate fresh enrichments when you submit the job.

For more information about enrichment types, see
[Create job > enrichments > type](/docs/web-search-api/api-reference/jobs/create-job#body-enrichments-items-type).

## Define custom enrichments

Submit enrichments in your request for consistent field names:

<CodeGroup>
  ```python Python theme={null}
  from newscatcher_catchall import CatchAllApi

  client = CatchAllApi(api_key="YOUR_API_KEY")

  # Use suggested enrichments (modified: removed departure_reason, kept 4 fields)
  job = client.jobs.create_job(
      query="Executive departures at Fortune 500 technology companies",
      context="Include executive name, position, company name, departure date",
      enrichments=[
          {
              "name": "executive_name",
              "description": "Extract the name of the departing executive",
              "type": "text"
          },
          {
              "name": "executive_position",
              "description": "Extract the position or title of the departing executive",
              "type": "text"
          },
          {
              "name": "subject_company",
              "description": "Extract the name of the company from which the executive is departing",
              "type": "company"
          },
          {
              "name": "departure_date",
              "description": "Extract the date of the executive's departure",
              "type": "date"
          }
      ]
  )
  ```

  ```typescript TypeScript theme={null}
  import { CatchAllApiClient } from "newscatcher-catchall-sdk";

  const client = new CatchAllApiClient({ apiKey: "YOUR_API_KEY" });

  // Use suggested enrichments (modified: removed departure_reason, kept 4 fields)
  const job = await client.jobs.createJob({
    query: "Executive departures at Fortune 500 technology companies",
    context: "Include executive name, position, company name, departure date",
    enrichments: [
      {
        name: "executive_name",
        description: "Extract the name of the departing executive",
        type: "text"
      },
      {
        name: "executive_position",
        description: "Extract the position or title of the departing executive",
        type: "text"
      },
      {
        name: "subject_company",
        description: "Extract the name of the company from which the executive is departing",
        type: "company"
      },
      {
        name: "departure_date",
        description: "Extract the date of the executive's departure",
        type: "date"
      }
    ]
  });
  ```
</CodeGroup>

Every job returns the same field names:

```json theme={null}
{
  "record_id": "12345",
  "record_title": "John Smith Departs Apple as CFO",
  "enrichment": {
    "enrichment_confidence": "high",
    "executive_name": "John Smith",
    "executive_position": "Chief Financial Officer",
    "subject_company": {
      "source_text": "Apple Inc.",
      "confidence": 0.99,
      "metadata": {
        "name": "Apple Inc.",
        "domain_url": "apple.com",
        "domain_url_confidence": "high"
      }
    },
    "departure_date": "2026-01-15"
  }
}
```

<Note>
  Fields of type `company` (like `subject_company` in the example above) return
  a structured object with confidence scores for entity identification and domain
  resolution. To learn more, see the [Company enrichment](/docs/web-search-api/api-reference/jobs/company-enrichment-dto) data model.
</Note>

## Working with dynamic schemas

If you don't define custom enrichments, field names vary between jobs.
Handle this with pattern matching:

<CodeGroup>
  ```python Python theme={null}
  def find_field(enrichment, patterns):
      """Find first field matching any pattern."""
      for key, value in enrichment.items():
          if any(pattern in key.lower() for pattern in patterns):
              return value
      return None

  # Usage - handles "revenue", "total_revenue", "sales", etc.
  revenue = find_field(record["enrichment"], ["revenue", "sales"])
  company = find_field(record["enrichment"], ["company", "company_name"])
  ```

  ```typescript TypeScript theme={null}
  function findField(enrichment: Record<string, any>, patterns: string[]): any | null {
    for (const [key, value] of Object.entries(enrichment)) {
      if (patterns.some(p => key.toLowerCase().includes(p))) {
        return value;
      }
    }
    return null;
  }

  // Usage - handles "revenue", "total_revenue", "sales", etc.
  const revenue = findField(record.enrichment, ["revenue", "sales"]);
  const company = findField(record.enrichment, ["company", "company_name"]);
  ```
</CodeGroup>

## See also

* [Initialize endpoint](/docs/web-search-api/api-reference/jobs/initialize-job)
* [Create job endpoint](/docs/web-search-api/api-reference/jobs/create-job)
* [Monitors](/docs/web-search-api/concepts/monitors)