Extract Structured Data

Crawl a website and return schema-shaped data you can drop directly into products, agents, and workflows.

POST /web/extract

JavaScript

JavaScript

import ContextDev from 'context.dev';

const client = new ContextDev({
  apiKey: process.env['CONTEXT_DEV_API_KEY'], // This is the default and can be omitted
});

const response = await client.web.extract({
  schema: {
    type: 'bar',
    properties: 'bar',
    required: 'bar',
    additionalProperties: 'bar',
  },
  url: 'https://example.com',
});

console.log(response.cache_metadata);

200

{
  "status": "<string>",
  "url": "<string>",
  "urls_analyzed": [
    "<string>"
  ],
  "data": {},
  "metadata": {
    "numUrls": 123,
    "maxCrawlDepth": 123,
    "numSucceeded": 123,
    "numFailed": 123,
    "numSkipped": 123,
    "numBlocked": 123,
    "actionsApplied": [
      {
        "instruction": "<string>",
        "status": "applied",
        "method": "<string>",
        "targetDescription": "<string>",
        "completionEvidence": "<string>",
        "error": "<string>",
        "durationMs": 123
      }
    ]
  },
  "cache_metadata": {
    "status": "hit",
    "age_ms": 1
  },
  "key_metadata": {
    "credits_consumed": 123,
    "credits_remaining": 123
  }
}

10 Credits

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <API_KEY>, where <API_KEY> is your api key.

Body

application/json

url · string<uri> · required

The starting website URL to crawl and extract from. Must include http:// or https://.

schema · object · required

JSON Schema for the returned data object. Image fields such as image_urls or product_photos automatically make page image references available to extraction, so product data and photos can be returned in one call. TypeScript Zod users can pass a JSON Schema generated from a Zod object; Python users can pass the equivalent JSON Schema object.

Example:

{
  "type": "object",
  "properties": {
    "mission_statement": {
      "type": "string",
      "description": "The company's stated mission."
    },
    "case_studies": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "title": { "type": "string" },
          "url": { "type": "string" }
        },
        "required": ["title", "url"],
        "additionalProperties": false
      }
    }
  },
  "required": ["mission_statement", "case_studies"],
  "additionalProperties": false
}

instructions · string

Optional extraction guidance, such as which facts to prioritize or how to interpret fields in the schema.

Maximum string length: 2000

factCheck · boolean · default: false

When true, every returned value must be grounded in facts stated on the page; fields that cannot be supported by the page are returned as null/empty. When false (default), the model may make reasonable inferences and derivations from the page content (e.g. ideal customer, competitor analysis, recommendations) while keeping verifiable specifics (names, quotes, URLs, dates, metrics) faithful to the source.

followSubdomains · boolean · default: false

When true, follow links on subdomains of the starting URL's domain.

maxPages · integer · default: 5

Maximum number of pages to analyze for extraction. Hard cap: 50. Defaults to 5.

Required range: 1 <= x <= 50

maxDepth · integer

Optional maximum link depth from the starting URL (0 = only the starting page). If omitted, there is no crawl depth limit.

Required range: x >= 0

pdf · object

Show child attributes

includeFrames · boolean · default: false

When true, iframe contents are included in Markdown before extraction.

maxAgeMs · integer · default: 604800000

Return cached scrape results if a prior scrape for the same parameters is younger than this many milliseconds. Defaults to 7 days (604800000 ms).

Required range: 0 <= x <= 2592000000

waitForMs · integer

Optional browser wait time in milliseconds after initial page load for each crawled page.

Required range: 0 <= x <= 30000

settleAnimations · boolean · default: false

When true, waits briefly for CSS and transition animations to settle before extracting each crawled page. Defaults to false. This adds a bit of latency in exchange for more stable output on animated pages.

actions · (Wait · object | Perform · object | Scroll · object)[]

Optional browser actions executed in order on the requested page after it loads, before links are discovered or additional pages are crawled. Requires a paid plan. When actions are provided and stopAfterMs is omitted, the crawl budget defaults to 110000 ms.

Maximum array length: 5

Browser action discriminated by do. Each variant exposes only its applicable fields.

Wait

Perform

Scroll

Show child attributes

stopAfterMs · integer · default: 80000

Soft time budget for the crawl in milliseconds. Min: 10000 (10s). Max: 110000 (110s). Defaults to 80000 (80s), or 110000 (110s) when browser actions are provided.

Required range: 10000 <= x <= 110000

timeoutMS · integer

Optional timeout in milliseconds for the request. If the request takes longer than this value, it will be aborted with a 408 status code. Maximum allowed value is 300000ms (5 minutes).

Required range: 1000 <= x <= 300000

tags · string[]

Optional tags for tracking usage. Up to 20 tags, each 1 to 50 characters.

Maximum array length: 20

Required string length: 1 - 50

Example:

["production", "team-alpha"]

Response

200

application/json

Successful response

status · string · required

Status of the response, e.g., 'ok'

url · string · required

The starting URL that was analyzed

urls_analyzed · string[] · required

List of URLs whose Markdown was used for extraction

data · object · required

Extracted data matching the request schema

metadata · object · required

Show child attributes

cache_metadata · object · required

Cache outcome for this response. Composite responses are hits only when every cache-controlled fetch contributing to the output was a hit; age_ms is the oldest contributing hit.

Show child attributes

key_metadata · object

Metadata about the API key used for the request. Included in every response whenever a valid API key is provided, even when the response status is not 200.

Show child attributes