Batches

Scrape up to 25,000 URLs, or crawl a site, as one asynchronous job. Read each page’s Markdown or HTML when the batch finishes.

A batch scrapes many pages as one asynchronous job. Submit up to 25,000 URLs, or a crawl from a start URL or sitemap, and you get a batch ID right away. When the batch finishes, read each page’s Markdown or HTML as paginated JSON or download it as files.

What batches support

Every batch has one mode and one output format:

Field	Values
input.mode	scrape scrapes a list of URLs (Submit a batch). crawl follows links from a start URL or reads a domain’s sitemap (Large crawls).
input.data.format	markdown or html.

Batches don’t return screenshots, images, bytes, parsed fields, highlights, JSON, or product data. They also don’t accept browser actions, custom headers, zdr, or timeoutOpts. For any of those, call Scrape once per URL.

Submit a URL list

This request scrapes two pages as Markdown. The examples read your API key from CONTEXT_DEV_API_KEY; the Quickstart shows how to create a key and install an SDK.

TypeScript

import ContextDev from "context.dev";

const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY });

const batch = await client.batch.submit({
  input: {
    mode: "scrape",
    data: {
      format: "markdown",
      urls: [
        { url: "https://example.com/about", itemId: "about" },
        { url: "https://example.com/careers", itemId: "careers" },
      ],
    },
  },
  "Idempotency-Key": "site-import-2026-09-26",
});
console.log(batch.id, batch.status);

The API returns 202 with the new batch (trimmed):

{
  "id": "batch_9f2c8a",
  "status": "queued",
  "mode": "scrape",
  "format": "markdown",
  "input": {
    "reserved": 2,
    "reserved_is_ceiling": false,
    "submitted": 2,
    "duplicates": 0,
    "invalid": 0
  },
  "invalid_urls": [],
  "created_at": "2026-09-26T12:00:00.000Z"
}

Save id. input.submitted counts the URLs you sent. input.reserved counts the pages the batch will attempt once duplicates and invalid URLs are removed. The Idempotency-Key header makes a retried submit return this batch instead of creating a second one; see Idempotency.

Lifecycle

Status	Meaning
queued	Accepted and waiting to start.
running	Pages are being scraped.
cancelling	A cancel was requested. Pages already in progress finish first.
completed	Every accepted page was attempted. Individual pages can still have failed.
cancelled	Stopped by a cancel request.
failed	The batch itself stopped. failure.code says why.

completed, cancelled, and failed are final. Poll the batch or add a webhook to find out when it reaches one, then read the results. See Status and results.

Access

A restricted API key needs batches:write to submit, cancel, or delete batches and to replay their webhook deliveries. batches:read covers status, results, the batch list, and delivery history. See API keys.

Choose an endpoint

You need	Use
One page, or an output or option that batches don’t support	Scrape
Up to 500 linked pages returned in one response	Crawl
Up to 25,000 known URLs, or a crawl of more than 500 pages	A batch
A list of a site’s URLs to put in a batch	Map URLs

Related

• Submit a batch: URL lists, page options, deduplication, and idempotency.
• Status and results: progress, result records, files, and how long results are kept.
• Cancel, limits, and errors: cancel and delete, concurrency, and error codes.
• Webhooks: signatures, retries, and replay for batch.* events.
• Submit a batch reference