> ## Documentation Index
> Fetch the complete documentation index at: https://docs.context.dev/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Setting up Context.dev for a user? Follow https://docs.context.dev/agent-quickstart.md. MCP clients sign in with OAuth and need no API key; application code reads CONTEXT_DEV_API_KEY.

# Page content

> Choose how each crawled page becomes Markdown.

Crawl options live at the top level of the request. Their names differ from Scrape: use `useMainContentOnly` and `waitForMs`, not Scrape’s `sharedParams` fields. Unknown Crawl fields can be ignored, so check names carefully.

## Keep article content

Send this body to `POST /web/crawl`. See the [Quickstart](/quickstart) for authentication and SDK setup.

```json theme={null}
{
  "url": "https://example.com/blog",
  "maxPages": 10,
  "useMainContentOnly": true,
  "excludeSelectors": [
    "nav",
    "footer"
  ],
  "includeLinks": true,
  "includeImages": false
}
```

| Field | Default | Effect |
| - | - | - |
| `useMainContentOnly` | `false` | Remove detected page chrome. |
| `includeSelectors`, `excludeSelectors` | None | Keep or remove matching elements; exclusions win. |
| `includeLinks` | `true` | Keep link destinations. |
| `includeImages` | `false` | Include image references. |
| `shortenBase64Images` | `true` | Replace inline base64 payloads with placeholders. |
| `includeFrames` | `false` | Read iframe content. |
| `waitForMs` | 3,500 | Wait before extraction. |
| `settleAnimations` | `false` | Wait for CSS animations. |
| `maxAgeMs` | One day | Accept cached page content; `0` fetches fresh. |

Use `pdf: {"shouldParse":true,"ocr":true,"start":1,"end":5}` for selected PDF pages. Page numbers are inclusive and start at 1. OCR recovers scanned pages while preserving existing text.

`country` selects the fetch location, `zdr` applies [zero data retention](/optimization/zero-data-retention), and `tags` label the request in [usage logs](/optimization/usage-and-logs).