Scrape URL and convert to Markdown

Scrapes the given URL, converts the HTML content to Markdown, and returns the result.

GET /web/scrape/markdown

JavaScript

JavaScript

import ContextDev from 'context.dev';

const client = new ContextDev({
  apiKey: process.env['CONTEXT_DEV_API_KEY'], // This is the default and can be omitted
});

const response = await client.web.webScrapeMd({ url: 'https://example.com' });

console.log(response.markdown);

200

{
  "success": true,
  "markdown": "<string>",
  "url": "<string>"
}

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Query Parameters

url · string<uri> · required

Full URL to scrape and convert to markdown (must include http:// or https:// protocol)

includeLinks · boolean · default: true

Preserve hyperlinks in Markdown output

includeImages · boolean · default: false

Include image references in Markdown output

shortenBase64Images · boolean · default: true

Shorten base64-encoded image data in the Markdown output

useMainContentOnly · boolean · default: false

Extract only the main content of the page, excluding headers, footers, sidebars, and navigation

Response

200

application/json

Successful response

success · enum<boolean> · required

Indicates success

Available options: true

markdown · string · required

Page content converted to GitHub Flavored Markdown

url · string · required

The URL that was scraped