POST /v1/data/web/crawl/run
Pricing: $0.003 per page ($3.00 per 1,000).
Request
Send the inputs underparams. Only the keys below are accepted; any other key returns 422 invalid_params.
The page to start from, like https://docs.example.com. Links are followed from here.Format e.g.
https://www.example.com.Example
Response fields
Each row indata is one page, keyed by these fields.
Example response
Abridged: long text and lists are shortened.Related APIs
For one page useweb/scrape; for just the URLs, up to 5,000, much cheaper use web/map; for named fields as JSON use web/extract.