QUICKSTART

Your first successful request.

The public API runs at https://www.scraply.net/v1. Accounts and credits persist in the connected backend. External scraping and checkout services are simulated.

Authentication

Create an account, then generate a key in API Keys. Keys are displayed only once; store yours safely. Supply X-API-Key: YOUR_API_KEY on /v1 requests. Authorization: Bearer YOUR_API_KEY is also supported. Dashboard management endpoints use a session token; do not use an API key to access administrator routes.

GET /v1/scrape

Send a public HTTP or HTTPS URL. Private networks, embedded credentials, and ports other than 80/443 are rejected. A validated standard success costs one credit. Language extraction adds two and JavaScript rendering adds three, only on success. An HTTP 200 with non-empty content and no unresolved captcha is required.

curl · /v1/scrape

curl -G "https://www.scraply.net/v1/scrape" \
  --data-urlencode "url=https://example.com" \
  -H "X-API-Key: YOUR_API_KEY"

Replace YOUR_API_KEY with your saved key.

Parameter	Description
url	Required target URL.
format	html (default), json, or markdown. JSON needs an extraction template.
template	product enables structured product extraction.
render_js	true enables isolated mock browser navigation; +3 credits on success.
mock_scenario	success, retry, blocked, empty, captcha, captcha_fail, http_error.

If processing takes longer than the synchronous window, the API returns HTTP 202 with a job ID. Poll GET /v1/jobs/:id or use the returned result URL. HTTP 202 is not a completed, billable success.

Jobs & structured extraction

POST /v1/jobs accepts a URL, output format, optional CSS selectors, required_fields, render_js, wait_for, and callback_url. Use Idempotency-Key to retry the same submission without creating a second job. POST /v1/batches accepts up to the batch size of your plan.

{
  "url": "https://example.com/product",
  "template": "product",
  "format": "json",
  "render_js": true,
  "wait_for": ".product-price"
}

GET /v1/me returns your current account balance. GET /v1/requests returns your last 50 completed results. Callback delivery is opt-in and requires a public endpoint. See Jobs & Extraction for your account callback settings.

Advanced API preview

GET /v1/capabilities reports available features. POST /v1/batch submits up to 20 URLs to the durable queue; GET /v1/batch/:id returns persisted progress. Each successful individual job costs its quoted credits. GET /v1/analytics/costs shows your usage by domain.

POST /v1/schedules supports five-field UTC cron and explicit is_active:true. Pause or resume with POST /v1/schedules/:id/toggle. Runs honor the creating API key’s daily limit and revocation. Batch webhook_url enables signed delivery after all jobs finish. Advanced transport and marketplace script execution remain pending. Unsupported scrape options return HTTP 501 before creating a job or charging credits. Existing JavaScript rendering remains available.

POST /v1/webhooks/test explicitly sends a signed test event to a public endpoint, at no credit cost. Proxy credentials saved through session-protected account settings are encrypted and never returned. Download the OpenAPI contract below for request formats and limits.

Aggregate callbacks retry up to five times with a stable event ID and body. Verify the HMAC over timestamp + "." + raw body, enforce timestamp freshness, and deduplicate event IDs. Callbacks cost zero credits. GET /v1/analytics/errors returns the latest 50 failures with error_reason diagnostics.

Language extraction & JavaScript

POST /v1/jobs accepts extract_prompt and an optional extract_schema. JSON is the default output for this mode. Omit the schema to use title, price, currency, and in_stock. The current mock maps these product fields from simulated HTML; it does not interpret arbitrary natural language. Unknown required fields fail with zero credits.

{
  "url": "https://example.com/product",
  "render_js": true,
  "wait_for_selector": ".product-price",
  "extract_prompt": "Extract the product price",
  "extract_schema": {
    "type": "object",
    "properties": {
      "price": {
        "type": "number",
        "minimum": 0
      }
    },
    "required": [
      "price"
    ],
    "additionalProperties": false
  }
}

This combined request costs 6 credits only after rendering, clean HTML validation, and schema validation succeed. Language-only costs 3, JavaScript-only costs 4, and standard scraping costs 1. Retries do not multiply the success charge. Failed output costs zero. New jobs return success_credit_cost; replay preserves the original quote. Daily key limits reserve the full quote of active jobs.

Schemas compile offline, default to draft 2020-12, and cannot load remote references. Language extraction cannot be combined with CSS selectors or product templates. render_js navigates an isolated browser to the logical target URL using intercepted mock HTML; external network loading remains disabled. Existing accepted jobs keep their previous price. Credit pack prices are unchanged.

Automation examples

Request type

POST /v1/jobs

# Keep this key unchanged when retrying the same submission.
curl -X POST 'https://www.scraply.net/v1/jobs' \
  -H "X-API-Key: $SCRAPLY_API_KEY" \
  -H 'Content-Type: application/json' \
  -H 'Idempotency-Key: replace-with-a-unique-submission-id' \
  --data '{
  "url": "https://example.com/product",
  "render_js": true,
  "extract_prompt": "Extract the product price",
  "extract_schema": {
    "type": "object",
    "properties": {
      "price": {
        "type": "number"
      }
    },
    "required": [
      "price"
    ],
    "additionalProperties": false
  }
}'

An accepted job is not a billable success. Poll GET /v1/jobs/:id until succeeded, failed, or dead. Poll batches with GET /v1/batch/:id. Schedules start paused in this example; explicitly resume to permit automatic billable work. A reused idempotency key with a different payload returns 409.

MCP connection

Connect a Streamable HTTP MCP client to the local endpoint below. Supply your API key as a header. The server supports initialize, tool discovery, and the scrape, scrape_status, and credit_balance tools. The scrape tool requires a stable idempotency_key; it enqueues a job and returns its ID. Poll scrape_status to obtain the final result. Access and credit limits match the REST API. This local endpoint accepts localhost hosts and same-origin browser requests.

{
  "url": "http://localhost:8080/mcp/v1/scrape",
  "transport": "streamable-http",
  "headers": {
    "X-API-Key": "YOUR_API_KEY"
  }
}

Client configuration syntax varies. Never publish your key in source code or expose it to untrusted tools.

Error codes

400	Invalid URL, options, or request body.
401	Missing, invalid, or expired credentials.
402	Insufficient credits; no successful result charged.
403	Administrator permissions required.
404	Resource not found or not owned by this account.
409	Idempotency key or payment ID payload mismatch.
429	Submission rate or active-job capacity exceeded.
501	Requested capability is not enabled. Zero credits charged.
502	Scraping or validation failed. Zero credits charged.
503	A required service is unavailable.

Credits & crypto checkout

Starter: $8 / 10,000 credits. Pro: $18 / 25,000. Business: $60 / 100,000. The browser calls authenticated POST /api/webhook/ccpayment with a package and unique payment ID. The server selects the price and credit amount. The separate provider webhook /webhook/ccpayment requires a server-side secret and must never be called with merchant secrets from browser code.

All current checkout events are simulated. No cryptocurrency is transferred. Accounts, sessions, balances and usage are persisted by the backend. Browser sessions use HttpOnly cookies.

Integration toolkit

Import the API contract, try the collection, or run an example with your own key. Examples handle asynchronous HTTP 202 responses and read credentials from environment variables.

OpenAPI specification Postman collection Python example JavaScript example

Set SCRAPLY_API_KEY. Optional SCRAPLY_BASE_URL defaults to https://www.scraply.net. Python requires requests; JavaScript uses the built-in fetch API in Node.js 18+.