Overview

Every endpoint of the Scrapely API with its parameters, request and response examples in cURL, Python and Node.js, generated from the OpenAPI document. Start here for the base URL, authentication and the conventions every endpoint shares.

Embed
Preview of the card
<a href="https://scrapelyserp.com/reference/overview"><img src="https://scrapelyserp.com/reference/_api/public/orgs/affc5b0f-d3c2-423a-8143-9d10db5736ae/spaces/e4cbe065-c1c2-4572-b134-0292dedd9c39/pages/2fc28f17-745d-4ea2-b142-d23aa625335e/social-card.svg?v=3e7ae33eec8b" alt="Overview" width="600" height="315"></a>

Every endpoint of the Scrapely API with its parameters, request and response examples in cURL, Python and Node.js, generated from the OpenAPI document. Start here for the base URL, authentication and the conventions every endpoint shares.

Base URL

https://api.scrapelyserp.com/v1

Every endpoint lives under /v1, over HTTPS. The machine-readable description is https://api.scrapelyserp.com/v1/openapi.json (OpenAPI 3.1): import it into Postman, Insomnia or a client generator. New fields and endpoints are added without notice, so ignore fields you do not know; breaking changes ship as a new version.

Authentication

Send an API key from Dashboard → Developers → API keys with every call. The samples read it from the SCRAPELY_API_KEY environment variable.

How

Example

Notes

Bearer header

Authorization: Bearer sk_live_…

Recommended.

API key header

X-API-Key: sk_live_…

For clients that reserve Authorization.

Query parameter

?api_key=sk_live_…

Accepted for clients that cannot set headers. Prefer a header: URLs end up in logs.

Request

curl "https://api.scrapelyserp.com/v1/account" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY"

GET /v1/platforms and GET /v1/openapi.json need no key. A missing, unknown or revoked key is a 401 UNAUTHORIZED. More: Authentication.

Requests and responses

  • POST, PUT and PATCH take a JSON body (Content-Type: application/json); GET and DELETE take query parameters.

  • Request parameters and resource fields are snake_case. Product fields inside item results, snapshots and changes are camelCase (inStock, newMinPrice), the same names everywhere.

  • Timestamps are ISO 8601 in UTC. Prices are numbers in the marketplace currency, named beside them (currency).

  • A field the retailer does not show is left out rather than sent empty; null means "known to be nothing".

Every response carries these headers (the credit headers on item calls):

Header

Meaning

X-Request-Id

Unique per request. Quote it to support.

X-Scrapely-Credits

Credits this request cost (0 when refunded).

X-Scrapely-Balance

Your credit balance after this request.

X-RateLimit-Limit

Calls allowed per window for this key.

X-RateLimit-Remaining

Calls left in the current window.

X-RateLimit-Reset

Unix time (seconds) when the window frees a call.

Retry-After

On a 429 (and some 503s): seconds to wait before retrying.

Idempotent-Replayed

true when the response was replayed for a repeated Idempotency-Key.

The item envelope

POST /v1/products, POST /v1/offers, GET /v1/products/{id}/offers and POST /v1/offers/check answer one shape. A well-formed call is always 200, even when every item failed: read each result's status.

Field

Type

Description

platform

string

The platform that was read.

marketplace

string

The marketplace that was read; every price is in its currency.

tier

string

The tier that answered (standard on one-tier platforms). One of: pulse, lite, full, standard.

include

string[]

Echoes the request's include; absent without it. It changes nothing. One of: delivery.

results

object[]

One per unique requested id, in request order. Every id appears exactly once.

results[].id

string

The item id as the platform knows it (the ASIN on Amazon).

results[].status

string

ok: returned with no errors (charged). partial: returned, but some fields could not be read: do not trust price or stock, re-fetch at a higher tier (refunded). not_found: the retailer says the item does not exist: delist, do not retry (charged). unavailable: we could not get an answer this time: retry later, not evidence the item is gone (refunded). One of: ok, partial, not_found, unavailable.

stats

object

—

stats.requested

integer

Unique items after de-dup.

stats.ok

integer

Answered in full (charged).

stats.partial

integer

Answered with fields missing (refunded). Always 0 where an answer cannot be partial (offers).

stats.notFound

integer

The platform says the item does not exist (charged).

stats.unavailable

integer

No answer this time (refunded; retry later).

stats.elapsedMs

integer

How long the call took, in milliseconds.

stats.delivery

object

lite + full.

credits

integer

Credits charged for the call (also X-Scrapely-Credits).

Item statuses (ok, partial, not_found, unavailable) and what each is charged: Responses & statuses.

Lists and pagination

List endpoints answer { "object": "list", "data": [...], "has_more": bool } and take limit.

Endpoint

limit

Paging

GET /v1/watchlists

1–100, default 100

Newest first. At most 100 lists per account, so one page holds them all.

GET /v1/watchlists/{id}/items

1–1,000, default 100

In id order. Pass the previous page's next_starting_after as starting_after until has_more is false. changed_since and state filter.

GET /v1/watchlists/{id}/runs

1–100, default 20

Newest first; the newest 500 runs are kept.

GET /v1/webhook_endpoints

—

Every endpoint (at most 10).

Idempotency

A client that times out on a create cannot tell whether it happened. POST /v1/watchlists therefore accepts an Idempotency-Key header (or an idempotency_key body field) of up to 255 printable characters, ideally a UUID, remembered for 24 hours:

  • A repeat after success returns the original response with Idempotent-Replayed: true, and creates nothing.

  • A repeat while the first request is still running is a 409 CONFLICT: retry in a few seconds.

  • The same key with a different body is a 422 with INVALID_PARAM and param: "idempotency_key".

  • A repeat after a failure (any non-2xx: nothing was created) runs again.

Item calls need no key: they create nothing, and a retried call is billed only for what it answers. POST /v1/webhook_endpoints deliberately has none: a replayable response would be a stored secret.

Errors

Every error has one shape. Branch on code; message is for humans and may be reworded; param names the parameter at fault; docs links to the code's entry.

{
  "error": {
    "code": "INVALID_PARAM",
    "message": "ids[1] (\"12\") is not a TCIN: expected the numeric Target item number (TCIN), 6 to 10 digits, e.g. 83887639.",
    "param": "ids[1]",
    "docs": "/docs/errors#invalid_param"
  }
}

Code

HTTP

Meaning

What to do

MISSING_PARAM

400

A required parameter was not sent. param names it.

Add the parameter. JSON bodies need Content-Type: application/json.

INVALID_PARAM

400

A parameter was sent but its value is not accepted: wrong type, out of range, an unsupported marketplace, a private webhook URL, an entry in the id list that is not a valid id on that platform (param is then the list and its index, e.g. ids[i], asins[i] on Amazon), or two options that cannot be combined. param names it.

Read message: it says exactly what is expected.

UNAUTHORIZED

401

No API key was sent, or the key is unknown or revoked.

Send your key as Authorization: Bearer <key>, X-API-Key: <key> or ?api_key=<key>. Keys are in the dashboard under Developers → API keys.

DEMO_READ_ONLY

403

The key belongs to the public, read-only demo workspace. It can be seen in the dashboard but cannot make requests or spend credits.

Create an account (5,000 free credits) and use your own API key.

INSUFFICIENT_CREDITS

402

Your credit balance cannot cover this request. Nothing was charged. A call is all-or-nothing: every item in it must be affordable.

Buy a credit pack (credits never expire), or enable auto top-up in Billing.

NOT_FOUND

404

The endpoint, watchlist, webhook endpoint or structured endpoint does not exist, or belongs to another account.

Check the id and the path. The API reference lists every endpoint.

GONE

410

The endpoint was retired and will not come back. The generic URL scraper (/v1/scrape, /v1/jobs, /v1/batches) answers this: Scrapely serves retailer data only.

Use the product API, POST /v1/products, with a platform, a marketplace and item ids; watchlists (/v1/watchlists) run it on a schedule.

CONFLICT

409

The request conflicts with the resource's current state: a request with the same Idempotency-Key is still in progress, a watchlist that is already running was asked to run again, or a per-account limit was reached.

Retry idempotency and run conflicts after a few seconds; for limits, delete something first.

CONCURRENCY_LIMIT

429

All of your concurrent request slots are busy. The request was refused immediately and nothing was charged.

Retry with backoff. Concurrency grows with lifetime credits purchased.

RATE_LIMITED

429

Too many API calls with this key in a short window. Separate from concurrency: this counts calls, including cheap ones such as listing watchlist changes.

Wait Retry-After seconds. Prefer webhooks to polling in a tight loop.

TARGET_FAILED

502

We could not get a usable answer from the retailer after our retries: it blocked us, rate-limited us, errored, or never responded. You were not charged.

Retry with backoff. Items that fail inside a successful call come back unavailable and are refunded on their own.

UNAVAILABLE

503

A dependency on our side is down or overloaded. Nothing was charged.

Retry with exponential backoff. Status updates at /status.

NOT_IMPLEMENTED

501

The platform, marketplace, endpoint or option is announced but not available yet. It answers with "coming_soon": true (structured endpoints also with "message": "Coming soon"); nothing is charged.

Use a live platform and marketplace (GET /v1/platforms lists them) with POST /v1/products. Contact support if you need something sooner.

INTERNAL

500

Something failed on our side. Nothing was charged, and the failure was logged with your X-Request-Id.

Retry. If it persists, contact support with the X-Request-Id header value.

Coming-soon platforms, marketplaces and endpoints add "coming_soon": true to their 501. When to retry: Errors & retries.

Every endpoint

Products

Method

Path

Does

POST

/v1/products

Get products

Offers

Method

Path

Does

POST

/v1/offers

Get every offer for a list of items

POST

/v1/offers/check

Check whether exact offers can be bought

GET

/v1/products/{id}/offers

Get every offer for one item

Platforms

Method

Path

Does

GET

/v1/platforms

List platforms and marketplaces

Watchlists

Method

Path

Does

POST

/v1/watchlists

Create a watchlist

GET

/v1/watchlists

List watchlists

GET

/v1/watchlists/{id}

Get a watchlist

PATCH

/v1/watchlists/{id}

Update a watchlist

DELETE

/v1/watchlists/{id}

Delete a watchlist

POST

/v1/watchlists/{id}/run

Run a watchlist now

GET

/v1/watchlists/scrapers

List pipeline scrapers

GET

/v1/watchlists/{id}/runs

List runs

GET

/v1/watchlists/{id}/items

List items

POST

/v1/watchlists/{id}/items

Add or remove items

POST

/v1/watchlists/{id}/asins

Add or remove items (legacy path)

Webhook endpoints

Method

Path

Does

GET

/v1/webhook_endpoints

List webhook endpoints

POST

/v1/webhook_endpoints

Create a webhook endpoint

GET

/v1/webhook_endpoints/{id}

Get a webhook endpoint

PATCH

/v1/webhook_endpoints/{id}

Update a webhook endpoint

DELETE

/v1/webhook_endpoints/{id}

Delete a webhook endpoint

POST

/v1/webhook_endpoints/{id}/rotate

Rotate the signing secret

POST

/v1/webhook_endpoints/{id}/test

Send a test event

S3 export

Method

Path

Does

GET

/v1/exports/s3

Get the S3 export destination

PUT

/v1/exports/s3

Create or update the S3 export destination

DELETE

/v1/exports/s3

Remove the S3 export destination

POST

/v1/exports/s3/test

Test the S3 connection

Account & usage

Method

Path

Does

GET

/v1/account

Get account

GET

/v1/usage

Get usage

Webhook events

What Scrapely POSTs to your endpoints: watchlist.product.changed, watchlist.product.matched, watchlist.product.delisted, watchlist.run.completed, webhook.test.

Updated