Hacker News

Search Hacker News stories and comments by keyword, author, story, points and date, and read up to 100 items by id in one call, as typed JSON.

At a glance

Two GET endpoints under /v1/structured/hackernews/. Each answers { endpoint, url, credits, duration_ms, data }. Each call is charged for the data it moves, and a call we could not read is 502 TARGET_FAILED and costs nothing. Rates are estimates until production traffic measures them.

curl -G "https://api.scrapelyserp.com/v1/structured/hackernews/search" \
  -H "Authorization: Bearer $SCRAPELY_API_KEY" \
  --data-urlencode "query=web scraping" \
  -d sort=date \
  -d min_points=20

import os
import requests

HEADERS = {"Authorization": f"Bearer {os.environ['SCRAPELY_API_KEY']}"}

r = requests.get(
    "https://api.scrapelyserp.com/v1/structured/hackernews/search",
    headers=HEADERS,
    params={"query": "web scraping", "sort": "date", "min_points": "20"},
    timeout=30,
)
r.raise_for_status()
data = r.json()
for r in data["data"]["results"]:
    print(r["created_at"], r["points"], r["title"])

const res = await fetch(`https://api.scrapelyserp.com/v1/structured/hackernews/search?${new URLSearchParams({ query: 'web scraping', sort: 'date', min_points: '20' })}`, {
  headers: { Authorization: `Bearer ${process.env.SCRAPELY_API_KEY}` },
  signal: AbortSignal.timeout(30_000),
});
if (!res.ok) throw new Error((await res.json()).error.code);
const data = await res.json();
for (const r of data.data.results) console.log(r.created_at, r.points, r.title);

Search

Each result has id, type (story, comment, poll, pollopt or job), title, url (where a link story points), hn_url, author, points, comments, text (plain text of a text story or a comment), created_at, story_id, story_title and parent_id. A field an item does not have is null: a comment has no title and no points. total counts every match.

A query that matches nothing is an empty results with a total of 0. To watch for mentions, search with sort=date and a since that moves forward. A comment can match without its words being in the story, so filter on story to read one thread.

Full reference → GET /v1/structured/hackernews/search

Items by id

data.items are in the order you asked, with id, type, title, url, hn_url, author, points, comments (a story's whole comment count), text, created_at, parent_id and reply_ids, the ids of its direct replies, which you can send straight back as ids to read a thread level by level. A deleted item keeps only its id, type and time, with deleted set. An id that is not an item comes back in data.not_found, and a call where none is an item is a free 404 NOT_FOUND. If any id cannot be read after retries the whole call fails and costs nothing, so an answer is never partly missing.

Full reference → GET /v1/structured/hackernews/items