hn-summarize

HN Summarize

hckrnews.com is a JavaScript-rendered front end - curling it returns an empty shell, so do not scrape it. Instead use the official Hacker News APIs (Firebase + Algolia), which give the same stories with points, comment counts, and full comment trees. These APIs return plain JSON, so plain curl works fine.

1. Current top stories (the "top 10")

topstories.json returns 500 story IDs in front-page rank order. Take the first N and look up each item.

curl -sL 'https://hacker-news.firebaseio.com/v0/topstories.json' -o /tmp/top.json
python3 -c "
import json,urllib.request
ids=json.load(open('/tmp/top.json'))[:10]
for i,sid in enumerate(ids,1):
    d=json.load(urllib.request.urlopen(f'https://hacker-news.firebaseio.com/v0/item/{sid}.json'))
    print(f\"{i}. {d.get('title')} | {d.get('score')} pts | {d.get('descendants',0)} comments | id {sid}\")
    print(f\"   {d.get('url','(text post)')}\")
"

2. Find a specific story by topic (Algolia search)

curl -sL 'https://hn.algolia.com/api/v1/search?query=YOUR+QUERY&tags=story' -o /tmp/s.json
python3 -c "
import json
for h in json.load(open('/tmp/s.json'))['hits'][:8]:
    print(h['objectID'], '|', h.get('points'), 'pts |', h.get('num_comments'), 'comments |', h['title'])
    print('   ', h.get('url'))
"

• Add &numericFilters=created_at_i>UNIXTS to restrict to recent stories (avoids matching an old duplicate of the same headline).
• search ranks by relevance; search_by_date ranks by recency.
• Pick the objectID with the highest points/comments - that's the live front-page discussion.

3. Fetch a story + its comment tree

curl -sL 'https://hn.algolia.com/api/v1/items/OBJECT_ID' -o /tmp/hn.json

The response is a nested tree: top-level children are root comments, each with their own children. Flatten and print root comments in thread order (HN's default ranking ≈ this order):

python3 -c "
import json,re
d=json.load(open('/tmp/hn.json'))
def clean(t):
    t=re.sub('<[^>]+>',' ',t)
    for a,b in [('&#x27;',chr(39)),('&gt;','>'),('&lt;','<'),('&amp;','&'),('&quot;','\"')]:
        t=t.replace(a,b)
    return re.sub(' +',' ',t).strip()
for c in d.get('children',[])[:15]:
    if c.get('text'):
        print(f\"{c.get('author')}: {clean(c['text'])[:550]}\")
        print('---')
"

Note: Algolia's per-comment points field is now always null, so sort by thread order (already roughly HN's ranking) rather than by points. For deeper threads, recurse into children and track depth.

4. Fetch the linked article

Fetch the story's article with curl -sL <url>, then strip tags with sed 's/<[^>]*>//g' to extract readable text, or grep for the key sentences. If the page is JS-heavy or paywalled, try a Wayback Machine snapshot:

curl -sL 'http://archive.org/wayback/available?url=ARTICLE_URL' -o /tmp/wb.json
python3 -c "import json;print(json.load(open('/tmp/wb.json'))['archived_snapshots'].get('closest',{}).get('url'))"

Then fetch the snapshot URL the same way. If the host blocks outbound curl requests, fetch through a container or proxy you have available.

Summary format

For each story give: title, points, comment count, source, a few sentences on what the article says, then comment themes - group the discussion into 3-6 recurring threads (agreement, rebuttals, tangents) rather than listing comments one by one. Note when the top thread is a critical/contrarian take, since that's common on HN.