feedgrab

feedgrab — Universal Content Grabber

Give it a URL, get back structured Markdown. Supports 17+ platforms with deep extraction.

Trigger

Activate when user provides a URL and wants content fetched/extracted/read:

• /feedgrab <URL>
• "Grab this article"
• "Read this tweet/post"
• "抓取这个链接"
• Any URL from supported platforms

Prerequisites Check

Before fetching, verify feedgrab is installed:

which feedgrab 2>/dev/null || command -v feedgrab 2>/dev/null

If NOT installed, tell the user:

feedgrab is not installed. Run `/feedgrab-setup` or manually:
  pip install feedgrab[all]
  feedgrab setup

Then stop — do not proceed without feedgrab.

Supported Platforms

Pipeline

Step 1: Fetch Content

feedgrab "$ARGUMENTS"

The CLI auto-detects the platform and routes to the appropriate fetcher.

Step 2: Locate Output File

feedgrab saves output to OUTPUT_DIR (default: ./output/). Check the CLI output for the saved file path, typically:

• output/X/author_date：title.md
• output/mpweixin/author_date：title.md
• output/XHS/author_date：title.md
• output/YouTube/author_date：title.md
• output/GitHub/author_date：title.md
• output/LinuxDo/author_date：title.md
• output/IDCFlare/author_date：title.md
• output/Feishu/author_date：title.md
• output/KDocs/author_date：title.md
• output/FlowUs/author_date：title.md
• output/NoteYouDao/author_date：title.md
• output/Zhihu/author_date：title.md
• output/Bilibili/author_date：title.md
• output/Xiaoyuzhou/author_date：title.md
• output/Ximalaya/author_date：title.md
• output/Web/author_date：title.md (paywall / generic pages)

Step 3: Read and Present

Read the output .md file and present the content to the user. The file includes:

• YAML front matter (title, source, author, published, likes, tags, etc.)
• Full article/tweet/post content in Markdown
• Images (as remote URLs or local paths if media download is enabled)

Clipboard Mode

If the user says "grab from clipboard" or the URL contains & (which breaks PowerShell):

feedgrab clip

This reads the URL from the system clipboard.

Error Handling

Tips

• For Twitter deep extraction (views, bookmarks, threads): configure cookies via feedgrab login twitter
• For WeChat articles: no login needed for single articles
• For Xiaohongshu: pip install xhshow for API mode (faster, no browser needed)
• For GitHub: set GITHUB_TOKEN for higher rate limits (5000/hr vs 60/hr)
• For LinuxDo / Discourse: feedgrab login linuxdo if the topic is private or the site requires a browser session / Cloudflare cookie
• For IDCFlare / Discourse: feedgrab login idcflare if the topic is private or the site requires a browser session / Cloudflare cookie
• For Discourse forums: default reply mode is OP + topic-author follow-up replies only; switch with LINUXDO_REPLY_MODE / IDCFLARE_REPLY_MODE (author / all / none)
• For Reddit: use REDDIT_REPLY_MODE=tree for nested replies or REDDIT_REPLY_MODE=all for extra morechildren expansion; use /feedgrab-batch for reddit-so search
• For Feishu: set FEISHU_APP_ID + FEISHU_APP_SECRET for Open API access
• For KDocs: feedgrab login kdocs to save session (or enable KDOCS_CDP_ENABLED=true to reuse running Chrome)
• For FlowUs: feedgrab login flowus for paid/private docs; default images use signed online URLs, set FLOWUS_DOWNLOAD_IMAGES=true for local attachments
• For Zhihu: feedgrab login zhihu to save session (enables full answer content)
• For Bilibili subtitles: free tier (player/v2 + WBI) works out of the box; set BILIBILI_SUBTITLE_WHISPER=true for Whisper fallback on videos without subtitles
• For Xiaoyuzhou / Ximalaya: set GROQ_API_KEY for Whisper transcription (free tier = metadata + shownotes only)
• For paywall sites: PAYWALL_ENABLED=true by default, no extra config
• Run feedgrab doctor to diagnose issues