Examples for every URL pattern
Working AgentFetch examples for 16 of the most-fetched URL patterns — from Hacker News threads to arXiv papers to JS-rendered Notion docs. Each shows a curl that returns clean, token-budgeted Markdown.
Fetch a Hacker News thread for an AI agent
Get clean Markdown of any Hacker News story or comment thread, token-budgeted for LLM context windows.
→ Jina Reader
Fetch a Twitter / X thread for an AI agent
Read JS-rendered Twitter and X threads as clean Markdown, with quoted tweets, reply chains, and token budgeting.
→ FireCrawl (JS-rendered)
Fetch a GitHub README for an AI agent
Read any GitHub repo's README as clean Markdown with token estimation. Skip the rendered HTML, get the raw doc.
→ Trafilatura → Jina fallback
Fetch an arXiv paper for an AI agent
Extract clean Markdown from arXiv abstracts and full PDF papers. Auto-routes between HTML and PDF extractors.
→ Trafilatura (abstract) / pypdf (full paper)
Fetch and extract text from a PDF for an AI agent
Pull text out of any PDF URL with one tool call. No need to download, parse, or handle binary streams yourself.
→ pypdf (local)
Fetch a public Notion page for an AI agent
Read a publicly-shared Notion doc as clean Markdown. Handles Notion's heavy JS rendering automatically.
→ FireCrawl (JS-rendered)
Fetch a Reddit thread for an AI agent
Read any Reddit post + top-level comments as clean Markdown. Bypasses Reddit's gating layer.
→ FireCrawl (JS-rendered)
Fetch a news article for an AI agent
Strip ads, popups, and nav from any news article. Returns just the headline, byline, and body in clean Markdown.
→ Jina Reader
Fetch a docs page for an AI agent
Read any technical documentation page (Read the Docs, MkDocs, GitBook, custom) as token-budgeted Markdown.
→ Trafilatura → Jina fallback
Fetch a public LinkedIn post for an AI agent
Read public LinkedIn posts and articles. Handles LinkedIn's auth wall via JS-rendering.
→ FireCrawl (JS-rendered)
Fetch a Medium article for an AI agent
Read Medium and Substack posts past the paywall preview as clean Markdown.
→ Jina Reader
Fetch an Amazon product page for an AI agent
Extract product title, price, and description from any Amazon listing without dealing with their bot mitigation.
→ FireCrawl (anti-bot bypass)
Fetch a YouTube video metadata + description for an AI agent
Get a YouTube video's title, channel, view count, and full description as text. Doesn't return the video itself.
→ Trafilatura
Fetch a Stack Overflow question for an AI agent
Read any Stack Overflow question + accepted answer as Markdown with code blocks preserved.
→ Trafilatura
Fetch a Wikipedia article for an AI agent
Get any Wikipedia article as clean, token-budgeted Markdown. Skip the navigation, infoboxes, and references.
→ Trafilatura
Fetch a GitHub issue or PR for an AI agent
Read any public GitHub issue or PR thread, including comments and code review, as Markdown.
→ Trafilatura