What's the cheapest way to scrape the web for an LLM agent?

The cheapest way to scrape the web for an LLM agent is to combine a free or low-cost markdown-converting fetch tool (AgentFetch free tier, Jina Reader) with aggressive caching and HTML preprocessing before sending to the model. The dominant cost is *model tokens*, not scraping infra — at Claude Sonnet $3/M input pricing, raw HTML is 5-10x more expensive than clean markdown for the same content. So step one is using a tool that returns markdown. Step two is caching: if your agent re-reads the same URL within an hour, hit a local cache instead of refetching. Step three is selecting the right tier: Jina Reader is free for hobby volume, AgentFetch has a free tier sized for agent workloads, Firecrawl Hobby is $19/mo for 3,000 pages, Apify scales lower per-page at high volume. Self-hosting a fetch service on a $5/mo VPS plus readability-lxml is the absolute cheapest at scale but burns engineering time. For most builders, AgentFetch's free tier + caching + clean-markdown output is the lowest total cost (compute + tokens + dev time) under ~10k fetches/month.