External LangChain integration

Web retrieval that respects your agent's token budget.

AgentFetch is available as an external integration in the official LangChain tools directory. It turns web pages and PDFs into clean Markdown, estimates size before retrieval, and applies a hard token cap before content reaches your model.

View official listing View on PyPI

Install

pip install langchain-agentfetch

Python 3.10+ · requires an AgentFetch API key

Working LangChain example

Estimate first, then fetch with a 2,000-token ceiling. Both classes are native LangChain tools from the package.

import os
from langchain_agentfetch import AgentFetchTool, EstimateTokensTool

os.environ["AGENTFETCH_API_KEY"] = "af_xxx"
url = "https://news.ycombinator.com"

estimator = EstimateTokensTool()
estimate = estimator.invoke({"url": url})
print(estimate)

fetcher = AgentFetchTool()
content = fetcher.invoke({"url": url, "max_tokens": 2000})
print(content)

Why not a raw webpage loader?

Raw loader

  • Returns navigation, scripts, cookie banners, and footer boilerplate.
  • Discovers page size only after the full response enters your pipeline.
  • Leaves truncation, retries, PDFs, JS rendering, and caching to your code.

AgentFetch

  • Returns cleaned Markdown that is typically 70–85% smaller than raw HTML.
  • Lets the agent estimate tokens before fetching and skip oversized pages.
  • Enforces max_tokens server-side, before content reaches the model context.
  • Caches repeat reads for six hours and auto-routes HTML, JS-heavy pages, and PDFs.

Give your LangChain agent a bounded web tool.

500 free fetches. No credit card.

Get an API key