Deep Fetch

Search the web, fetch the top results, and synthesize content from multiple sources into a single coherent output — in one API call.

vs. Research Agents: Deep Fetch is simpler and faster — it searches, fetches, and merges content without LLM inference. Use Research Agents when you need an LLM to reason, synthesize, or answer a question across sources.

Endpoint

POST/v1/deep-fetchAuth Required

Search the web for a query, fetch the top results, and return synthesized content in the chosen format.

Request Body

Parameter	Type	Default	Description
query	string	—	Required. The search query. Results are fetched and combined.
count	number	`5`	Number of search results to fetch. Range: 1–10.
format	string	`"merged"`	Output format. See Formats below.
maxChars	number	`32000`	Maximum total characters in the combined output.

Output Formats

merged

All fetched pages are combined into a single continuous Markdown document, with source headers between sections. Good for passing to an LLM as context.

structured

Returns an array of individual source objects, each with title, URL, and content. Good for processing sources separately.

comparison

Returns sources side-by-side with overlap analysis — useful for comparing how different sources cover the same topic.

Response

{
  "query": "WebPeel API pricing",
  "format": "merged",
  "count": 3,
  "sources": [
    { "url": "https://webpeel.dev/pricing", "title": "Pricing — WebPeel" },
    { "url": "https://webpeel.dev/docs", "title": "Documentation — WebPeel" }
  ],
  "content": "# Source 1: Pricing — WebPeel\nhttps://webpeel.dev/pricing\n\nWebPeel offers a generous free tier...\n\n---\n\n# Source 2: Documentation — WebPeel\n..."
}

{
  "query": "WebPeel API pricing",
  "format": "structured",
  "count": 2,
  "sources": [
    {
      "url": "https://webpeel.dev/pricing",
      "title": "Pricing — WebPeel",
      "content": "WebPeel offers a generous free tier with 1,000 requests/month...",
      "relevanceScore": 0.94
    },
    {
      "url": "https://webpeel.dev",
      "title": "WebPeel — Fast Web Fetching for AI",
      "content": "# WebPeel\n\nSmart web scraping API...",
      "relevanceScore": 0.81
    }
  ]
}

{
  "query": "WebPeel API pricing",
  "format": "comparison",
  "count": 2,
  "sources": [
    { "url": "https://webpeel.dev/pricing", "title": "Pricing — WebPeel", "content": "..." },
    { "url": "https://competitor.com/pricing", "title": "Competitor Pricing", "content": "..." }
  ],
  "comparison": {
    "overlap": ["monthly pricing", "free tier", "API credits"],
    "unique": {
      "source1": ["bulk discounts", "enterprise plan"],
      "source2": ["per-seat pricing", "trial period"]
    }
  }
}

Examples

curl -X POST https://api.webpeel.dev/v1/deep-fetch \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "best JavaScript web scraping libraries 2024",
    "count": 5,
    "format": "merged",
    "maxChars": 20000
  }'

curl -X POST https://api.webpeel.dev/v1/deep-fetch \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "OpenAI GPT-4 pricing",
    "count": 3,
    "format": "structured"
  }'

// Use deep-fetch to gather context before passing to an LLM
const res = await fetch('https://api.webpeel.dev/v1/deep-fetch', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.WEBPEEL_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    query: 'React Server Components best practices 2024',
    count: 5,
    format: 'merged',
    maxChars: 30000,
  }),
});
const result = await res.json();

// Pass merged content to your LLM
const answer = await openai.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [
    { role: 'system', content: 'Answer based on the following web research context.' },
    { role: 'user', content: result.content },
    { role: 'user', content: 'What are the top 3 best practices for React Server Components?' },
  ],
});
console.log(answer.choices[0].message.content);

Use Cases

RAG context gathering: Fetch relevant web content before calling your LLM
Competitive research: Gather pricing/feature info from multiple sources simultaneously
Content aggregation: Merge news or blog content on a topic
Fact checking: Cross-reference multiple sources for a claim

Deep Fetch vs. Alternatives

Feature	Deep Fetch	Search API	Research Agents
Searches web	✅	✅	✅
Fetches page content	✅	Optional	✅
LLM synthesis	❌ No LLM	❌	✅ BYOK
Multi-step reasoning	❌	❌	✅
Structured JSON output	❌	✅	✅
Best for	Context gathering	URL discovery	Answering questions

Limits

Max results: 10 per request
Max output: 32,000 characters (configurable via maxChars)
Credits: 1 credit per source URL fetched