Documentation / Search
💡 Try this endpoint in the Playground →

Search API

Search the web and return structured results. Supports web, news, and image searches. Optionally scrape the content of each result page.

Free by default: WebPeel uses DuckDuckGo by default — no search API key needed. For higher-quality results, pass provider=brave with your Brave Search API key. Results are cached for 15 minutes.

Endpoint

GET/v1/searchAuth Required

Search the web and return structured results with optional page scraping.

Query Parameters

ParameterTypeDefaultDescription
q string — Required. Search query.
count number 5 Number of results per source (1–10).
sources string web Comma-separated sources: web, news, images. Example: web,news.
scrapeResults boolean false If true, fetches and includes the full Markdown content of each result URL.
provider string auto Search provider: auto, duckduckgo, brave, google, stealth.
searchApiKey string — API key for paid providers (Brave, Google). Also via X-Search-Api-Key header.
categories string — Filter results by category (comma-separated): github, docs, blog, news, video, social, pdf.
tbs string — Time filter (Google-style): qdr:h (past hour), qdr:d (past day), qdr:w (past week).
country string — Country code for localized results (e.g. US, GB, DE).
location string — Location string for geo-targeted results (e.g. New York, NY).

Response

Results for each requested source are returned under their respective keys:

{
  "success": true,
  "data": {
    "web": [
      {
        "title": "WebPeel — Fast Web Fetching for AI",
        "url": "https://webpeel.dev",
        "snippet": "Smart web scraping that auto-escalates from HTTP to headless browser...",
        "content": "# WebPeel\n\nFast web fetching for AI agents...",
        "relevanceScore": 0.87
      }
    ],
    "news": [
      {
        "title": "WebPeel Launches New Batch API",
        "url": "https://techcrunch.com/...",
        "snippet": "...",
        "source": "TechCrunch",
        "date": "2024-03-04"
      }
    ],
    "images": [
      {
        "title": "WebPeel Logo",
        "url": "https://example.com/image.png",
        "thumbnail": "...",
        "source": "example.com"
      }
    ]
  }
}

Each web result includes a relevanceScore (0–1) when scrapeResults=true: a BM25-based score of how closely the fetched page content matches your query. Use this to rank or filter results before passing them to your LLM.

Response Headers

HeaderDescription
X-CacheHIT or MISS — whether results were served from cache.
X-Cache-AgeSeconds since this result was cached (only on HIT).
X-Credits-UsedCredits consumed by this request (0 on cache hit).

Examples

curl "https://api.webpeel.dev/v1/search?q=WebPeel+API&count=5" \
  -H "Authorization: Bearer YOUR_API_KEY"
# Search web + news in one request
curl "https://api.webpeel.dev/v1/search?q=AI+news+2024&sources=web,news&count=5" \
  -H "Authorization: Bearer YOUR_API_KEY"
# Return full markdown content for each result page
curl "https://api.webpeel.dev/v1/search?q=web+scraping+API&scrapeResults=true&count=3" \
  -H "Authorization: Bearer YOUR_API_KEY"
# Use Brave Search for higher-quality results (BYOK)
curl "https://api.webpeel.dev/v1/search?q=WebPeel&provider=brave&searchApiKey=YOUR_BRAVE_KEY&count=10" \
  -H "Authorization: Bearer YOUR_API_KEY"

# Or use a header for the search API key
curl "https://api.webpeel.dev/v1/search?q=WebPeel&provider=brave&count=10" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Search-Api-Key: YOUR_BRAVE_KEY"
const params = new URLSearchParams({
  q: 'best web scraping APIs 2024',
  count: '5',
  sources: 'web,news',
  categories: 'docs,github',
});

const res = await fetch(`https://api.webpeel.dev/v1/search?${params}`, {
  headers: { 'Authorization': `Bearer ${process.env.WEBPEEL_API_KEY}` },
});
const { data } = await res.json();

console.log(`Found ${data.web?.length} web results, ${data.news?.length} news results`);
data.web?.forEach(r => console.log(r.title, r.url));

Caching

Search results are cached for 15 minutes using an LRU cache (max 500 entries, 50MB). Cache hits are free — no credits consumed. The cache key includes all query parameters, so different parameter combinations are cached separately.

Search Providers

ProviderAPI KeyQualityNotes
duckduckgoNot requiredGoodDefault free provider. HTML scraping of DDG results.
braveBrave API key (BYOK)ExcellentBest quality. Supports tbs, country, location filtering.
googleGoogle API key (BYOK)ExcellentGoogle Custom Search JSON API.
stealthNot requiredGoodStealthy DDG variant for restricted environments.
autoOptionalBest availableAutomatically selects the best provider. Uses SearXNG (self-hosted meta-search aggregating Google, Bing, Brave, Startpage, and more) when available, then falls back to DuckDuckGo or configured keys.

Need an answer, not just a list of results? Use /v1/ask — it combines search + fetching + BM25 scoring to return the best passage from the top results, with enriched source metadata.

Unlike /v1/search which returns raw results, /v1/ask ranks sources using four signals:

SignalWeightDescription
BM25 relevance 40% Term-frequency score of the best matching passage in the fetched page content
Domain authority 25% Tiered trust score: official (.gov, docs.*, official product pages) → institutional (.org, major news) → major (TechCrunch, Ars) → general
Freshness 20% Page publish date from Open Graph / article metadata: recent (≤30 days), this-month (≤90 days), this-year, older. Weight doubles for pricing/version queries.
Primary source 15% Whether the source domain matches the query entity (e.g. openai.com for "OpenAI API pricing")

Each source in the /v1/ask response includes these fields:

{
  "url": "https://platform.openai.com/docs/pricing",
  "title": "Pricing — OpenAI Platform",
  "snippet": "GPT-4o input: $2.50 / 1M tokens. Output: $10.00 / 1M tokens.",
  "confidence": 0.91,
  "authority": "official",
  "freshness": "recent",
  "isPrimarySource": true
}

→ Read the full /v1/ask documentation

See Also