MCP Server

Connect WebPeel to Claude Desktop, Cursor, Windsurf, OpenClaw, and other MCP-compatible tools. Give your AI assistant powerful web scraping capabilities.

What is MCP?

The Model Context Protocol (MCP) is an open standard developed by Anthropic that lets AI assistants access external tools and data sources. With WebPeel's MCP server, your AI can:

• Fetch and extract content from any URL
• Search the web for current information
• Crawl entire websites to build knowledge bases
• Extract structured data with CSS selectors or AI prompts
• Monitor pages for changes
• Batch process multiple URLs

Quick Start

Connect to WebPeel via MCP in two ways:

• Hosted MCP endpoint — one URL, Streamable HTTP transport (recommended)
• Local MCP server — run via npx and connect over stdio

Hosted MCP endpoint (recommended)

Use the hosted endpoint at https://api.webpeel.dev/mcp. Any MCP client that supports Streamable HTTP can connect with just:

{
  "url": "https://api.webpeel.dev/mcp",
  "headers": {
    "Authorization": "Bearer <key>"
  }
}

Stateless: no session management required.

Local MCP server

Run WebPeel's MCP server locally via npx — no installation required.

Start the Server

npx webpeel mcp

The local server runs on stdio and is automatically managed by your MCP client.

Configuration

Add WebPeel to your MCP client's configuration file:

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"]
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"]
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"]
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"]
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"]
    }
  }
}

Claude Code (CLI)

For Claude Code, use the one-liner:

claude mcp add webpeel -- npx -y webpeel mcp

Using an API Key

To use WebPeel's hosted API instead of local scraping, set the WEBPEEL_API_KEY environment variable:

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"],
      "env": {
        "WEBPEEL_API_KEY": "your-api-key-here"
      }
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"],
      "env": {
        "WEBPEEL_API_KEY": "your-api-key-here"
      }
    }
  }
}

{
  "mcpServers": {
    "webpeel": {
      "command": "npx",
      "args": ["webpeel", "mcp"],
      "env": {
        "WEBPEEL_API_KEY": "your-api-key-here"
      }
    }
  }
}

Smithery Installation

Install via Smithery registry (one-command setup):

npx @smithery/cli install @webpeel/mcp

Available Tools

WebPeel provides 7 MCP tools covering all common web tasks. Describe your goal in plain English — the smart NL intent parser routes to the right tool automatically.

All 20 legacy tool names are still routed for backward compatibility.

webpeel

General fetch and extract from any URL. The main entry point — give it a URL and a task description and it handles the rest.

Parameters

• url (string, required) — The URL to fetch
• task (string) — Plain-English description of what you want (e.g., "get the article text", "find all prices")
• format (string) — Output format: markdown, text, or html (default: markdown)
• question (string) — Ask a question, get an AI answer about the page
• summary (boolean) — Return a concise extractive summary
• render (boolean) — Force browser rendering (default: false)
• stealth (boolean) — Use stealth mode to bypass bot detection (default: false)

Example

// Ask your AI assistant:
"Use webpeel to get the main article from https://example.com"
"Fetch https://shop.com/product and tell me the price"

webpeel_read

Fetch and read page content as clean text or markdown. Optimized for articles, documentation, and long-form content.

Parameters

• url (string, required) — The URL to read
• format (string) — markdown or text (default: markdown)
• readable (boolean) — Apply reader mode to strip nav/ads (default: true)
• budget (number) — Max token budget for returned content

Example

// Ask your AI assistant:
"Read https://docs.example.com/getting-started and summarize the setup steps"

webpeel_see

Take a screenshot and analyze visuals. Returns a base64 image plus optional design analysis (colors, fonts, layout).

Parameters

• url (string, required) — URL to screenshot
• mode (string) — basic, browser, or design-analysis (default: browser)
• fullPage (boolean) — Capture full scrollable page (default: false)
• width (number) — Viewport width in pixels (default: 1280)

Example

// Ask your AI assistant:
"Screenshot https://stripe.com and tell me what colors they use"
"Take a full-page screenshot of https://example.com"

webpeel_find

Web search — returns titles, URLs, and snippets for a query.

Parameters

• query (string, required) — Search query
• count (number) — Number of results (1–10, default: 5)

Example

// Ask your AI assistant:
"Search for 'best Node.js web scraping libraries 2025'"

webpeel_extract

Structured data extraction with CSS selectors or a JSON schema. Returns typed fields instead of raw text.

Parameters

• url (string, required) — URL to extract from
• selectors (object) — Map field names to CSS selectors (e.g., {"title": "h1", "price": ".price"})
• schema (object) — JSON schema describing expected output
• render (boolean) — Use browser rendering

Example

// Ask your AI assistant:
"Extract the product name, price, and rating from https://shop.com/item/123"

webpeel_monitor

Watch a URL for changes — generates a content fingerprint you can poll to detect updates.

Parameters

• url (string, required) — URL to monitor
• selector (string) — CSS selector to scope monitoring to a specific element
• render (boolean) — Use browser rendering

Example

// Ask your AI assistant:
"Monitor https://example.com/pricing for changes and tell me when the price section updates"

webpeel_act

Interact with dynamic pages — click buttons, fill forms, scroll, and extract after interaction.

Parameters

• url (string, required) — URL to interact with
• actions (array, required) — Array of actions: click, fill, scroll, wait
• extract (object) — What to extract after actions complete

Example

// Ask your AI assistant:
"Go to https://example.com/search, type 'TypeScript' into the search box, click Search, and return the results"

Usage Tips

When to Use Render Mode

• JavaScript-heavy sites (SPAs, React apps)
• Content loaded dynamically after page load
• Sites with lazy-loading images or infinite scroll

When to Use Stealth Mode

• Sites protected by Cloudflare or similar bot detection
• Pages that block headless browsers
• Sites with aggressive rate limiting

Best Practices

1. Start simple — Use basic fetch first, escalate to render/stealth only if needed
2. Use selectors — Extract only what you need with selector and exclude
3. Set maxTokens — Prevent large pages from overwhelming your context window
4. Batch similar requests — Use webpeel_batch for multiple URLs from the same site
5. Respect rate limits — Add delays when crawling large sites

Troubleshooting

Server not starting

Check your MCP client's logs. Common issues:

• npx not in PATH → Install Node.js v18+
• WebPeel package not found → Run npx webpeel mcp manually first
• JSON syntax error → Validate your config file

Tools not appearing

1. Restart your MCP client
2. Check the config file syntax
3. Verify the server starts: npx webpeel mcp

Fetch timing out

• Use render=false for simple HTML pages (faster)
• Increase wait time for slow-loading dynamic content
• Check if the site blocks automated access (try stealth=true)

Next Steps