Documentation / Getting Started

Getting Started with WebPeel

WebPeel is an AI-ready web fetching API that turns any URL into clean, structured content — markdown, text, HTML, screenshots, or structured JSON — so your AI apps and agents can work with the web without the noise.

What is WebPeel?

WebPeel handles the messy parts of web fetching: JavaScript rendering, bot detection, content extraction, and token optimization. Instead of writing Playwright scripts or fighting with HTML parsers, you make one API call and get back clean content your LLM can actually use. It works as a REST API, Node.js/Python SDK, CLI tool, and MCP server for AI assistants.

Install

Pick your preferred integration:

Node.js SDK

npm install webpeel

Python SDK

pip install webpeel

CLI (no install required)

# Verb-first aliases — fetch, get, scrape, and peel all work
npx webpeel fetch https://example.com
npx webpeel get https://example.com
npx webpeel scrape https://example.com

# Control output format
npx webpeel fetch https://example.com --format markdown
npx webpeel fetch https://example.com --format text
npx webpeel fetch https://example.com --format json

# Diagnose your setup
npx webpeel doctor

The webpeel doctor command checks your API key, network connectivity, and configuration — run it first if anything isn't working.

Your First Fetch

Here's a complete example fetching a page and getting back clean markdown:

curl

curl "https://api.webpeel.dev/v1/fetch?url=https://example.com" \
  -H "Authorization: Bearer wp_live_xxxx"

Node.js

import { WebPeel } from 'webpeel';

const wp = new WebPeel({ apiKey: 'wp_live_xxxx' });

const result = await wp.fetch('https://example.com');
console.log(result.content); // Clean markdown
console.log(result.title);   // Page title
console.log(result.tokens);  // Token count

Python

from webpeel import WebPeel

wp = WebPeel(api_key="wp_live_xxxx")

result = wp.fetch("https://example.com")
print(result.content)  # Clean markdown
print(result.title)    # Page title
print(result.tokens)   # Token count

Get Your API Key

You'll need an API key to authenticate requests.

  1. Sign up (free) at app.webpeel.dev/signup
  2. Navigate to app.webpeel.dev/keys
  3. Click Create Key — copy and save it immediately (shown once)

Your key starts with wp_live_ for production or wp_test_ for sandbox. Pass it in the Authorization header on every request.

LLM-Free Web Q&A

The /v1/ask endpoint answers questions from the live web — no LLM API key needed. It searches, fetches, and ranks the best sentences using BM25 scoring.

# Ask a question from the CLI
npx webpeel webask "What is the capital of France?"

# Or via the REST API
curl "https://api.webpeel.dev/v1/ask?q=What+is+BM25" \
  -H "Authorization: Bearer wp_live_xxxx"

See the /v1/ask documentation for the full response format and options.

Next Steps