Getting started
Two ways to run webclaw. The hosted API needs nothing installed and is below; self-hosting the open-source binary comes after it.
Hosted API
Nothing to install. Sign up at webclaw.io, start your 7-day trial, copy the key from your dashboard, and run this. Card required, cancel anytime. One credit is one page, and that includes JavaScript rendering, protected sites and AI extraction.
curl -X POST https://api.webclaw.io/v1/scrape \
-H "Authorization: Bearer wc_your_key" \
-H "Content-Type: application/json" \
-d '{"url": "https://news.ycombinator.com", "formats": ["markdown"]}'That returns the page as clean Markdown. Swap the URL for one you actually care about, including sites that usually block scrapers. Keys are prefixed with wc_.
webclaw.io/auth.md and it will handle the key exchange on its own.Self-hosting: install the CLI
The CLI, server and MCP server are AGPL-3.0 and run on your own hardware with no account. They do not include the hosted bot-protection handling or residential egress.
Windows x64: prebuilt ZIP
Download the Windows x64 ZIP from GitHub Releases, extract it, and open PowerShell in the folder containing webclaw.exe. This installation does not require Rust.
.\webclaw.exe --version
.\webclaw.exe https://example.com --format markdownPowerShell requires the current-directory prefix shown above. Add the extracted folder to your user PATH to run webclaw from other folders. If Windows reports a missing Visual C++ runtime DLL, install the Microsoft Visual C++ Redistributable for x64. Native Windows ARM64 is not covered.
With Cargo from Git
Requires Rust and native build tools. The package is webclaw-cli and the executable is webclaw; it is not installed from crates.io. A first build can take several minutes.
cargo install --git https://github.com/0xMassi/webclaw.git --tag v0.7.2 --locked webclaw-cliFrom source
Clone the repository and build all three binaries in release mode.
git clone --branch v0.7.2 https://github.com/0xMassi/webclaw.git
cd webclaw
cargo build --release --lockedThe binaries will be at target/release/webclaw, target/release/webclaw-server, and target/release/webclaw-mcp.
Docker
Pull the official image and run the API server in a container.
docker pull ghcr.io/0xmassi/webclaw:latestdocker run --rm -p 127.0.0.1:3000:3000 -e WEBCLAW_API_KEY=local-example-key ghcr.io/0xmassi/webclaw:latest webclaw-serverThe server will be available at http://localhost:3000. Send Authorization: Bearer local-example-key with requests to this example container. Choose your own key for regular use.
Your first extraction
The simplest usage: pass a URL. webclaw extracts the main content and outputs clean markdown by default.
webclaw https://example.comSwitch to LLM-optimized output for the most token-efficient representation. This runs a pipeline that strips images, removes emphasis, deduplicates links, merges stat blocks, and collapses whitespace.
webclaw https://example.com -f llmUse JSON format to get the full ExtractionResult with metadata, content, word count, and extracted URLs.
webclaw https://example.com -f jsonStart the API server
The OSS REST server exposes supported extraction operations over HTTP. Start it on a local port.
webclaw-server --port 3000Test it with a scrape request:
curl -X POST http://localhost:3000/v1/scrape \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'--api-key your_secret to require a Bearer token on all requests.MCP server
The MCP server lets AI agents use webclaw as a tool. It communicates over stdio transport and works with Claude Desktop, Claude Code, Cursor, Windsurf, OpenCode, Codex, Antigravity, and any MCP-compatible client.
Add webclaw to your Claude Desktop configuration at ~/Library/Application Support/Claude/claude_desktop_config.json:
{
"mcpServers": {
"webclaw": {
"command": "npx",
"args": ["-y", "@webclaw/mcp"]
}
}
}npx fetches the launcher automatically — no binary path or build step. To run a source build instead, point command at target/release/webclaw-mcp.
The local MCP server exposes 14 tools:
| Tool | Description |
|---|---|
scrape | Extract content from a single URL |
select | Pick values with CSS, XPath, text or tables |
screenshot | Capture a rendered page as PNG, JPEG, PDF or MHTML |
crawl | BFS crawl a website with depth control |
map | Discover URLs from sitemap.xml and robots.txt |
batch | Extract content from multiple URLs |
extract | LLM-powered JSON schema or prompt extraction |
summarize | LLM-powered content summarization |
diff | Track content changes between snapshots |
brand | Extract brand identity (colors, fonts, logo) |
search | Web search with structured results |
research | Deep multi-source research (needs API key) |
list_extractors | List available site-specific extractors |
vertical_scrape | Run a site-specific extractor for typed JSON |
Cloud API
For managed infrastructure, sign up at webclaw.io and create an API key from the dashboard. Keys are prefixed with wc_.
curl -X POST https://api.webclaw.io/v1/scrape \
-H "Authorization: Bearer wc_your_key" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com", "formats": ["markdown", "llm"]}'Hosted and OSS endpoints overlap, but their options and response shapes differ. In particular, hosted crawls are asynchronous and OSS crawls return results directly. Check the self-hosting capability matrix before switching base URLs.