Documentation menu
Documentation
CLI reference
Use the lightweight scorch executable to call a running API for search, extraction, discovery, crawling, benchmarking, and MCP.
scorch is an HTTP client. Start scorchd first, or point
the client at an existing service with SCORCH_API_URL.
Global options
scorch [--api-url URL] <COMMAND> | Option | Default | Meaning |
|---|---|---|
--api-url <URL> | http://127.0.0.1:33000 | Scorch HTTP API base URL. Also read from SCORCH_API_URL. |
-h, --help | — | Print top-level or subcommand help. |
-V, --version | — | Print the client version. |
The global option may appear before the subcommand:
scorch --api-url https://scorch.internal.example search "rust" scorch scrape
scorch scrape [OPTIONS] <URL> Fetch or render one public page and print the API response as formatted JSON.
| Option | Default | Values |
|---|---|---|
--format | markdown | Comma-separated: markdown, html, text, links, metadata, screenshot. |
--render | auto | auto, always, or never. |
--browser | service default | obscura or chromium; must be allowed by
server policy. |
--timeout-ms | 30000 | Total scrape deadline from 100 to 120,000 ms. |
--wait-for-ms | 0 | Explicit wait after load; server caps it at 60,000 ms. |
--full-content | off | Keep the full page instead of focusing extraction on main content. |
--full-page-screenshot | off | Capture the complete page instead of the viewport. |
# Direct-first Markdown and links
scorch scrape https://example.com --format markdown,links
# Force JavaScript rendering with an allowed backend
scorch scrape https://example.com/app \
--render always \
--browser obscura \
--wait-for-ms 500
# Return a full-page PNG data URI
scorch scrape https://example.com \
--format screenshot \
--full-page-screenshot scorch search
scorch search [OPTIONS] <QUERY> | Option | Default | Meaning |
|---|---|---|
--limit | 5 | Number of results, from 1 to 20. |
--scrape | off | Enrich each result through the guarded scrape pipeline. |
--country | us | Country hint sent to applicable engines. |
--language | en | Language hint sent to applicable engines. |
scorch search "rust async runtime" --limit 8
scorch search "web extraction" --country gb --language en --scrape
There is no engine option. Metasearch routing is controlled exclusively by scorchd.
scorch map
scorch map [OPTIONS] <URL> | Option | Default | Meaning |
|---|---|---|
--limit | 100 | Maximum normalized URLs; server maximum is 1,000. |
--include-subdomains | off | Allow discovered links on subdomains of the requested site. |
scorch map https://docs.example.com --limit 300 --include-subdomains scorch crawl
scorch crawl [OPTIONS] <URL> | Option | Default | Meaning |
|---|---|---|
--limit | 20 | Maximum pages; server maximum is 100. |
--max-depth | 2 | Discovery depth; server maximum is 5. |
--concurrency | 4 | Per-job concurrency, capped by the service global limit. |
--wait | off | Poll until the job completes, is cancelled, or fails. |
# Start and return the job summary immediately
scorch crawl https://example.com --limit 40 --max-depth 3
# Wait and print the first result page at terminal state
scorch crawl https://example.com --limit 20 --wait Read crawl status
scorch crawl-status <ID> --cursor 0 --page-size 10 --cursor defaults to 0 and --page-size defaults to 10.
API page sizes range from 1 to 50. Use nextCursor from a response
to continue.
Cancel and remove a crawl
scorch crawl-cancel <ID> Cancellation removes the job. A later status request for that ID returns not found.
scorch benchmark
scorch benchmark [--runs N] <URLS>... The benchmark command calls the configured API, not the engine directly. It runs scrape requests using fetch, auto, and browser modes and prints comparative JSON.
scorch benchmark --runs 5 \
https://example.com \
https://example.org
The default is three runs per input. Use optimized scorchd builds
when measurements need to represent production behavior.
scorch mcp
SCORCH_API_URL=http://127.0.0.1:33000 scorch mcp Starts a Model Context Protocol server over stdio. It is designed to be launched by an MCP host rather than used interactively. Every tool call is forwarded over HTTP to the configured API. See the MCP integration guide.
Output and errors
Normal command results are pretty-printed JSON on stdout. Diagnostics are written to stderr. This makes piping reliable:
scorch search "rust scraping" --limit 10 | jq '.results[].url'
HTTP errors include the API error code and message. Every API response
carries an x-request-id header; service logs use the same ID for
correlation.