Skip to content
Documentation menu

Documentation

CLI reference

Use the lightweight scorch executable to call a running API for search, extraction, discovery, crawling, benchmarking, and MCP.

scorch is an HTTP client. Start scorchd first, or point the client at an existing service with SCORCH_API_URL.

Global options

scorch [--api-url URL] <COMMAND>
OptionDefaultMeaning
--api-url <URL>http://127.0.0.1:33000Scorch HTTP API base URL. Also read from SCORCH_API_URL.
-h, --helpPrint top-level or subcommand help.
-V, --versionPrint the client version.

The global option may appear before the subcommand:

scorch --api-url https://scorch.internal.example search "rust"

scorch scrape

scorch scrape [OPTIONS] <URL>

Fetch or render one public page and print the API response as formatted JSON.

OptionDefaultValues
--formatmarkdownComma-separated: markdown, html, text, links, metadata, screenshot.
--renderautoauto, always, or never.
--browserservice defaultobscura or chromium; must be allowed by server policy.
--timeout-ms30000Total scrape deadline from 100 to 120,000 ms.
--wait-for-ms0Explicit wait after load; server caps it at 60,000 ms.
--full-contentoffKeep the full page instead of focusing extraction on main content.
--full-page-screenshotoffCapture the complete page instead of the viewport.
# Direct-first Markdown and links
scorch scrape https://example.com --format markdown,links

# Force JavaScript rendering with an allowed backend
scorch scrape https://example.com/app \
  --render always \
  --browser obscura \
  --wait-for-ms 500

# Return a full-page PNG data URI
scorch scrape https://example.com \
  --format screenshot \
  --full-page-screenshot
scorch search [OPTIONS] <QUERY>
OptionDefaultMeaning
--limit5Number of results, from 1 to 20.
--scrapeoffEnrich each result through the guarded scrape pipeline.
--countryusCountry hint sent to applicable engines.
--languageenLanguage hint sent to applicable engines.
scorch search "rust async runtime" --limit 8
scorch search "web extraction" --country gb --language en --scrape

There is no engine option. Metasearch routing is controlled exclusively by scorchd.

scorch map

scorch map [OPTIONS] <URL>
OptionDefaultMeaning
--limit100Maximum normalized URLs; server maximum is 1,000.
--include-subdomainsoffAllow discovered links on subdomains of the requested site.
scorch map https://docs.example.com --limit 300 --include-subdomains

scorch crawl

scorch crawl [OPTIONS] <URL>
OptionDefaultMeaning
--limit20Maximum pages; server maximum is 100.
--max-depth2Discovery depth; server maximum is 5.
--concurrency4Per-job concurrency, capped by the service global limit.
--waitoffPoll until the job completes, is cancelled, or fails.
# Start and return the job summary immediately
scorch crawl https://example.com --limit 40 --max-depth 3

# Wait and print the first result page at terminal state
scorch crawl https://example.com --limit 20 --wait

Read crawl status

scorch crawl-status <ID> --cursor 0 --page-size 10

--cursor defaults to 0 and --page-size defaults to 10. API page sizes range from 1 to 50. Use nextCursor from a response to continue.

Cancel and remove a crawl

scorch crawl-cancel <ID>

Cancellation removes the job. A later status request for that ID returns not found.

scorch benchmark

scorch benchmark [--runs N] <URLS>...

The benchmark command calls the configured API, not the engine directly. It runs scrape requests using fetch, auto, and browser modes and prints comparative JSON.

scorch benchmark --runs 5 \
  https://example.com \
  https://example.org

The default is three runs per input. Use optimized scorchd builds when measurements need to represent production behavior.

scorch mcp

SCORCH_API_URL=http://127.0.0.1:33000 scorch mcp

Starts a Model Context Protocol server over stdio. It is designed to be launched by an MCP host rather than used interactively. Every tool call is forwarded over HTTP to the configured API. See the MCP integration guide.

Output and errors

Normal command results are pretty-printed JSON on stdout. Diagnostics are written to stderr. This makes piping reliable:

scorch search "rust scraping" --limit 10 | jq '.results[].url'

HTTP errors include the API error code and message. Every API response carries an x-request-id header; service logs use the same ID for correlation.