crw-extract
Extract a typed JSON object from one or more web pages against a JSON Schema with fastCRW. Use when you need structured data — "get the price and stock status", "extract all job listings as JSON", "pull structured fields from this page". Step 6 of the crw workflow ladder.
How do I install this agent skill?
npx skills add https://github.com/us/crw --skill crw-extractIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill provides instructions for extracting structured JSON from web pages using the fastCRW tool. The only identified concern is the inherent risk associated with processing untrusted content from external websites, which could potentially contain malicious instructions designed to influence the extraction process (indirect prompt injection).
- Socketpass
No alerts
- Snykwarn
Risk: MEDIUM · 1 issue
What does this agent skill do?
crw-extract — typed JSON from pages
When to use
- You need a structured JSON object from a page, not prose.
- Step 6 in the crw ladder. First scrape succeeds (step 2),
but you need machine-readable fields. If the source is a local PDF, use
crw-parse (step 5) with
formats:["json"]instead. - Schema-driven = deterministic output shape. No schema = exploratory; use a
promptto describe what you want.
How extraction works in crw
crw has no /v1/extract endpoint (Firecrawl's dedicated extract API). Instead,
extraction runs in two ways, both backed by the same LLM pipeline:
| Path | When to use | Sync? |
|---|---|---|
Per-page: formats:["json"] + jsonSchema on scrape | Single URL, or inline during a crawl | Synchronous |
Async multi-URL: POST /v2/extract → poll GET /v2/extract/{id} | Many URLs, fire-and-forget | Async |
/v2/extract is marked deprecated in the server (it recommends /v2/scrape
with formats:["json"]), but it works and is useful for multi-URL batches.
Requires a server-side LLM. Set [extraction.llm] in the server config
(provider, api_key, model). Without it, requests return an error (HTTP 4xx) —
e.g. 422 "no LLM configured". Use crw setup to configure the LLM for the CLI.
Quick start
CLI — per-page extraction via --extract:
# Inline schema
crw scrape "https://example.com/product" \
--extract '{"type":"object","properties":{"price":{"type":"number"},"inStock":{"type":"boolean"}}}'
# Schema from file
crw scrape "https://example.com/job" --extract @schema.json -o result.json
MCP — pass formats:["json"] with jsonSchema on a scrape:
crw_scrape(
url="https://example.com/product",
formats=["json"],
extract={"schema": {"type":"object","properties":{"price":{"type":"number"}}}}
)
Note: the MCP crw_scrape accepts extract.schema (Firecrawl style). The
REST API also accepts jsonSchema as a top-level alias.
REST — per-page (synchronous):
curl -X POST "$CRW_API_URL/v1/scrape" \
-H "Authorization: Bearer $CRW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com/product",
"formats": ["json"],
"jsonSchema": {
"type": "object",
"properties": {
"price": {"type": "number"},
"inStock": {"type": "boolean"}
}
}
}'
REST — async multi-URL (deprecated endpoint, still functional):
# Start job
curl -X POST "$CRW_API_URL/v2/extract" \
-H "Authorization: Bearer $CRW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://example.com/p1", "https://example.com/p2"],
"schema": {"type":"object","properties":{"price":{"type":"number"}}}
}'
# → {"success":true,"id":"<uuid>","warnings":["...use /v2/scrape..."], ...}
# Poll until completed
curl "$CRW_API_URL/v2/extract/<uuid>" -H "Authorization: Bearer $CRW_API_KEY"
# → {"success":true,"status":"completed|scraping|failed","data":{...}}
Options
| Need | CLI | MCP / REST |
|---|---|---|
| JSON schema | --extract '<schema>' or @file.json | jsonSchema / extract.schema |
| Free-text prompt (no schema) | — | prompt on /v2/extract |
| Save output | -o FILE | write the response yourself |
| Multi-URL async | not available | POST /v2/extract with urls:[...] |
| LLM override | --llm-provider, --llm-key, --llm-model | server config only |
Tips
- Schema = deterministic, prompt = exploratory. A schema pins the output
shape; a free-text
promptis useful for exploration but less reliable. Start with a schema when you know the fields you want. - Don't over-specify. Narrow schemas ("give me exactly these three fields") extract more reliably than wide ones with fifty optional fields.
- Crawl + extract in one pass.
crw_crawlaccepts ajsonSchemaparameter — each page in the crawl gets extracted against the schema, saving a second round-trip. - Check
data.jsonin the scrape response. Per-page extraction lands indata.json, notdata.markdown. Themarkdownfield is also populated for reference.
See also
- crw-scrape — scrape a page without schema extraction
- crw-parse — extract structured data from a local PDF
- crw-best-practices — SDK usage patterns
- crw — ladder overview
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/us/crw/crw-extract">View crw-extract on skillZs</a>