skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
us/crw63 installs

crw-extract

Extract a typed JSON object from one or more web pages against a JSON Schema with fastCRW. Use when you need structured data — "get the price and stock status", "extract all job listings as JSON", "pull structured fields from this page". Step 6 of the crw workflow ladder.

How do I install this agent skill?

npx skills add https://github.com/us/crw --skill crw-extract
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The skill provides instructions for extracting structured JSON from web pages using the fastCRW tool. The only identified concern is the inherent risk associated with processing untrusted content from external websites, which could potentially contain malicious instructions designed to influence the extraction process (indirect prompt injection).

  • Socketpass

    No alerts

  • Snykwarn

    Risk: MEDIUM · 1 issue

What does this agent skill do?

crw-extract — typed JSON from pages

When to use

  • You need a structured JSON object from a page, not prose.
  • Step 6 in the crw ladder. First scrape succeeds (step 2), but you need machine-readable fields. If the source is a local PDF, use crw-parse (step 5) with formats:["json"] instead.
  • Schema-driven = deterministic output shape. No schema = exploratory; use a prompt to describe what you want.

How extraction works in crw

crw has no /v1/extract endpoint (Firecrawl's dedicated extract API). Instead, extraction runs in two ways, both backed by the same LLM pipeline:

PathWhen to useSync?
Per-page: formats:["json"] + jsonSchema on scrapeSingle URL, or inline during a crawlSynchronous
Async multi-URL: POST /v2/extract → poll GET /v2/extract/{id}Many URLs, fire-and-forgetAsync

/v2/extract is marked deprecated in the server (it recommends /v2/scrape with formats:["json"]), but it works and is useful for multi-URL batches.

Requires a server-side LLM. Set [extraction.llm] in the server config (provider, api_key, model). Without it, requests return an error (HTTP 4xx) — e.g. 422 "no LLM configured". Use crw setup to configure the LLM for the CLI.

Quick start

CLI — per-page extraction via --extract:

# Inline schema
crw scrape "https://example.com/product" \
  --extract '{"type":"object","properties":{"price":{"type":"number"},"inStock":{"type":"boolean"}}}'

# Schema from file
crw scrape "https://example.com/job" --extract @schema.json -o result.json

MCP — pass formats:["json"] with jsonSchema on a scrape:

crw_scrape(
  url="https://example.com/product",
  formats=["json"],
  extract={"schema": {"type":"object","properties":{"price":{"type":"number"}}}}
)

Note: the MCP crw_scrape accepts extract.schema (Firecrawl style). The REST API also accepts jsonSchema as a top-level alias.

REST — per-page (synchronous):

curl -X POST "$CRW_API_URL/v1/scrape" \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/product",
    "formats": ["json"],
    "jsonSchema": {
      "type": "object",
      "properties": {
        "price": {"type": "number"},
        "inStock": {"type": "boolean"}
      }
    }
  }'

REST — async multi-URL (deprecated endpoint, still functional):

# Start job
curl -X POST "$CRW_API_URL/v2/extract" \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com/p1", "https://example.com/p2"],
    "schema": {"type":"object","properties":{"price":{"type":"number"}}}
  }'
# → {"success":true,"id":"<uuid>","warnings":["...use /v2/scrape..."], ...}

# Poll until completed
curl "$CRW_API_URL/v2/extract/<uuid>" -H "Authorization: Bearer $CRW_API_KEY"
# → {"success":true,"status":"completed|scraping|failed","data":{...}}

Options

NeedCLIMCP / REST
JSON schema--extract '<schema>' or @file.jsonjsonSchema / extract.schema
Free-text prompt (no schema)—prompt on /v2/extract
Save output-o FILEwrite the response yourself
Multi-URL asyncnot availablePOST /v2/extract with urls:[...]
LLM override--llm-provider, --llm-key, --llm-modelserver config only

Tips

  • Schema = deterministic, prompt = exploratory. A schema pins the output shape; a free-text prompt is useful for exploration but less reliable. Start with a schema when you know the fields you want.
  • Don't over-specify. Narrow schemas ("give me exactly these three fields") extract more reliably than wide ones with fifty optional fields.
  • Crawl + extract in one pass. crw_crawl accepts a jsonSchema parameter — each page in the crawl gets extracted against the schema, saving a second round-trip.
  • Check data.json in the scrape response. Per-page extraction lands in data.json, not data.markdown. The markdown field is also populated for reference.

See also

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/us/crw/crw-extract">View crw-extract on skillZs</a>