skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
us/crw64 installs

crw-scrape

Scrape a single known URL into clean markdown / HTML / links / structured JSON with fastCRW. Use when you already have the URL and want the page content — "scrape", "grab", "fetch", "pull", "read this page", "get the content of". Handles JavaScript-rendered SPAs automatically. Step 2 of the crw workflow ladder.

How do I install this agent skill?

npx skills add https://github.com/us/crw --skill crw-scrape
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The skill is safe and functions as intended for web scraping. It retrieves content from external URLs into a clean format using the crw utility or curl, following standard practices for secret management and tool usage.

  • Socketpass

    No alerts

  • Snykwarn

    Risk: MEDIUM · 1 issue

What does this agent skill do?

crw-scrape — single-page extraction

When to use

  • You have one (or a handful of) known URLs and want their content.
  • Step 2 in the crw ladder: if you don't have a URL yet, go to crw-search (step 1) first. For many pages under a site, use crw-crawl (step 4). For a local PDF, use crw-parse (step 5).
  • JS-heavy page? You usually don't need anything special — crw auto-detects and renders. This is a crw advantage: no separate "interact"/browser step.

Quick start

CLI (binary on PATH):

crw scrape "https://example.com"                       # → markdown to stdout
crw scrape "https://example.com" --format json -o page.json
crw scrape "https://example.com" --js --css "article.main"
crw scrape "https://example.com" --format links -o .crw/links.txt

MCP (inside an agent harness):

crw_scrape(url="https://example.com", formats=["markdown"], onlyMainContent=true)

REST (drop-in for Firecrawl SDKs — just swap the base URL):

curl -X POST "$CRW_API_URL/v1/scrape" -H "Authorization: Bearer $CRW_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com","formats":["markdown"],"onlyMainContent":true}'

Options

NeedCLI flagMCP / REST field
Output format--format markdown|html|rawhtml|text|links|jsonformats: [...]
Strip nav/footer/sidebar(on by default; --raw to disable)onlyMainContent: true
Force JS rendering--jsrenderJs: true (null = auto)
Wait after load—waitFor: 2000 (ms)
Keep only selectors--css "article" / --xpath …includeTags: ["article"]
Drop selectors—excludeTags: ["nav","footer"]
Pick renderer—renderer: "auto|lightpanda|chrome|chrome_proxy|playwright" (auto is default)
Save to file-o FILE(write the response yourself)
Structured JSON--extract '<schema>'extract: {schema: {...}} — see crw-extract
Use a proxy--proxy URL --stealthproxy, proxyRotation, stealth

Tips

  • Quote URLs — ? and & are shell-special. Always wrap in quotes.
  • Multiple URLs = run them concurrently. Fire several crw scrape … & and wait, or issue parallel MCP calls.
  • Blank page / loading skeleton? Add --js / renderJs: true, optionally a waitFor. crw's auto-detect covers most SPAs without it.
  • Don't dump huge pages into context. Write to .crw/, then grep/head. MCP truncates to ~15 000 chars (maxLength: 0 to opt out).
  • Want a typed object, not prose? --format json returns the raw full-page object (metadata + content), not schema-extracted data. For structured extraction against a schema use --extract '<schema>' — this calls an LLM and requires a configured LLM provider. See the dedicated crw-extract skill.
  • Source is a file, not a URL? Use crw-parse instead.

See also

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/us/crw/crw-scrape">View crw-scrape on skillZs</a>