agent-browser-scrape
[INTERNAL TEMPLATE] Data extraction execution protocol. Called by agent-browser orchestrator.
How do I install this agent skill?
npx skills add https://github.com/owenleezy/agent-browser-skill --skill agent-browser-scrapeIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill provides a protocol for web scraping using the agent-browser tool. It defines procedures for data extraction, pagination handling, and anti-bot evasion. No security issues were detected.
- Socketwarn
1 alert: gptAnomaly
- Snykwarn
Risk: MEDIUM · 1 issue
What does this agent skill do?
Data Extraction — Execution Protocol
Internal template. Entry point:
agent-browserskill.
NEVER
NEVER `eval` before `wait --load networkidle` — JS-rendered content won't exist yet
NEVER assume lightpanda handles JS-heavy pages — always have a --engine chrome fallback ready
NEVER scrape without checking `get count` first — validates that your selector found items
Execution Sequence
# 1. Try lightpanda first (fastest, no JS overhead)
AGENT_BROWSER_ENGINE=lightpanda agent-browser open <url>
AGENT_BROWSER_ENGINE=lightpanda agent-browser wait --load networkidle
# 2. Understand structure
AGENT_BROWSER_ENGINE=lightpanda agent-browser snapshot -c
# 3. Verify selector finds items before extracting
AGENT_BROWSER_ENGINE=lightpanda agent-browser get count ".item"
# 4. Extract
AGENT_BROWSER_ENGINE=lightpanda agent-browser eval "[...document.querySelectorAll('.item')].map(el => ({
name: el.querySelector('h2')?.textContent?.trim(),
price: el.querySelector('.price')?.textContent?.trim()
}))"
# 5. If empty/wrong → fallback to chrome engine
agent-browser open <url>
agent-browser wait --load networkidle
agent-browser get count ".item" # verify selector still works
agent-browser eval "<same query>"
Note on
AGENT_BROWSER_NATIVE=1: This env var was used in older workflows as a shorthand for the Rust daemon. In 0.22+, the daemon is always active — useAGENT_BROWSER_ENGINE=chrome(or just default) instead.AGENT_BROWSER_NATIVE=1is still accepted but no longer documented.
Pagination Handling
# Scroll-based infinite scroll
agent-browser scroll down
agent-browser wait --load networkidle
agent-browser get count ".item" # check if count increased
agent-browser eval "<extract again>"
# Repeat until count stops increasing
# Click-based pagination
agent-browser find role button click --name "Next"
agent-browser wait --load networkidle
# Wait for specific text to confirm page loaded
agent-browser wait --text "Page 2 of"
Output Parsing Tips
# Add --content-boundaries to help parse output when using --max-output
AGENT_BROWSER_CONTENT_BOUNDARIES=1 agent-browser eval "<query>" --max-output 50000
# JSON output for structured parsing
agent-browser get text @e1 --json
agent-browser snapshot -i --json
Anti-Bot Bypass
# Level 1: custom user-agent
agent-browser --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36" open <url>
# Level 2: disable automation detection (combine with level 1)
agent-browser \
--user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36" \
--args "--disable-blink-features=AutomationControlled" \
open <url>
Recovery Patterns
| Failure | Recovery |
|---|---|
| lightpanda returns empty | Switch to --engine chrome (or default) |
| Detected as bot | Add --user-agent, then combine with --args |
eval before content loads | wait --load networkidle before every eval |
| Infinite scroll not handled | scroll down + re-eval loop, check get count for progress |
| Output truncated | Add --max-output 100000 or AGENT_BROWSER_MAX_OUTPUT env |
Judgment Gates
Pause when:
- Anti-bot measures block all approaches (CAPTCHA requires human)
get countreturns 0 after fallback — showsnapshot -cto user, ask how to identify items
Pitfalls
| Mistake | Fix |
|---|---|
| lightpanda returns empty | Switch to --engine chrome |
eval before load | wait --load networkidle first |
Bot detected despite --user-agent | Add --args "--disable-blink-features=AutomationControlled" |
| Extracted 0 items silently | Always get count "<selector>" before eval |
| Output cut off mid-JSON | Increase --max-output or use --content-boundaries |
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/owenleezy/agent-browser-skill/agent-browser-scrape">View agent-browser-scrape on skillZs</a>