skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
scraperapi/scraperapi-skills108 installs

scraperapi-ruby-sdk

Best-practices reference for the ScraperAPI Ruby SDK (scraperapi gem). Consult whenever the user is writing, debugging, or reviewing Ruby code that calls ScraperAPI. Use when user asks: "scrape a website with Ruby and ScraperAPI", "ScraperAPI Ruby example", "how do I use the ScraperAPI gem", "Ruby ScraperAPI render", "ScraperAPI Ruby premium proxy", "ScraperAPI gem sessions", "ScraperAPI Ruby error handling". Covers gem setup, all request parameters, the escalation ladder, error handling by status code, and credit costs. Note: Transmits user-supplied queries, URLs, and content to ScraperAPI.

How do I install this agent skill?

npx skills add https://github.com/scraperapi/scraperapi-skills --skill scraperapi-ruby-sdk
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    This skill is a safe instructional guide for using the ScraperAPI Ruby SDK. It follows security best practices by recommending the use of environment variables for API keys and only references official vendor resources.

  • Socketpass

    No alerts

  • Snykwarn

    Risk: MEDIUM · 1 issue

What does this agent skill do?

ScraperAPI — Ruby SDK Best Practices

Requires: Ruby >= 2.0, gem install scraperapi (or gem 'scraperapi' in Gemfile), SCRAPERAPI_API_KEY environment variable.

Setup

require "scraper_api"

client = ScraperAPI::Client.new(ENV["SCRAPERAPI_API_KEY"])

Never hardcode the API key. Read it from the environment every time.

Basic Usage

# Simple GET — returns raw HTML string via .raw_body
html = client.get("https://example.com/").raw_body
puts html

# With a single parameter
html = client.get("https://example.com/", render: true).raw_body

# With multiple parameters
html = client.get(
  "https://example.com/",
  render: true,
  country_code: "us"
).raw_body

Parameters are passed as keyword arguments after the URL. .raw_body extracts the HTML string from the response object.

Decision Guide

SituationApproach
Single URL, synchronousclient.get(url, **params).raw_body
Page loads content via JavaScriptPass render: true
Site blocks datacenter proxiesPass premium: true
Toughest anti-bot protectionPass ultra_premium: true
Multi-step / paginated flow on same domainUse session_number:
20+ URLs or batch jobsUse async endpoint via Net::HTTP or Faraday
Supported platform (Amazon, Google, etc.)Use structured data endpoint directly

Parameter Reference

Rendering

# Render JavaScript before returning HTML
# Use when: page is a React/Vue/Angular SPA, or initial scrape returns empty/partial content
# Cost: +10 credits
html = client.get("https://spa-site.com/", render: true).raw_body

# Wait for a specific DOM element (requires render: true)
html = client.get(
  "https://spa-site.com/",
  render: true,
  wait_for_selector: ".product-list"
).raw_body

Don't add render: true by default — try without it first. It increases cost and latency.

Proxies and Geotargeting

# Route through a country-specific proxy — no extra credit cost
html = client.get("https://example.com/", country_code: "gb").raw_body

# Premium residential/mobile IPs — for sites that block datacenter proxies
# Cost: 10 credits (25 with render: true)
html = client.get("https://hard-site.com/", premium: true).raw_body

# Ultra-premium — for the toughest anti-bot protections
# Cost: 30 credits (75 with render: true)
# Note: incompatible with keep_headers — custom headers are discarded
html = client.get("https://hardest-site.com/", ultra_premium: true).raw_body

premium and ultra_premium are mutually exclusive — never pass both. Escalation order: standard (1 cr) → render (10 cr) → premium (10 cr) → ultra_premium (30 cr).

Sessions (Sticky Proxy)

# Reuse the same proxy IP across requests — useful for pagination and multi-step flows
# Sessions expire 15 minutes after last use; any integer is a valid session ID
html1 = client.get("https://example.com/page1", session_number: 42).raw_body
html2 = client.get("https://example.com/page2", session_number: 42).raw_body

Headers and Device Type

# Forward custom headers to the target site
# Note: keep_headers is ignored when ultra_premium: true
html = client.get(
  "https://example.com/",
  keep_headers: true
  # Pass additional headers via the underlying request object as needed
).raw_body

# Emulate a mobile or desktop browser user-agent
html = client.get("https://example.com/", device_type: "mobile").raw_body

Autoparse

# Return structured JSON instead of HTML for supported sites
# Use for Amazon, Google, and other supported platforms when you want clean data
json_result = client.get("https://amazon.com/dp/B09V3KXJPB", autoparse: true).raw_body

Escalation Ladder

Always start with the cheapest option and escalate only when blocked.

def scrape_with_escalation(client, url)
  tiers = [
    {},
    { render: true },
    { premium: true },
    { premium: true, render: true },
    { ultra_premium: true },
  ]

  tiers.each do |params|
    result = client.get(url, **params).raw_body
    return result if result&.include?("<html")
  end

  nil
end

Async Jobs (for Batches)

The SDK is synchronous — each client.get blocks until the response arrives (up to 70 seconds). For 20+ URLs, submit async jobs via the REST endpoint.

require "net/http"
require "json"

API_KEY = ENV["SCRAPERAPI_API_KEY"]

def submit_job(url, api_params = {})
  uri = URI("https://async.scraperapi.com/jobs")
  payload = { apiKey: API_KEY, url: url, apiParams: api_params }
  response = Net::HTTP.post(uri, payload.to_json, "Content-Type" => "application/json")
  JSON.parse(response.body) # { "id" => "...", "statusUrl" => "..." }
end

def poll_job(job, max_wait: 120, interval: 5)
  deadline = Time.now + max_wait
  while Time.now < deadline
    uri = URI(job["statusUrl"])
    data = JSON.parse(Net::HTTP.get(uri))
    return data.dig("response", "body") if data["status"] == "finished"
    raise "Job #{job['id']} failed" if data["status"] == "failed"
    sleep interval
  end
  raise "Job #{job['id']} timed out"
end

# Submit all URLs, then collect results
urls = ["https://example.com/page1", "https://example.com/page2"]
jobs = urls.map { |u| submit_job(u) }
results = jobs.map { |j| poll_job(j) }

Structured Data Endpoints

For supported platforms, use structured endpoints to get clean JSON without parsing HTML.

require "net/http"
require "json"

def structured_get(vertical, params = {})
  query = URI.encode_www_form({ api_key: API_KEY }.merge(params))
  uri = URI("https://api.scraperapi.com/structured/#{vertical}?#{query}")
  response = Net::HTTP.get_response(uri)
  raise "Error #{response.code}" unless response.is_a?(Net::HTTPSuccess)
  JSON.parse(response.body)
end

# Google SERP
results = structured_get("google/search", { query: "ruby web scraping" })

# Amazon product details
product = structured_get("amazon/product", { asin: "B09V3KXJPB" })

# eBay search
listings = structured_get("ebay/search", { query: "mechanical keyboard" })

Error Handling

The SDK raises exceptions on HTTP errors. Check the status code to determine the right action.

def safe_scrape(client, url, params = {})
  client.get(url, **params).raw_body
rescue => e
  status = e.respond_to?(:response) ? e.response&.code&.to_i : nil
  case status
  when 401 then raise "Invalid API key — check SCRAPERAPI_API_KEY"
  when 403 then raise "Blocked or out of credits — try premium: true or ultra_premium: true"
  when 429 then raise "Rate limit hit — reduce concurrency or switch to async"
  when 500, 503 then raise "Transient error — retry with exponential backoff"
  else raise
  end
end

Status code reference: 200 success, 401 bad key, 403 blocked/no credits, 404 target not found, 429 rate limit, 500/503 transient (not charged — safe to retry).

Credit Cost Reference

Request typeCredits
Standard1
render: true10
premium: true10
premium: true, render: true25
ultra_premium: true30
ultra_premium: true, render: true75

Documentation

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/scraperapi/scraperapi-skills/scraperapi-ruby-sdk">View scraperapi-ruby-sdk on skillZs</a>