skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
boringmarketer/boring-marketing-skills114 installs

site-audit

Crawls a website the way Google and the AI crawlers see it and returns a short, ranked fix list in plain words. It covers pages blocked from Google, noindex and canonical mistakes, AI crawlers blocked in robots.txt or at the CDN, sitemap health, broken links, duplicate and thin pages, Core Web Vitals from Google's free PageSpeed API, and the content signals that make a page worth citing. Use for a technical SEO audit, when pages aren't showing up on Google, to check whether AI crawlers can reach the site, or to re-check a site after fixes and see what changed.

How do I install this agent skill?

npx skills add https://github.com/boringmarketer/boring-marketing-skills --skill site-audit
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The site-audit skill is a technical SEO tool used to crawl websites and report on indexing, performance, and AI-readiness. It is well-structured and uses official APIs from Google and Firecrawl. The primary security consideration is a low risk of indirect prompt injection from processing external web content.

  • Socketpass

    No alerts

  • Snykwarn

    Risk: MEDIUM · 1 issue

What does this agent skill do?

Site Audit

Most sites that don't get found have a short list of reasons, and a 75-row audit table hides them. This skill crawls the site politely, the way Google and the AI crawlers meet it, and hands back the five to ten fixes that matter most for getting found, each backed by what the crawl saw. Done means an owner or their developer can act on the list today, and anyone can re-check every claim on it. The checks and the ranking below are a floor. When the business in front of you argues for a different order, change it and say why in one line.

Context

Only if ./brand/, ./brands/ or MARKETING.md exists (nothing to load otherwise), run: python3 ${CLAUDE_SKILL_DIR}/../start-here/scripts/brand_context.py --for site-audit (${CLAUDE_SKILL_DIR} is this skill's folder; outside Claude Code use the folder this file was read from, e.g. .agents/skills/site-audit. If the script or python3 is unavailable, read the brand folder directly: ./brands/<slug>/ when ./brands/active names one, else ./brand/; then MARKETING.md, preferences.md and the files in reads:. If none exist, proceed standalone and ask only for the URL.)

The business facts in brand.yaml (services, towns, category) and the keyword plan tell you which pages make money. Use them to judge severity. Without them, infer money pages from the menu and the URL paths (services, locations, products, pricing, booking, quote).

Auditing a prospect's site rather than the user's own? Don't load or write brand files for it. The report lands in ./campaigns/ and nothing else changes.

No URL, or only a placeholder (example.com, yourdomain.com)? Ask for the URL in one line. Never crawl a placeholder.

What the evidence says

Load references/crawler-facts.md before describing any crawler, CDN or PageSpeed behavior (check its header; refresh it if stale). The judgment that doesn't expire:

  • Being crawlable and indexable comes first. One Disallow or noindex on a money page outweighs every other item on the list. Most real audits turn up two or three of these, and a pile of cosmetic warnings.
  • Classic organic rank is upstream of AI citation. On Google's AI answers most cited pages already rank, and other engines lean on their own search indexes. So AI readiness mostly means letting the AI search crawlers in, putting the text in the HTML, and getting the same fundamentals right. Never promise that a technical fix produces citations.
  • Optional hygiene stays optional. llms.txt and generic structured data are cheap and unproven. They go in "also worth doing", never at the top. Schema earns its place only where the page content supports it, and a business markup block that names the business consistently is the one piece worth adding everywhere (references/schema-and-entity.md).
  • Content signals with evidence behind them are reported for articles: statistics, quotations and cited sources (the one controlled study of generative answers found these lift visibility), visible publish and updated dates, a named author, and answer capsules (a question heading followed by a short, self-contained, link-light answer). Signals, not rules. A missing date on a blog post is never a top-five fix.
  • Severity is business impact, not how many pages a warning touches. 300 images without alt text matter less than one service page marked noindex.

Modes

The user saysModeWhat runs
a URL, "audit my site's SEO", "technical SEO audit"Full (default)Up to 200 pages (--max-pages, hard cap 1,000), the AI crawler checks, PageSpeed on the homepage and three key templates
"quick", "is anything blocking Google", "are AI crawlers blocked"QuickHomepage plus about 20 pages, robots and AI crawler access, PageSpeed on the homepage
"re-audit", "what changed", "check again after the fixes"Re-auditFull or quick, compared with the last findings file

A narrow question ("is GPTBot blocked?") gets Quick mode and a few lines of answer, with the findings still saved. start-here owns the marketing review ("review my site", "audit my site"); this skill is the technical half.

Phase 1: Set up the run

Settle three things before crawling, from the brand files or the request. Ask only what you can't infer, in one message.

  1. The site and the money pages. The homepage URL, and the pages that bring customers (service, location, product, pricing, booking). These set severity later.
  2. Does the business want AI visibility? Default yes for any business that wants customers. If robots.txt blocks AI search crawlers on purpose (a publisher protecting its content, say), ask once before calling it a problem. Blocking AI training crawlers is a business choice. Report it once, never as a fix.
  3. Time and politeness. A full crawl of 200 pages takes about 2 to 6 minutes, plus 1 to 3 minutes of PageSpeed calls. Quick takes about a minute. The crawler names itself in its user agent, honors robots.txt for its own crawling, keeps about 4 requests in flight with a gap between request starts, and slows to one request at a time when robots.txt sets a crawl delay. The CDN check sends one homepage request per AI crawler name, which is a light load even for a prospect's site.

PageSpeed key. Google's PageSpeed Insights API works without a key, but the keyless quota is shared and often used up; the audit then says Core Web Vitals weren't checked. A free key fixes it: in the Google Cloud console, enable the PageSpeed Insights API, create an API key, and set PAGESPEED_API_KEY. Enabling the Chrome UX Report API on the same project lets that key fetch real-visitor data too (or set CRUX_API_KEY).

Phase 2: Run the crawl

python3 ${CLAUDE_SKILL_DIR}/scripts/crawl_audit.py <url>                  # full
python3 ${CLAUDE_SKILL_DIR}/scripts/crawl_audit.py <url> --mode quick     # quick
python3 ${CLAUDE_SKILL_DIR}/scripts/crawl_audit.py <url> --compare auto   # re-audit against the last run for this site

The script (Python 3 standard library, no installs) reads robots.txt and every sitemap it lists, fetches the homepage, crawls same-site links breadth first with a sample of sitemap URLs mixed in, checks the internal link targets it didn't crawl, compares how the homepage answers a browser and each AI crawler, probes a made-up URL for soft 404s, looks for llms.txt and ai.txt, and runs PageSpeed. It writes findings.json and pages.jsonl to ./campaigns/site-audit-<domain>-<YYYY-MM-DD>/ (a second run that day gets -2, so earlier runs are never overwritten) and prints a summary. Useful flags: --max-pages N, --no-psi, --no-bot-check, --psi-templates N, --max-minutes N. Run --help for the rest.

No python3? Do a sample audit by hand: fetch robots.txt, the sitemap, the homepage and 5 to 10 money pages (WebFetch or curl), run the same checks on what you read, and say in the Sources and assumptions note that it was a sample of N pages.

Sites that build their text with JavaScript. When most pages send almost no text in their HTML, the crawl reports js_rendered_content and leaves those pages out of the content checks rather than calling them thin. Say so plainly in the report. Google renders JavaScript, later than HTML; AI crawlers may read only the HTML they're sent (the AI search facts in /ai-search list server-rendered text as a prerequisite for ChatGPT). To audit what visitors actually see, re-run with rendered HTML. We use Firecrawl for this (free tier, see TOOLS.md): with FIRECRAWL_API_KEY set, add --render firecrawl (up to 25 pages, each counted against your Firecrawl credits). With the Firecrawl MCP instead, scrape each page with the rawHtml format, save the results as JSON lines ({"url": ..., "html": ...}), and pass --rendered <file>. Without either, list the skipped checks and stop there.

Phase 3: Read the findings

findings.json holds findings (each with id, severity, count, key_pages_affected, examples, urls, observation, fix, effort, confidence), checked_and_fine, not_checked, a site block (robots, AI crawler policy per bot, the CDN comparison, sitemaps, AI files, host variants, soft 404, content-signal summary) and cwv. Load references/checks.md for what each finding means, when to raise or lower it, its edge cases, and the fix by platform.

Before anything goes in the top list:

  • Look at one example yourself. Fetch the example URL and find the line that proves it (the meta tag, the Disallow rule, the 404). The script's heuristics have edge cases, and an owner who catches one wrong item stops trusting the rest. Note what you verified in notes.md.
  • Keep "couldn't check" apart from "fine". not_checked items go in the Sources and assumptions note as unknowns. Only checked_and_fine items may be called fine.
  • Respect the scope. When crawl.truncated is true, counts mean "at least N among the pages crawled", and orphan pages are weak evidence.
  • The CDN check uses stand-in requests. It sends the crawler's name from this machine, not from the vendor's IP ranges. A refusal is strong evidence that the CDN or firewall blocks that name, and the fix gets confirmed in the CDN's bot settings or server logs. Say exactly that.
  • Field beats lab. PageSpeed returns real-visitor data (Chrome UX Report, 28 days) only for pages with enough traffic. When only lab data exists, say "a simulated phone load", never "your visitors wait".

Phase 4: Rank the fixes

Rank by how much each fix helps the business get found, then by effort. The default order, which you may change with a stated reason:

  1. Anything that keeps Google out of the site or a money page. The homepage down or failing HTTPS, robots.txt erroring or disallowing /, noindex on money pages (every page noindex is usually one setting, such as WordPress's "Discourage search engines" box), money pages canonicalized to another page or another domain.
  2. AI crawlers blocked when the business wants AI visibility. The search and assistant crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User, Claude-User, Perplexity-User), in robots.txt or at the CDN.
  3. Broken paths to money pages. Broken links in the menu or from the homepage, redirect loops, server errors, a sitemap that lists noindex, broken or blocked URLs.
  4. Content search engines can't see or tell apart. Text built by JavaScript, duplicate and near-copy pages (town pages with the city swapped), thin money pages.
  5. Real visitors waiting on money pages. Poor field Core Web Vitals. Lab-only slowness sits lower.
  6. Basics on money pages. Missing titles, no H1, no mobile viewport.
  7. Everything else (description length, alt text, share tags, HSTS, URL style, llms.txt, generic schema, content signals on old posts) goes to "also worth doing" or the one-line fine list.

Judgment for the close calls:

  • Weight by page value. One service page outranks fifty tag archives. A ten-minute fix that unblocks indexing goes ahead of a redesign.
  • Merge findings with one cause. A page that is noindex and also in the sitemap is one item. Twenty broken links from one menu entry are one fix.
  • Pick five to ten. A healthy site may have three; say so and stop. Padding the list hides the items that matter.
  • When rank is the real problem, technical fixes won't produce AI citations or traffic. Say so and route to /keyword-research or /local-seo-plan.

Hard rules: never call something broken without the observation behind it; never present "couldn't check" as fine; never invent traffic, revenue or ranking impact; never promise citations from AI-readiness fixes.

Phase 5: Write the report

Write for the owner, in plain words. No script paths, tool names used as process, check ids or severity labels in the report; those stay in notes.md. Length: the plan budget (about 1,200 to 2,500 words) is a ceiling, and most audits need 600 to 1,200. Fix first carries item 1 in full; the numbered list holds the rest in order, so nothing is said twice. Each item names what's wrong, why it matters for this business, the affected pages (count plus two or three examples), the fix, and rough effort.

A sample, for the sample business Hartline Plumbing, Dallas TX (illustrative, not a real site or real numbers):

## SITE AUDIT: hartlineplumbing.com (sample)
Nov 16, 2026 · 184 pages crawled, for getting found on Google and in AI answers

**Fix first:** the water heater pages tell Google not to index them. All 6 pages under
/water-heaters/ carry `noindex` (for example /water-heaters/tankless-installation/), and 4
of them are in the sitemap. They are the services you advertise most, so they can't rank
or be cited until this changes. Fix: untick "hide from search engines" on those pages in
the SEO plugin, then ask Google to recrawl them in Search Console. About 15 minutes.

**Then, in this order:**

1. **ChatGPT's and Perplexity's search crawlers are blocked.** robots.txt has
   `User-agent: OAI-SearchBot` / `Disallow: /` and the same for PerplexityBot, so the site
   can't be cited in their answers. Pages: the whole site. Fix: delete those two groups;
   keep GPTBot blocked if you don't want AI training. 5 minutes.
2. **The "Book online" link in the menu is broken.** It points at /book-now/, which
   returns 404, on all 184 pages. Fix: point the menu at /schedule/. 5 minutes.
3. **Twelve town pages are near copies.** /plumber-plano/ and /plumber-allen/ differ only
   in the town name (12 pages, e.g. /plumber-frisco/). Fix: add jobs done there, local
   details and photos, or merge them into one service-area page. An hour per page.
4. **Phones wait on the homepage.** Real visitors on phones see the main image after
   about 4.6 seconds (Google's good mark is 2.5). Fix: compress and resize the hero photo
   and stop loading the chat widget before the page. Half a day for a developer.

**Also worth doing:** add a self-referencing canonical to the 40 pages without one; add
alt text to the gallery photos; add business markup (name, phone, address) to the
homepage.

**Checked and fine:** HTTPS and redirects, robots.txt for Google, sitemap format, no
server errors, titles present and unique, mobile viewport on every page.

**Sources and assumptions:**
- Crawled 184 of ~210 URLs on Nov 16, 2026 (the crawl stops at 200 pages), plus 150
  internal link checks; PageSpeed real-visitor data for the homepage, lab data for two
  service pages.
- The CDN check sends the crawlers' names from this machine, not from their own networks;
  it found no blocks.
- Not checked: which pages Google has indexed (needs Search Console access).

For a re-audit, open with What changed since <date> (resolved, new, still open, from changes in the findings file), then the current fix list. Each run crawls within its own page limit, so a page seen in only one run may just have been outside the other run's sample. Say so instead of calling it new or gone.

Output

Write site-audit.md (the report) into the run's folder, ./campaigns/site-audit-<domain>-<YYYY-MM-DD>/, beside the script's findings.json and pages.jsonl, and notes.md for the working record: the examples you verified by hand, findings merged or left out and why, raw Core Web Vitals, and anything the user should forward to a developer. Append one line to ./brand/assets.md when a brand folder exists (never for a prospect). Then the chat summary: header, the fix list, Files saved, What's next.

When the user edits your work

Apply the edit, then make it stick: save a gold case to ./brand/gold/<this skill>/ and one to three yes/no checks to ./brand/checks.md, and confirm in one line. Before handing over any work, answer the checks the context block loaded, silently, and fix every no. How: ${CLAUDE_SKILL_DIR}/../start-here/references/taste-loop.md.

What's next

  • /ai-search: now that crawlers can reach the site, check whether ChatGPT, Perplexity and Google's AI actually name and cite the business (~20 min)
  • /keyword-research: when the pages are sound but nothing ranks, find what the site should be ranking for (~15 min)
  • /local-seo-plan: for a local business, decide which service and town pages to build, merge or drop (~30 min)
  • /first-screen-check: see the money pages the way a customer does on a phone (~10 min)

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/boringmarketer/boring-marketing-skills/site-audit">View site-audit on skillZs</a>