Seo collector

Skill jgoullet/seo-geo-audit/skills/seo-collector

SEO-GEO Skill System for Claude Code & Claude.ai — 5 connected skills: Collector → Auditor (/90) → Implementer → Monitor. Technical SEO, Core Web Vitals, E-E-A-T, Entity SEO, AI visibility (ChatGPT, Perplexity, Claude), competitive analysis. English by default.

Install

npx -y skills add jgoullet/seo-geo-audit --skill seo-collector

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

4 stars4 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Automated SEO data collector. Crawls a website and produces a structured site-data.json file containing: title/meta/H1 tags, robots.txt, sitemap, HTTP headers, Core Web Vitals (via PageSpeed Insights), schema markup detection, AI bot access analysis in robots.txt, and sample page snapshots. Optionally integrates Ahrefs MCP for real backlink data, domain rating, organic keywords, and competitor identification. This skill is Step 1 of the SEO-GEO pipeline: it collects raw data that seo-geo-audit consumes to produce the audit. Use this skill when the user says: "collect SEO data for [url]", "crawl this site", "technical analysis of [url]", "prepare data for the audit", "run the collector on [url]", "scrape SEO metrics", "check the Core Web Vitals of", "check the robots.txt of", "analyze the meta tags of", "can AI bots access [url]", "get backlink data for [url]". Also triggers for: "collecte les données SEO de", "analyse technique de", "crawl ce site".

SKILL.md

18.3 KB, as published. Nobody here has run it

Skill: SEO Collector — Automated technical data collection

All responses and logs are produced in English by default. Switch to the user's language if they explicitly request it.

Overview

This skill automatically crawls a website and produces a structured site-data.json file, ready to be consumed by the seo-geo-audit skill. It replaces manual information gathering (Step 0 of the audit).

Position in the pipeline

[seo-shared-context] → client-profile.md
                            ↓
              ┌─────────────────────────┐
              │  SEO COLLECTOR (this)   │
              └─────────────────────────┘
                            ↓
                     site-data.json
                            ↓
              ┌─────────────────────────┐
              │     SEO-GEO AUDIT       │
              └─────────────────────────┘

Step 0: Prerequisites

Required inputs

Site URL (required)
client-profile.md (optional — if available, load it for mode, competitors, CMS info)

Optional MCP integrations

Ahrefs MCP (recommended — replaces estimates with real data)

Ahrefs MCP is available on Lite, Standard, Advanced, and Enterprise plans (from $29/month Starter). It provides real backlink data, domain rating, organic keywords, and competitor identification — all data that the collector would otherwise estimate.

If Ahrefs MCP is connected, the collector will use it to pull:

Ahrefs endpoint	What it provides	Replaces
`domain-rating`	Real DR score (0-100) + Ahrefs Rank	Estimated authority
`backlinks-stats`	Total backlinks, referring domains, dofollow/nofollow	"Estimated from web search"
`broken-backlinks`	Broken inbound links list	Not available without Ahrefs
`organic-keywords`	Keywords the site ranks for + positions + volumes	Manual keyword search
`organic-competitors`	Top organic competitors identified by Ahrefs	"Identify via web_search"
`top-pages`	Pages driving the most organic traffic	Sample crawl guesswork
`batch-url-analysis`	SEO metrics for multiple URLs at once	One-by-one page crawl
`brand-radar`	Brand mentions across AI agents (ChatGPT, Perplexity...)	Manual GEO visibility test

How to connect: The user provides their Ahrefs account via MCP. No API key pasting needed — MCP handles authentication.

If Ahrefs MCP is NOT connected: All metrics are estimated from observable signals via web_search and web_fetch. Clearly flag this in the output: "source": "estimated" vs "source": "ahrefs_mcp".

GA4 MCP (optional)

If connected (github.com/googleanalytics/google-analytics-mcp), pull real organic traffic, bounce rates, conversions.

GSC MCP (optional)

If connected (github.com/ahonn/mcp-server-gsc), pull real rankings, impressions, AI Overviews data, URL inspection.

Context window budget — `text_content_token_limit`

⚠️ CRITICAL — Apply token limits to ALL web_fetch calls.

A full audit generates 25-40 web calls. Without limits, web_fetch returns entire pages (10-15K tokens each), consuming 100-175K tokens and risking context overflow before the report is generated.

Always pass text_content_token_limit to web_fetch. The SEO-relevant content (title, meta, H1, schema, headers) is concentrated in the <head> and early <body> — the rest is navigation, footer, ads, and noise.

Fetch target	`text_content_token_limit`	Why this limit
Homepage (HTML)	4000	`<head>` + H1/H2 + first content block — enough for title, meta, schema, OG tags, CMS markers
robots.txt	1000	Always short — full AI bot rules fit easily
sitemap.xml	3000	URL count + sample URLs + lastmod — don't need every URL
Secondary/inner pages	3000	Same extraction as homepage, lighter
Competitor pages	3000	Only need title, meta, H1, schema presence
PageSpeed / external data	(use `web_search`, no fetch needed)	Search snippets are pre-truncated

Example:

web_fetch: [URL]  → text_content_token_limit: 4000
web_fetch: [URL]/robots.txt  → text_content_token_limit: 1000

Estimated savings: ~100K tokens per audit (from ~175K down to ~75K).

Base verification

Before starting, test that the site is accessible:

web_fetch: [URL] (text_content_token_limit: 4000)

If error (timeout, 5xx, maintenance) → note and stop. If redirect → follow and note the final URL.

Step 1: Homepage metadata collection

Use web_fetch on the main URL (with text_content_token_limit: 4000) and extract:

1.1 Essential HTML tags

Extract and structure: <title>, meta description, H1 (count), H2 list, canonical URL, lang attribute, hreflang tags, viewport meta, Open Graph tags, Twitter Card.

1.2 CMS detection

Look for markers in the HTML source: WordPress (wp-content/), Shopify (cdn.shopify.com), Webflow (webflow.com), Wix (wix.com), Next.js (__NEXT_DATA__), and others.

Step 2: Robots.txt & Sitemap

2.1 Robots.txt

Fetch and parse robots.txt (with text_content_token_limit: 1000). For each AI bot (Googlebot, GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, anthropic-ai, Bingbot), determine if explicitly blocked, allowed, or inheriting from User-agent: *.

2.2 Sitemap XML

Fetch sitemap.xml (with text_content_token_limit: 3000). Extract: existence, URL, type (index or urlset), total URL count, sample URLs, last modified date.

Step 3: Core Web Vitals

Search for PageSpeed Insights data via web_search. Structure results for LCP, CLS, INP, FCP, TTFB — both mobile and desktop. Note data source and freshness.

If data is not found, estimate based on HTML analysis (page weight, script count, image optimization).

Step 4: HTTPS & Security headers

Analyze HTTP headers via web_fetch (with text_content_token_limit: 2000): HTTPS status, HSTS, Content-Security-Policy, X-Frame-Options, mixed content detection.

Step 5: Schema markup detection

⚠️ CRITICAL LIMITATION: web_fetch cannot detect JavaScript-injected JSON-LD. Flag uncertainty if CMS is detected but no schema found in static HTML.

Search for <script type="application/ld+json">, itemscope/itemtype (Microdata), and typeof/property (RDFa) in the HTML.

Step 5.5: Analytics & tracking audit

Analyze the HTML source from the homepage (and sample pages in Step 6) to detect which analytics and tracking tools are installed, and whether basic conversion tracking appears to be configured.

5.5.1 Analytics tool detection

Search the HTML for these markers:

Tool	Detection pattern	What it means
Google Analytics 4 (GA4)	`gtag.js`, `G-XXXXXXX`, `googletagmanager.com/gtag`	Modern analytics — good
Google Tag Manager (GTM)	`googletagmanager.com/gtm.js`, `GTM-XXXXXXX`	Tag container — enables event tracking without code changes
Universal Analytics (UA)	`analytics.js`, `UA-XXXXXXX`	Legacy — deprecated July 2024, should be migrated to GA4
Facebook/Meta Pixel	`fbq(`, `connect.facebook.net/en_US/fbevents.js`	Ad tracking — indicates paid social activity
Hotjar / Clarity	`hotjar.com`, `clarity.ms`	Heatmaps / session recording — indicates UX focus
Segment	`analytics.segment.com`, `cdn.segment.com`	CDP — indicates mature tracking stack
Mixpanel	`mixpanel.com/track`, `cdn.mxpnl.com`	Product analytics — indicates event-level tracking
Matomo / Piwik	`matomo.js`, `piwik.js`	Privacy-first analytics
Pinterest Tag	`pintrk(`, `s.pinimg.com`	Pinterest ad tracking
TikTok Pixel	`ttq.load`, `analytics.tiktok.com`	TikTok ad tracking
LinkedIn Insight	`snap.licdn.com/li.lms-analytics`	LinkedIn B2B tracking

5.5.2 Conversion tracking signals

Look for indicators that conversion events are configured:

Signal	What to look for	Status
Enhanced ecommerce	`purchase`, `add_to_cart`, `begin_checkout` in gtag calls or dataLayer pushes	✅/❌
Form tracking	Event listeners on `<form>` submit, or GTM form trigger patterns	✅/❌
Scroll tracking	`scroll` event in gtag or GTM	✅/❌
Click tracking	`click` events on CTAs, outbound links	✅/❌
dataLayer present	`window.dataLayer` initialized	✅/❌
Consent management	Cookie banner / CMP detected (Cookiebot, OneTrust, Axeptio, Didomi)	✅/❌

5.5.3 UTM parameter handling

Check if the site preserves UTM parameters:

Are UTMs visible in internal links (leak) or stripped properly?
Does the site use consistent UTM naming? (search for utm_source, utm_medium, utm_campaign patterns)

5.5.4 Output structure

{
  "analytics_tracking": {
    "tools_detected": {
      "ga4": { "present": true, "measurement_id": "G-XXXXXXX" },
      "gtm": { "present": true, "container_id": "GTM-XXXXXXX" },
      "ua_legacy": { "present": false },
      "meta_pixel": { "present": true },
      "hotjar_clarity": { "present": false, "tool": null },
      "segment": { "present": false },
      "mixpanel": { "present": false },
      "other": []
    },
    "conversion_tracking": {
      "ecommerce_events": false,
      "form_tracking": false,
      "scroll_tracking": false,
      "click_tracking": false,
      "datalayer_present": true,
      "consent_management": { "present": true, "tool": "Axeptio" }
    },
    "utm_handling": {
      "utm_params_detected_in_links": false,
      "consistent_naming": "unknown"
    },
    "maturity_level": "basic|intermediate|advanced",
    "issues": [
      "No ecommerce conversion tracking detected",
      "No heatmap/session recording tool installed"
    ],
    "recommendations": [
      "Set up GA4 ecommerce events (add_to_cart, purchase)",
      "Install Hotjar or Clarity for UX insights"
    ]
  }
}

Maturity levels:

basic: GA4 or GTM present but no conversion events detected
intermediate: GA4 + GTM + some conversion events + consent management
advanced: Full event tracking + CDP (Segment) or product analytics (Mixpanel) + heatmaps + structured dataLayer

This data feeds directly into the auditor's Step 6: UX & Conversion section, replacing guesswork with detected tracking state.

Step 6: Sample pages crawl

Crawl 3-5 additional pages (from sitemap or navigation) (each with text_content_token_limit: 3000). For each, collect the same data as Step 1.

Step 7: Ahrefs MCP data (if connected)

This entire step is skipped if Ahrefs MCP is not connected. All data points are then marked as "source": "estimated" in the output.

7.1 Domain authority & backlinks

Query Ahrefs MCP:

Domain Rating: domain-rating endpoint → real DR score
Backlinks stats: backlinks-stats → total backlinks, referring domains, dofollow ratio
Broken backlinks: broken-backlinks → list of broken inbound links (for the Implementer to generate redirects)

{
  "ahrefs": {
    "source": "ahrefs_mcp",
    "domain_rating": 45,
    "ahrefs_rank": 123456,
    "backlinks_total": 12500,
    "referring_domains": 890,
    "dofollow_ratio": 0.72,
    "broken_backlinks": [
      { "url_from": "https://partner.com/old-link", "url_to": "https://example.com/404-page" }
    ]
  }
}

7.2 Organic keywords & positions

Query Ahrefs MCP:

Organic keywords: organic-keywords → top 50 keywords the site ranks for with positions and volumes
Top pages: top-pages → pages driving the most organic traffic

{
  "ahrefs_keywords": {
    "total_organic_keywords": 2340,
    "top_keywords": [
      { "keyword": "linen curtains", "position": 3, "volume": 8100, "url": "/collections/lin" },
      { "keyword": "velvet cushions", "position": 7, "volume": 3200, "url": "/collections/coussins-velours" }
    ],
    "top_pages": [
      { "url": "/collections/rideau", "organic_traffic": 4500, "keywords_count": 120 }
    ]
  }
}

7.3 Competitors identification

Query Ahrefs MCP:

Organic competitors: organic-competitors → sites competing for the same keywords

{
  "ahrefs_competitors": [
    { "domain": "laredoute.fr", "common_keywords": 450, "competition_level": "high" },
    { "domain": "maison-monde.com", "common_keywords": 280, "competition_level": "medium" }
  ]
}

This replaces the manual "identify competitors via web_search" step. The auditor can use these directly.

7.4 Brand Radar — AI visibility (if available on user's plan)

Query Ahrefs MCP:

Brand Radar: brand-radar → brand mentions across AI agents

{
  "ahrefs_brand_radar": {
    "source": "ahrefs_mcp",
    "ai_mentions": {
      "chatgpt": { "mentioned": true, "citation_count": 5 },
      "perplexity": { "mentioned": true, "citation_count": 3 },
      "google_ai_overview": { "mentioned": false },
      "claude": { "mentioned": true, "citation_count": 1 }
    },
    "real_prompts_with_citations": [
      { "platform": "chatgpt", "prompt": "best linen curtains France", "cited": true }
    ]
  }
}

This is a direct replacement for the manual GEO visibility test (Step 5.6 in the auditor). Real data instead of manual searches.

Step 8: Indexation & external presence

8.1 Google indexation

web_search: "site:[domain]" — estimate indexed page count.

8.2 Bing indexation

web_search: "[domain] bing indexed" — verify presence.

8.3 Brave Search indexation

web_search: "[domain] brave search" — verify presence (critical for Claude citations).

8.4 Local directories (Local mode)

Search for business on key directories (Yelp, TripAdvisor, Healthgrades, etc.).

Step 9: Assemble site-data.json

Assemble all collected data into a single structured JSON file:

{
  "metadata": {
    "collector_version": "2.0",
    "collection_date": "[ISO 8601]",
    "url_audited": "[URL]",
    "url_final": "[URL after redirects]",
    "domain": "[extracted domain]",
    "data_completeness": "high|medium|low",
    "mcp_integrations": {
      "ahrefs": true|false,
      "ga4": true|false,
      "gsc": true|false
    }
  },
  "homepage": { "..." },
  "cms_detected": "shopify|wordpress|...",
  "robots_txt": { "..." },
  "sitemap": { "..." },
  "core_web_vitals": { "..." },
  "security": { "..." },
  "schema_markup": { "..." },
  "analytics_tracking": { "..." },
  "pages_sample": [ "..." ],
  "indexation": { "..." },
  "ahrefs": { "..." },
  "ahrefs_keywords": { "..." },
  "ahrefs_competitors": [ "..." ],
  "ahrefs_brand_radar": { "..." },
  "issues_summary": {
    "critical": ["[list]"],
    "warnings": ["[list]"],
    "info": ["[list]"]
  }
}

Save to seo-project-[domain]/site-data.json.

If no client-profile.md exists, create a pre-filled one with detected information.

Step 10: Summary for the user

After collection, present a clear summary:

═══════════════════════════════════════════
SEO DATA COLLECTION COMPLETE — [domain]
Date: [date]
═══════════════════════════════════════════

📊 Data collected:
   ✅ Homepage: title, meta, H1, OG tags
   ✅ Robots.txt: [X] AI bots allowed / [Y] blocked
   ✅ Sitemap: [N] URLs found
   ✅ Core Web Vitals: [source]
   ✅ Security: HTTPS [yes/no]
   ✅ Schema: [types found or "check needed"]
   ✅ Analytics: [tools detected] | Maturity: [basic/intermediate/advanced]
   ✅ Sample: [N] pages analyzed
   ✅ Indexation: ~[N] pages on Google

[If Ahrefs MCP connected:]
   ✅ Ahrefs DR: [score]/100 | Referring domains: [N]
   ✅ Organic keywords: [N] tracked | Top position: [N]
   ✅ Competitors: [N] identified
   ✅ Brand Radar: cited by [N] AI platforms

[If Ahrefs MCP NOT connected:]
   ℹ️ Backlinks & keywords: estimated (connect Ahrefs MCP for real data)

⚠️ Issues detected: [N critical] / [N warning]
   [Top 3 most important issues]

📁 File generated: site-data.json
   → Ready for seo-geo-audit

Would you like to run the full audit now?

Error handling

Situation	Action
Site inaccessible (timeout)	Retry after 5s. If still down → note and stop.
Robots.txt missing	Note `exists: false`, continue.
Sitemap missing	Check robots.txt for sitemap URL, otherwise note `exists: false`.
PageSpeed not found	Estimate via HTML analysis, note `source: "estimation"`.
Page 403/401	Note, continue with other pages.
Rate limiting detected	Space out requests, note in report.
Ahrefs MCP timeout	Note the failure, continue without Ahrefs data. Flag in output.
Ahrefs MCP not connected	Skip Step 7 entirely, all metrics marked `"source": "estimated"`.

Pipeline chaining

The site-data.json file is designed to be read directly by seo-geo-audit. When the user asks to run the audit after collection:

Verify site-data.json exists
Pass it in context to the audit skill
The audit skill skips Step 0 (manual collection) and goes directly to analysis
If Ahrefs data is present, the auditor uses real metrics instead of estimates

Pattern 2 applied: Output-as-input chaining. The collector writes structured JSON, the auditor reads it. Loose coupling, shared files as the API layer.

One thing to look at

What its author says it does

SKILL.md

Skill: SEO Collector — Automated technical data collection

Overview

Position in the pipeline

Step 0: Prerequisites

Required inputs

Optional MCP integrations

Ahrefs MCP (recommended — replaces estimates with real data)

GA4 MCP (optional)

GSC MCP (optional)

Context window budget — text_content_token_limit

Base verification

Step 1: Homepage metadata collection

1.1 Essential HTML tags

1.2 CMS detection

Step 2: Robots.txt & Sitemap

2.1 Robots.txt

2.2 Sitemap XML

Step 3: Core Web Vitals

Step 4: HTTPS & Security headers

Step 5: Schema markup detection

Step 5.5: Analytics & tracking audit

5.5.1 Analytics tool detection

5.5.2 Conversion tracking signals

5.5.3 UTM parameter handling

5.5.4 Output structure

Step 6: Sample pages crawl

Step 7: Ahrefs MCP data (if connected)

7.1 Domain authority & backlinks

7.2 Organic keywords & positions

7.3 Competitors identification

7.4 Brand Radar — AI visibility (if available on user's plan)

Step 8: Indexation & external presence

8.1 Google indexation

8.2 Bing indexation

8.3 Brave Search indexation

8.4 Local directories (Local mode)

Step 9: Assemble site-data.json

Step 10: Summary for the user

Error handling

Pipeline chaining

Keep looking

Context window budget — `text_content_token_limit`