feat(seo-skills): multi-backend Data Source Selection (#7)
Replaces single-vendor (Ahrefs-only) tool defaults with a per-task backend menu across all 14 SEO skills. Each skill now lists every capable MCP in allowed-tools and documents how to pick between Semrush, Ahrefs, OurSEO Agent (CLI + MCP), DataForSEO, and GSC in its SKILL.md Data Source Selection section. Tool stubs (~40 new files) populated per skill with capability deltas, call patterns, and explicit "not for this skill when" callouts so the menu is self-correcting. Skills affected: 19-keyword-strategy, 20-serp-analysis, 21-position-tracking, 22-link-building, 23-content-strategy, 24-ecommerce, 25-kpi-framework, 26-international, 27-ai-visibility, 28-knowledge-graph, 31-competitor-intel, 32-crawl-budget, 33-migration-planner, 34-reporting-dashboard. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
53
custom-skills/32-seo-crawl-budget/desktop/tools/ourseo.md
Normal file
53
custom-skills/32-seo-crawl-budget/desktop/tools/ourseo.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# OurSEO Agent (CLI + MCP)
|
||||
|
||||
OurSEO contributes the **crawled URL inventory** (vs. server log observations), **index status check**, and **distributed crawl** for large sites. Pair with `scripts/log_parser.py` + `scripts/crawl_budget_analyzer.py` for the full crawl-budget picture.
|
||||
|
||||
## CLI commands (Claude Code)
|
||||
|
||||
```bash
|
||||
# URL inventory crawl
|
||||
our collect crawl https://<site> --max-pages 5000
|
||||
our collect crawl https://<site> --stealth --backend scrapy
|
||||
|
||||
# Distributed crawl for large sites
|
||||
our collect distributed https://<site> --workers 8 --max-pages 50000
|
||||
|
||||
# Index status (Google index coverage)
|
||||
our research google index --domain <site>
|
||||
|
||||
# Technical audit (redirects, canonicals, etc.)
|
||||
our audit tech https://<site>
|
||||
|
||||
# MySQL batch analysis on crawl results
|
||||
our analyze mysql-batch --session <id>
|
||||
our analyze mysql-batch --session <id> --export issues --format excel
|
||||
```
|
||||
|
||||
## MCP tools (Claude Desktop)
|
||||
|
||||
```
|
||||
mcp__ourseo__crawl_website(url="<site>", max_pages=5000)
|
||||
mcp__ourseo__check_index(domain="<site>")
|
||||
mcp__ourseo__audit_page(url="<url>", audit_type="tech")
|
||||
```
|
||||
|
||||
## Strengths for this skill
|
||||
|
||||
- **Configurable crawler** — backend choice (aiohttp, Scrapy), stealth mode, resume support.
|
||||
- **Distributed crawl** scales to 50K+ URL audits.
|
||||
- **`check_index`** uses Google PSE to spot-check indexed-or-not per URL.
|
||||
- Crawl output drops into MySQL workspace — easy to diff against log-derived URL sets.
|
||||
|
||||
## How OurSEO pairs with log analysis
|
||||
|
||||
1. **Logs** (`scripts/log_parser.py`) → what bots actually crawled at the origin
|
||||
2. **OurSEO crawl** → what URLs exist on the site today
|
||||
3. **Sitemap** → what URLs the site claims to publish
|
||||
4. **GSC** → what Google considers indexed
|
||||
5. **Diff all four** — orphan, ghost, and waste candidates emerge from set differences
|
||||
|
||||
## Not for this skill when
|
||||
|
||||
- **Actual bot User-Agent + frequency** — only logs show this.
|
||||
- **Per-bot status distribution** — only logs.
|
||||
- **First-party Googlebot crawl stats** — GSC.
|
||||
Reference in New Issue
Block a user