e527fb4b0f
Replaces single-vendor (Ahrefs-only) tool defaults with a per-task backend menu across all 14 SEO skills. Each skill now lists every capable MCP in allowed-tools and documents how to pick between Semrush, Ahrefs, OurSEO Agent (CLI + MCP), DataForSEO, and GSC in its SKILL.md Data Source Selection section. Tool stubs (~40 new files) populated per skill with capability deltas, call patterns, and explicit "not for this skill when" callouts so the menu is self-correcting. Skills affected: 19-keyword-strategy, 20-serp-analysis, 21-position-tracking, 22-link-building, 23-content-strategy, 24-ecommerce, 25-kpi-framework, 26-international, 27-ai-visibility, 28-knowledge-graph, 31-competitor-intel, 32-crawl-budget, 33-migration-planner, 34-reporting-dashboard. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
54 lines
1.9 KiB
Markdown
54 lines
1.9 KiB
Markdown
# OurSEO Agent (CLI + MCP)
|
|
|
|
OurSEO contributes the **crawled URL inventory** (vs. server log observations), **index status check**, and **distributed crawl** for large sites. Pair with `scripts/log_parser.py` + `scripts/crawl_budget_analyzer.py` for the full crawl-budget picture.
|
|
|
|
## CLI commands (Claude Code)
|
|
|
|
```bash
|
|
# URL inventory crawl
|
|
our collect crawl https://<site> --max-pages 5000
|
|
our collect crawl https://<site> --stealth --backend scrapy
|
|
|
|
# Distributed crawl for large sites
|
|
our collect distributed https://<site> --workers 8 --max-pages 50000
|
|
|
|
# Index status (Google index coverage)
|
|
our research google index --domain <site>
|
|
|
|
# Technical audit (redirects, canonicals, etc.)
|
|
our audit tech https://<site>
|
|
|
|
# MySQL batch analysis on crawl results
|
|
our analyze mysql-batch --session <id>
|
|
our analyze mysql-batch --session <id> --export issues --format excel
|
|
```
|
|
|
|
## MCP tools (Claude Desktop)
|
|
|
|
```
|
|
mcp__ourseo__crawl_website(url="<site>", max_pages=5000)
|
|
mcp__ourseo__check_index(domain="<site>")
|
|
mcp__ourseo__audit_page(url="<url>", audit_type="tech")
|
|
```
|
|
|
|
## Strengths for this skill
|
|
|
|
- **Configurable crawler** — backend choice (aiohttp, Scrapy), stealth mode, resume support.
|
|
- **Distributed crawl** scales to 50K+ URL audits.
|
|
- **`check_index`** uses Google PSE to spot-check indexed-or-not per URL.
|
|
- Crawl output drops into MySQL workspace — easy to diff against log-derived URL sets.
|
|
|
|
## How OurSEO pairs with log analysis
|
|
|
|
1. **Logs** (`scripts/log_parser.py`) → what bots actually crawled at the origin
|
|
2. **OurSEO crawl** → what URLs exist on the site today
|
|
3. **Sitemap** → what URLs the site claims to publish
|
|
4. **GSC** → what Google considers indexed
|
|
5. **Diff all four** — orphan, ghost, and waste candidates emerge from set differences
|
|
|
|
## Not for this skill when
|
|
|
|
- **Actual bot User-Agent + frequency** — only logs show this.
|
|
- **Per-bot status distribution** — only logs.
|
|
- **First-party Googlebot crawl stats** — GSC.
|