Add SEO skills 19-28, 31-32 with full Python implementations
12 new skills: Keyword Strategy, SERP Analysis, Position Tracking, Link Building, Content Strategy, E-Commerce SEO, KPI Framework, International SEO, AI Visibility, Knowledge Graph, Competitor Intel, and Crawl Budget. ~20K lines of Python across 25 domain scripts. Updated skill 11 pipeline table and repo CLAUDE.md. Enhanced skill 18 local SEO workflow from jamie.clinic audit. Note: Skill 26 hreflang_validator.py pending (content filter block). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
39
custom-skills/32-seo-crawl-budget/desktop/SKILL.md
Normal file
39
custom-skills/32-seo-crawl-budget/desktop/SKILL.md
Normal file
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: seo-crawl-budget
|
||||
description: |
|
||||
Crawl budget optimization and log analysis. Triggers: crawl budget, log analysis, bot crawling, Googlebot, crawl waste, orphan pages, crawl efficiency.
|
||||
---
|
||||
|
||||
# Crawl Budget Optimizer
|
||||
|
||||
Analyze server access logs to identify crawl budget waste and generate optimization recommendations for search engine bots.
|
||||
|
||||
## Capabilities
|
||||
|
||||
1. **Log Analysis**: Parse Nginx/Apache/CloudFront access logs to extract bot crawl data
|
||||
2. **Bot Profiling**: Per-bot behavior analysis (Googlebot, Yeti, Bingbot, Daumoa)
|
||||
3. **Waste Detection**: Parameter URLs, redirect chains, soft 404s, duplicate URL variants
|
||||
4. **Orphan Pages**: Pages in sitemap but uncrawled, and crawled pages not in sitemap
|
||||
5. **Recommendations**: Prioritized action items for crawl budget optimization
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Parse server access log with `log_parser.py`
|
||||
2. Run crawl budget analysis with `crawl_budget_analyzer.py`
|
||||
3. Compare with sitemap URLs for orphan page detection
|
||||
4. Optionally compare with Ahrefs page history data
|
||||
5. Generate Korean-language report with recommendations
|
||||
6. Save to Notion SEO Audit Log database
|
||||
|
||||
## Tools Used
|
||||
|
||||
- **Ahrefs**: `site-explorer-pages-history` for indexed page comparison
|
||||
- **Notion**: Save audit report to database `2c8581e5-8a1e-8035-880b-e38cefc2f3ef`
|
||||
- **WebSearch**: Current best practices and bot documentation
|
||||
|
||||
## Output
|
||||
|
||||
All reports are saved to the OurDigital SEO Audit Log with:
|
||||
- Category: Crawl Budget
|
||||
- Audit ID format: CRAWL-YYYYMMDD-NNN
|
||||
- Content in Korean with technical English terms preserved
|
||||
8
custom-skills/32-seo-crawl-budget/desktop/skill.yaml
Normal file
8
custom-skills/32-seo-crawl-budget/desktop/skill.yaml
Normal file
@@ -0,0 +1,8 @@
|
||||
name: seo-crawl-budget
|
||||
description: |
|
||||
Crawl budget optimization and log analysis. Triggers: crawl budget, log analysis, bot crawling, Googlebot, crawl waste, orphan pages, crawl efficiency.
|
||||
allowed-tools:
|
||||
- mcp__ahrefs__*
|
||||
- mcp__notion__*
|
||||
- WebSearch
|
||||
- WebFetch
|
||||
17
custom-skills/32-seo-crawl-budget/desktop/tools/ahrefs.md
Normal file
17
custom-skills/32-seo-crawl-budget/desktop/tools/ahrefs.md
Normal file
@@ -0,0 +1,17 @@
|
||||
# Ahrefs MCP Tools
|
||||
|
||||
## site-explorer-pages-history
|
||||
|
||||
Get historical page data for a domain to compare indexed pages with crawled pages.
|
||||
|
||||
```
|
||||
mcp__ahrefs__site-explorer-pages-history
|
||||
```
|
||||
|
||||
**Parameters**:
|
||||
- `target` (string, required): Domain or URL to analyze
|
||||
- `date_from` (string): Start date (YYYY-MM-DD)
|
||||
- `date_to` (string): End date (YYYY-MM-DD)
|
||||
- `mode` (string): "domain", "prefix", "exact", "subdomains"
|
||||
|
||||
**Use case**: Compare Ahrefs indexed page counts with server log crawl data to identify indexing gaps and crawl budget inefficiencies.
|
||||
21
custom-skills/32-seo-crawl-budget/desktop/tools/notion.md
Normal file
21
custom-skills/32-seo-crawl-budget/desktop/tools/notion.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Notion MCP Tools
|
||||
|
||||
## notion-create-pages
|
||||
|
||||
Create a new page in the OurDigital SEO Audit Log database.
|
||||
|
||||
```
|
||||
mcp__notion__notion-create-pages
|
||||
```
|
||||
|
||||
**Database ID**: `2c8581e5-8a1e-8035-880b-e38cefc2f3ef`
|
||||
|
||||
**Required Properties**:
|
||||
- `Issue` (Title): Report title in Korean with date
|
||||
- `Site` (URL): Audited website URL
|
||||
- `Category` (Select): "Crawl Budget"
|
||||
- `Priority` (Select): Based on waste percentage (Critical >20%, High >10%, Medium >5%, Low <5%)
|
||||
- `Found Date` (Date): Audit date (YYYY-MM-DD)
|
||||
- `Audit ID` (Rich Text): Format CRAWL-YYYYMMDD-NNN
|
||||
|
||||
**Content**: Full crawl budget report in Korean with technical English terms preserved.
|
||||
18
custom-skills/32-seo-crawl-budget/desktop/tools/websearch.md
Normal file
18
custom-skills/32-seo-crawl-budget/desktop/tools/websearch.md
Normal file
@@ -0,0 +1,18 @@
|
||||
# WebSearch Tool
|
||||
|
||||
## Purpose
|
||||
|
||||
Search the web for current crawl budget best practices, search engine bot documentation, and robots.txt guidelines.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
WebSearch(query="Googlebot crawl budget optimization 2025")
|
||||
```
|
||||
|
||||
**Common queries**:
|
||||
- Search engine bot crawl rate documentation
|
||||
- robots.txt best practices for crawl budget
|
||||
- URL parameter handling for search engines
|
||||
- Crawl budget optimization techniques
|
||||
- Search engine bot user-agent strings
|
||||
Reference in New Issue
Block a user