feat(reference-curator): add --save-raw, --no-distill flags + OurSEO crawler integration

- Add --save-raw flag: saves intact crawled markdown in raw/ subdirectory
  alongside distilled content, with YAML frontmatter for traceability
- Add --no-distill flag: skip stages 4-5 (distiller + QA reviewer),
  pure archival mode for raw documentation capture
- Integrate OurSEO BaseAsyncClient + PageAnalyzer as seo-aiohttp backend:
  token-bucket rate limiting, semaphore concurrency, tenacity retries,
  full page metadata extraction (headings, links, schema, OG)
- New seo_crawler_adapter.py: crawl URLs, sitemaps, link discovery, manifests
  with progress tracking and resume support
- Update crawler selection: docs sites now default to seo-aiohttp
- Update crawl_config.yaml with seo-aiohttp routing rules
- Update pipeline orchestrator with flag resolution and skip paths
- Update README.md and USER-GUIDE.md with raw mode documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

custom-skills/90-reference-curator/USER-GUIDE.md

@@ -15,6 +15,12 @@ A modular skill suite for curating reference documentation with Claude Code.
 
 # With options
 /reference-curator-pipeline "MCP servers" --max-sources 10 --export-format fine_tuning
+
+# Save raw crawled content alongside distilled output
+/reference-curator-pipeline "Claude Code best practices" --save-raw
+
+# Pure archival — skip distillation, raw markdown only
+/reference-curator-pipeline https://docs.anthropic.com --no-distill
 ```
 
 ### Run as Background Agent
@@ -156,6 +162,89 @@ Use these when you need granular control over each stage.
 
 ---
 
+## Raw Mode
+
+### What is Raw Mode?
+
+By default, the pipeline distills crawled content down to 25-35% of the original, extracting key concepts, code snippets, and structured summaries. **Raw mode** preserves the full, unmodified crawled markdown.
+
+### `--save-raw` Flag
+
+Saves intact crawled markdown **alongside** the distilled content in a `raw/` subdirectory.
+
+```bash
+/reference-curator-pipeline "Anthropic Claude API" --save-raw
+```
+
+**Output:**
+```
+exports/claude-api/
+├── INDEX.md
+├── 00-prompt-caching.md           # Distilled (25-35% of original)
+├── 01-streaming.md
+├── raw/
+│   ├── 00-prompt-caching.raw.md   # Full crawled content
+│   ├── 01-streaming.raw.md
+│   └── claude-api-raw-complete.md # Combined raw bundle
+└── manifest.json
+```
+
+**Use cases:**
+- Need both concise reference AND full-text source material
+- Want to re-distill later with different parameters
+- Auditing what content was captured vs. what was extracted
+
+### `--no-distill` Flag
+
+Skips stages 4 (content-distiller) and 5 (quality-reviewer) entirely. The pipeline goes directly from crawl/store to export. Raw files become the **primary** output. Implies `--save-raw` automatically.
+
+```bash
+/reference-curator-pipeline https://docs.anthropic.com --no-distill
+```
+
+**Output:**
+```
+exports/anthropic-docs/
+├── README.md
+├── 00-getting-started.raw.md
+├── 01-prompt-engineering.raw.md
+├── anthropic-docs-raw-complete.md
+└── manifest.json
+```
+
+**Use cases:**
+- Site archival / preservation
+- Migration reference (need exact original content)
+- Training data collection (distillation would remove useful context)
+- NotebookLM / LLM context sources (full fidelity preferred)
+- Quick capture when you will distill manually later
+
+### Combining with Other Flags
+
+```bash
+# Deep archival crawl
+/reference-curator-pipeline https://legacy.example.com --depth full --no-distill
+
+# Standard crawl with raw backup and auto-approval
+/reference-curator-pipeline "MCP servers" --save-raw --auto-approve
+
+# Fine-tuning dataset from raw content
+/reference-curator-pipeline "prompt engineering" --no-distill --export-format fine_tuning
+```
+
+### Pipeline Stage Comparison
+
+| Stage | Default | `--save-raw` | `--no-distill` |
+|-------|---------|-------------|----------------|
+| 1. Discovery | Runs | Runs | Runs |
+| 2. Web Crawler | Runs | Runs + writes raw/ | Runs + writes raw |
+| 3. Repository | Runs | Runs | Runs |
+| 4. Distiller | Runs | Runs | **Skipped** |
+| 5. Reviewer | Runs | Runs | **Skipped** |
+| 6. Exporter | Distilled | Distilled + raw | **Raw only** |
+
+---
+
 ## Common Workflows
 
 ### Workflow 1: Quick Reference on a Topic
@@ -197,6 +286,8 @@ Claude: /markdown-exporter --format fine_tuning
 | Type | Location |
 |------|----------|
 | Raw content | `~/Documents/05_AI Agent/10_Reference Library/raw/` |
+| Raw output (--save-raw) | `{output}/{topic-slug}/raw/` |
+| Raw output (--no-distill) | `{output}/{topic-slug}/` (primary) |
 | Exports | `~/Documents/05_AI Agent/10_Reference Library/exports/` |
 | Config | `~/.config/reference-curator/` |
 
@@ -288,6 +379,34 @@ Claude: Starting reference-curator pipeline...
 Pipeline complete! Reference library ready.
 ```
 
+### Example: Archival Mode (`--no-distill`)
+
+```
+You: Archive the GA4 developer docs without distilling — just raw content
+
+Claude: Starting reference-curator pipeline (--no-distill mode)...
+
+[1/6] Reference Discovery
+- Skipped (URL input mode)
+
+[2/6] Web Crawling
+- Crawling developers.google.com/analytics with Firecrawl...
+- Saved 28 raw markdown files (.raw.md)
+
+[3/6] Content Repository
+- Stored 28 documents in MySQL
+
+[4/6] Content Distillation — SKIPPED (--no-distill)
+[5/6] Quality Review — SKIPPED (--no-distill)
+
+[6/6] Markdown Export (raw mode)
+- Generated 28 .raw.md files + combined raw bundle
+- Output: ~/Documents/.../google-analytics-4/
+
+Pipeline complete! 28 raw pages archived.
+Files ready for NotebookLM upload or direct LLM context use.
+```
+
 ---
 
 ## Need Help?