feat(reference-curator): add --save-raw, --no-distill flags + OurSEO crawler integration

- Add --save-raw flag: saves intact crawled markdown in raw/ subdirectory
  alongside distilled content, with YAML frontmatter for traceability
- Add --no-distill flag: skip stages 4-5 (distiller + QA reviewer),
  pure archival mode for raw documentation capture
- Integrate OurSEO BaseAsyncClient + PageAnalyzer as seo-aiohttp backend:
  token-bucket rate limiting, semaphore concurrency, tenacity retries,
  full page metadata extraction (headings, links, schema, OG)
- New seo_crawler_adapter.py: crawl URLs, sitemaps, link discovery, manifests
  with progress tracking and resume support
- Update crawler selection: docs sites now default to seo-aiohttp
- Update crawl_config.yaml with seo-aiohttp routing rules
- Update pipeline orchestrator with flag resolution and skip paths
- Update README.md and USER-GUIDE.md with raw mode documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

custom-skills/90-reference-curator/commands/reference-curator-pipeline.md

@@ -1,6 +1,6 @@
 ---
 description: Orchestrates full reference curation pipeline with configurable depth. Runs discovery > crawl > store > distill > review > export with QA loop handling.
-argument-hint: <topic|urls|manifest> [--depth light|standard|deep|full] [--output ~/Documents/reference-library/] [--max-sources 100] [--max-pages 50] [--auto-approve] [--threshold 0.85] [--max-iterations 3] [--export-format project_files]
+argument-hint: <topic|urls|manifest> [--depth light|standard|deep|full] [--output ~/Documents/reference-library/] [--max-sources 100] [--max-pages 50] [--auto-approve] [--threshold 0.85] [--max-iterations 3] [--export-format project_files] [--save-raw] [--no-distill]
 allowed-tools: WebSearch, WebFetch, Read, Write, Bash, Grep, Glob, Task
 ---
 
@@ -29,6 +29,8 @@ Full-stack orchestration of the 6-skill reference curation workflow.
 - `--export-format`: Output format: `project_files`, `fine_tuning`, `jsonl` (default: project_files)
 - `--include-subdomains`: Include subdomains in site mapping (default: false)
 - `--follow-external`: Follow external links found in content (default: false)
+- `--save-raw`: Save intact crawled markdown alongside distilled content in a `raw/` subdirectory. Each file uses `.raw.md` extension. A combined raw bundle `{topic-slug}-raw-complete.md` is also generated.
+- `--no-distill`: Skip stages 4-5 (content-distiller and quality-reviewer) entirely. Pure archival mode. Implies `--save-raw`. Raw crawled files become the primary output — no distilled subdirectory is created.
 
 ## Output Directory
 
@@ -38,13 +40,13 @@ The `--output` argument sets the base path for all pipeline output. If omitted,
 
 ### Directory Structure
 
+**Default (distilled output):**
 ```
 {output}/
-├── {topic-slug}/                   # One folder per pipeline run
+├── {topic-slug}/
 │   ├── README.md                   # Index with table of contents
-│   ├── 00-page-name.md             # Individual page files
+│   ├── 00-page-name.md             # Distilled individual page files
 │   ├── 01-page-name.md
-│   ├── ...
 │   ├── {topic-slug}-complete.md    # Combined bundle (all pages)
 │   └── manifest.json               # Crawl metadata
 ├── pipeline_state/                 # Resume state (auto-managed)
@@ -52,6 +54,31 @@ The `--output` argument sets the base path for all pipeline output. If omitted,
 └── exports/                        # Fine-tuning / JSONL exports
 ```
 
+**With `--save-raw` (distilled + raw):**
+```
+{output}/
+├── {topic-slug}/
+│   ├── README.md
+│   ├── 00-page-name.md             # Distilled
+│   ├── {topic-slug}-complete.md    # Combined distilled
+│   ├── raw/                        # Intact crawled markdown
+│   │   ├── 00-page-name.raw.md
+│   │   ├── 01-page-name.raw.md
+│   │   └── {topic-slug}-raw-complete.md  # Combined raw bundle
+│   └── manifest.json
+```
+
+**With `--no-distill` (raw only, archival):**
+```
+{output}/
+├── {topic-slug}/
+│   ├── README.md
+│   ├── 00-page-name.raw.md         # Raw crawled (primary output)
+│   ├── 01-page-name.raw.md
+│   ├── {topic-slug}-raw-complete.md # Combined raw bundle
+│   └── manifest.json
+```
+
 ### Resolution Rules
 
 1. If `--output` is provided, use that path exactly
@@ -158,15 +185,17 @@ full        ████████████████  Speed: slow
 
 ```
 1. reference-discovery  (topic mode only)
-2. web-crawler              <- depth controls this stage
+2. web-crawler              ← depth controls this stage
+                            ← --save-raw: writes raw/ alongside crawl
 3. content-repository
-4. content-distiller    <--------+
-5. quality-reviewer              |
+4. content-distiller    <--------+    ← SKIPPED when --no-distill
+5. quality-reviewer              |    ← SKIPPED when --no-distill
    +-- APPROVE -> export         |
    +-- REFACTOR -----------------+
    +-- DEEP_RESEARCH -> crawler -+
    +-- REJECT -> archive
-6. markdown-exporter
+6. markdown-exporter            ← --no-distill: exports raw files only
+                                ← --save-raw: also exports raw/ bundle
 ```
 
 ## Crawl Execution by Depth
@@ -234,6 +263,12 @@ After max iterations, document marked as `needs_manual_review`.
 
 # Resume from existing manifest
 /reference-curator ./manifest.json --auto-approve
+
+# Save raw crawled content alongside distilled output
+/reference-curator https://docs.anthropic.com --depth standard --save-raw
+
+# Pure archival — no distillation, just raw markdown files
+/reference-curator https://docs.anthropic.com --depth full --no-distill
 ```
 
 ## State Management