feat(reference-curator): add --save-raw, --no-distill flags + OurSEO crawler integration

- Add --save-raw flag: saves intact crawled markdown in raw/ subdirectory
  alongside distilled content, with YAML frontmatter for traceability
- Add --no-distill flag: skip stages 4-5 (distiller + QA reviewer),
  pure archival mode for raw documentation capture
- Integrate OurSEO BaseAsyncClient + PageAnalyzer as seo-aiohttp backend:
  token-bucket rate limiting, semaphore concurrency, tenacity retries,
  full page metadata extraction (headings, links, schema, OG)
- New seo_crawler_adapter.py: crawl URLs, sitemaps, link discovery, manifests
  with progress tracking and resume support
- Update crawler selection: docs sites now default to seo-aiohttp
- Update crawl_config.yaml with seo-aiohttp routing rules
- Update pipeline orchestrator with flag resolution and skip paths
- Update README.md and USER-GUIDE.md with raw mode documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

custom-skills/90-reference-curator/06-markdown-exporter/code/CLAUDE.md

@@ -10,6 +10,7 @@ Exports approved reference content as structured markdown files for project know
 | Type | Format | Use Case |
 |------|--------|----------|
 | `project` | Nested markdown | Claude Projects knowledge |
+| `raw` | Raw markdown | Archival, full-text reference, NotebookLM sources |
 | `finetuning` | JSONL | Model fine-tuning dataset |
 
 ## Workflow
@@ -74,11 +75,57 @@ JSONL format:
 uv run python scripts/exporter.py log --name "April 2026 Export" --type project_files --docs 45
 ```
 
+### Raw Export (when `--save-raw` or `--no-distill`)
+
+#### Raw alongside distilled (`--save-raw`)
+```bash
+uv run python scripts/exporter.py raw \
+  --output ~/reference-library/exports/{topic-slug}/raw/ \
+  --source-dir {crawl-raw-dir} \
+  --bundle {topic-slug}-raw-complete.md
+```
+
+Output structure:
+```
+exports/{topic-slug}/
+├── INDEX.md              # Includes links to raw/ section
+├── 00-page-name.md       # Distilled (normal)
+├── raw/
+│   ├── 00-page-name.raw.md
+│   ├── 01-page-name.raw.md
+│   └── {topic-slug}-raw-complete.md
+└── manifest.json
+```
+
+#### Raw-only mode (`--no-distill`)
+```bash
+uv run python scripts/exporter.py raw \
+  --output ~/reference-library/exports/{topic-slug}/ \
+  --source-dir {crawl-raw-dir} \
+  --bundle {topic-slug}-raw-complete.md \
+  --primary
+```
+
+Output structure (raw files are primary, no subdirectory wrapper):
+```
+exports/{topic-slug}/
+├── README.md
+├── 00-page-name.raw.md
+├── 01-page-name.raw.md
+├── {topic-slug}-raw-complete.md
+└── manifest.json
+```
+
+The `--primary` flag tells the exporter that raw files are the main output. README is generated from raw file frontmatter (source URLs, crawl timestamps).
+
+**Note:** When combined with `--export-format fine_tuning`, the JSONL exporter uses raw content directly as the assistant response with a simpler system prompt (no structured summaries available).
+
 ## Scripts
 
 | Command | Purpose |
 |---------|---------|
 | `exporter.py project` | Export as nested markdown files |
+| `exporter.py raw` | Export raw crawled markdown files with optional bundle |
 | `exporter.py finetuning` | Export as JSONL training dataset |
 | `exporter.py index` | Generate INDEX.md table of contents |
 | `exporter.py crossrefs` | Add cross-reference links |
@@ -90,4 +137,5 @@ uv run python scripts/exporter.py log --name "April 2026 Export" --type project_
 | From | To |
 |------|-----|
 | quality-reviewer (approved) | → |
+| web-crawler (raw files) | Raw markdown input (--save-raw / --no-distill) |
 | → | Project knowledge / Fine-tuning dataset |