Commit Graph

5 Commits

Author SHA1 Message Date
b1c2dca080 feat(reference-curator): add --save-raw, --no-distill flags + OurSEO crawler integration
- Add --save-raw flag: saves intact crawled markdown in raw/ subdirectory
  alongside distilled content, with YAML frontmatter for traceability
- Add --no-distill flag: skip stages 4-5 (distiller + QA reviewer),
  pure archival mode for raw documentation capture
- Integrate OurSEO BaseAsyncClient + PageAnalyzer as seo-aiohttp backend:
  token-bucket rate limiting, semaphore concurrency, tenacity retries,
  full page metadata extraction (headings, links, schema, OG)
- New seo_crawler_adapter.py: crawl URLs, sitemaps, link discovery, manifests
  with progress tracking and resume support
- Update crawler selection: docs sites now default to seo-aiohttp
- Update crawl_config.yaml with seo-aiohttp routing rules
- Update pipeline orchestrator with flag resolution and skip paths
- Update README.md and USER-GUIDE.md with raw mode documentation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 23:26:50 +09:00
f215c11c32 feat(reference-curator): implement Python scripts + Gemini quality gate
Build the refcurator shared Python package and 7 CLI scripts that were
previously specification-only. Add Gemini CLI as an independent pre-distillation
quality evaluator, replacing the circular Claude-self-review pattern.

Key changes:
- shared/lib/src/refcurator/: 7-module package (config, db, models, utils,
  manifest, gemini) with PyMySQL + JSON file dual backend
- 7 Click CLI scripts: discover, crawl_mgr, repo, distiller, reviewer,
  exporter, pipeline — each with subcommands for data management
- Gemini quality gate: evaluates raw content BEFORE distillation using
  5 criteria (relevance, authority, completeness, freshness, distill_value)
- Pipeline reordered: discovery → crawl → store → evaluate → distill → export
- Bug fixes from Codex adversarial review:
  - FileBackend now hard-fails on JOIN/aggregate/GROUP BY queries
  - Exporter uses MAX(review_id) to prevent shipping stale approvals
  - Distiller updates existing rows on refactor instead of forking
- Updated all 7 CLAUDE.md directives with real script references
- install.sh updated with refcurator package install step

51/51 E2E tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 18:22:28 +09:00
Andrew Yim
b6a478e1df feat: Add installation tool, Claude.ai export, and skill standardization (#1)
## Summary

- Add portable installation tool (`install.sh`) for cross-machine setup
- Add Claude.ai export files with proper YAML frontmatter
- Add multi-agent-guide v2.0 with consolidated framework template
- Rename `00-claude-code-setting` → `00-our-settings-audit` (avoid reserved word)
- Add YAML frontmatter to 25+ SKILL.md files for Claude Desktop compatibility

## Commits Included

- `93f604a` feat: Add portable installation tool for cross-machine setup
- `9b84104` feat: Add Claude.ai export for portable skill installation
- `f7ab973` fix: Add YAML frontmatter to Claude.ai export files
- `3fed49a` feat(multi-agent-guide): Add v2.0 with consolidated framework
- `3be26ef` refactor: Rename settings-audit skill and add YAML frontmatter

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 16:48:06 +07:00
d1cd1298a8 feat(reference-curator): Add pipeline orchestrator and refactor skill format
Pipeline Orchestrator:
- Add 07-pipeline-orchestrator skill with code/CLAUDE.md and desktop/SKILL.md
- Add /reference-curator-pipeline slash command for full workflow automation
- Add pipeline_runs and pipeline_iteration_tracker tables to schema.sql
- Add v_pipeline_status and v_pipeline_iterations views
- Add pipeline_config.yaml configuration template
- Update AGENTS.md with Reference Curator Skills section
- Update claude-project files with pipeline documentation

Skill Format Refactoring:
- Extract YAML frontmatter from SKILL.md files to separate skill.yaml
- Add tools/ directories with MCP tool documentation
- Update SKILL-FORMAT-REQUIREMENTS.md with new structure
- Add migrate-skill-structure.py script for format conversion

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:01:02 +07:00
6d7a6d7a88 feat(reference-curator): Add portable skill suite for reference documentation curation
6 modular skills for curating, processing, and exporting reference docs:
- reference-discovery: Search and validate authoritative sources
- web-crawler-orchestrator: Multi-backend crawling (Firecrawl/Node/aiohttp/Scrapy)
- content-repository: MySQL storage with version tracking
- content-distiller: Summarization and key concept extraction
- quality-reviewer: QA loop with approve/refactor/research routing
- markdown-exporter: Structured output for Claude Projects or fine-tuning

Cross-machine installation support:
- Environment-based config (~/.reference-curator.env)
- Commands tracked in repo, symlinked during install
- install.sh with --minimal, --check, --uninstall modes
- Firecrawl MCP as default (always available)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 00:20:27 +07:00