Files

Andrew Yim f215c11c32 feat(reference-curator): implement Python scripts + Gemini quality gate

Build the refcurator shared Python package and 7 CLI scripts that were
previously specification-only. Add Gemini CLI as an independent pre-distillation
quality evaluator, replacing the circular Claude-self-review pattern.

Key changes:
- shared/lib/src/refcurator/: 7-module package (config, db, models, utils,
  manifest, gemini) with PyMySQL + JSON file dual backend
- 7 Click CLI scripts: discover, crawl_mgr, repo, distiller, reviewer,
  exporter, pipeline — each with subcommands for data management
- Gemini quality gate: evaluates raw content BEFORE distillation using
  5 criteria (relevance, authority, completeness, freshness, distill_value)
- Pipeline reordered: discovery → crawl → store → evaluate → distill → export
- Bug fixes from Codex adversarial review:
  - FileBackend now hard-fails on JOIN/aggregate/GROUP BY queries
  - Exporter uses MAX(review_id) to prevent shipping stale approvals
  - Distiller updates existing rows on refactor instead of forking
- Updated all 7 CLAUDE.md directives with real script references
- install.sh updated with refcurator package install step

51/51 E2E tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-12 18:22:28 +09:00

3.2 KiB

Raw Blame History

Quality Reviewer

Pre-distillation quality gate using Gemini CLI as an independent evaluator. Assesses raw crawled content before distillation to filter out low-quality sources early. Also supports manual scoring and routing for edge cases.

Trigger Keywords

"review content", "quality check", "QA review", "evaluate sources", "check reference quality"

Primary Flow: Gemini Pre-Distillation Gate

[Raw Crawled Content]
       │
       ▼
┌────────────────────┐
│ Gemini CLI Eval    │ → relevance, authority, completeness, freshness, distill_value
└────────────────────┘
       │
       ├── ≥ 0.75 → APPROVE → proceed to distillation
       ├── 0.50-0.74 → DEEP_RESEARCH → re-crawl for better sources
       └── < 0.50 → REJECT → skip distillation entirely

Evaluation Criteria (Gemini)

Criterion	Weight	What It Checks
Relevance	0.25	Does content match the curation topic?
Authority	0.25	Official docs / research paper, or blog spam?
Completeness	0.20	Full article, or nav fragment / error page / stub?
Freshness	0.15	Up-to-date or outdated information?
Distill Value	0.15	Unique info worth summarizing, or redundant?

Workflow

Step 1: Evaluate Single Document

uv run python scripts/reviewer.py gemini-evaluate --doc-id 123 --topic "prompt engineering"

# With auto-logging of decision:
uv run python scripts/reviewer.py gemini-evaluate --doc-id 123 --topic "prompt engineering" --auto-approve

Step 2: Batch Evaluate All Pending

uv run python scripts/reviewer.py gemini-evaluate-pending --topic "prompt engineering" --auto-approve --limit 20

Step 3: Manual Review (Edge Cases)

For documents where Gemini evaluation fails or needs human judgment:

# Calculate score from manual assessment
uv run python scripts/reviewer.py calculate-score --assessment assessment.json

# Route based on score
uv run python scripts/reviewer.py route --score 0.78

# Log review decision
uv run python scripts/reviewer.py log-review \
  --distill-id 123 --decision approve --score 0.85 \
  --feedback "Manually verified"

Step 4: Review History

uv run python scripts/reviewer.py history --distill-id 123

Prerequisites

Gemini CLI: npm install -g @google/gemini-cli
Google auth: gemini (run once interactively to authenticate)
refcurator package installed

Scripts

Command	Purpose
`reviewer.py gemini-evaluate`	Evaluate single doc via Gemini CLI
`reviewer.py gemini-evaluate-pending`	Batch evaluate all pending docs
`reviewer.py calculate-score`	Manual weighted score calculation
`reviewer.py route`	Decision routing from score
`reviewer.py log-review`	Log review decision to DB
`reviewer.py load-pending`	Get pending reviews
`reviewer.py history`	Show review history

Integration

From	Action	To
content-repository (raw docs)	Gemini evaluation	→
→	APPROVE	content-distiller
→	DEEP_RESEARCH	web-crawler-orchestrator
→	REJECT	archive (skip distillation)

3.2 KiB Raw Blame History