directory changes and restructuring

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2025-12-22 02:01:41 +09:00
parent eea49f9f8c
commit 236be6c580
598 changed files with 0 additions and 0 deletions

View File

@@ -0,0 +1,95 @@
# Knowledge Capture Skill Evaluations
Evaluation scenarios for testing the Knowledge Capture skill across different Claude models.
## Purpose
These evaluations ensure the Knowledge Capture skill:
- Correctly identifies content types (how-to guides, FAQs, decision records, wikis)
- Extracts relevant information from conversations
- Structures content appropriately for each type
- Searches and places content in the right Notion location
- Works consistently across Haiku, Sonnet, and Opus
## Evaluation Files
### conversation-to-wiki.json
Tests capturing conversation content as a how-to guide for the team wiki.
**Scenario**: Save deployment discussion to wiki
**Key Behaviors**:
- Extracts steps, gotchas, and best practices from conversation
- Identifies content as How-To Guide
- Structures with proper sections (Overview, Prerequisites, Steps, Troubleshooting)
- Searches for team wiki location
- Preserves technical details (commands, configs)
### decision-record.json
Tests capturing architectural or technical decisions with full context.
**Scenario**: Document database migration decision
**Key Behaviors**:
- Extracts decision context, alternatives, and rationale
- Follows decision record structure (Context, Decision, Alternatives, Consequences)
- Captures both selected and rejected options with reasoning
- Places in decision log or ADR database
- Links to related technical documentation
## Running Evaluations
1. Enable the `knowledge-capture` skill
2. Submit the query from the evaluation file
3. Provide conversation context as specified
4. Verify all expected behaviors are met
5. Check success criteria for quality
6. Test with Haiku, Sonnet, and Opus
## Expected Skill Behaviors
Knowledge Capture evaluations should verify:
### Content Extraction
- Accurately captures key points from conversation context
- Preserves specific technical details, not generic placeholders
- Maintains context and nuance from discussion
### Content Type Selection
- Correctly identifies appropriate content type (how-to, FAQ, decision record, wiki page)
- Uses matching structure from reference documentation
- Applies proper Notion markdown formatting
### Notion Integration
- Searches for appropriate target location (wiki, decision log, etc.)
- Creates well-structured pages with clear titles
- Uses proper parent placement
- Includes discoverable titles and metadata
### Quality Standards
- Content is actionable and future-reference ready
- Technical accuracy is preserved
- Organization aids discoverability
- Formatting enhances readability
## Creating New Evaluations
When adding Knowledge Capture evaluations:
1. **Use realistic conversation content** - Include actual technical details, decisions, or processes
2. **Test different content types** - How-to guides, FAQs, decision records, meeting notes, learnings
3. **Vary complexity** - Simple captures vs. complex technical discussions
4. **Test discovery** - Finding the right wiki section or database
5. **Include edge cases** - Unclear content types, minimal context, overlapping categories
## Example Success Criteria
**Good** (specific, testable):
- "Structures content using How-To format with numbered steps"
- "Preserves exact bash commands from conversation"
- "Creates page with title format 'How to [Action]'"
- "Places in Engineering Wiki → Deployment section"
**Bad** (vague, untestable):
- "Creates good documentation"
- "Uses appropriate structure"
- "Saves to the right place"

View File

@@ -0,0 +1,31 @@
{
"name": "Save Conversation to Wiki",
"skills": ["knowledge-capture"],
"query": "Save this conversation about deploying our application to production to the team wiki",
"context": "Preceding conversation contains discussion about deployment process, including steps, gotchas, and best practices",
"expected_behavior": [
"Extracts key information from conversation context (deployment steps, gotchas, best practices)",
"Identifies content type as How-To Guide based on procedural nature",
"Structures content using How-To structure: Overview → Prerequisites → Steps (numbered) → Verification → Troubleshooting → Related",
"Organizes information into clear sections with proper headings",
"Includes specific commands, configurations, or examples from conversation",
"Adds context about why/when to use this process in Overview section",
"Notes common issues and solutions mentioned in discussion in Troubleshooting section",
"Uses Notion:notion-search to find team wiki location or asks user",
"Creates page using Notion:notion-create-pages with structured content and appropriate parent",
"Uses clear, descriptive title like 'How to Deploy to Production'",
"Applies Notion markdown formatting (headings, code blocks, bullets)",
"Suggests tags/categories for discoverability if wiki database"
],
"success_criteria": [
"Content is structured using How-To format from SKILL.md content types",
"Key points from conversation are captured accurately (not generic)",
"Information is organized with proper Notion markdown (##, ###, bullets, code blocks)",
"Specific technical details (commands, configs) are preserved from conversation",
"Document is written for future reference with clear step-by-step instructions",
"Title is searchable and descriptive (e.g., 'How to Deploy to Production')",
"Page is placed in appropriate wiki location (general wiki or specific section)",
"Uses correct tool name (Notion:notion-create-pages)"
]
}

View File

@@ -0,0 +1,31 @@
{
"name": "Create Decision Record",
"skills": ["knowledge-capture"],
"query": "Document our decision to use PostgreSQL instead of MongoDB for our new service",
"context": "User has just explained the decision with rationale, options considered, and trade-offs",
"expected_behavior": [
"Recognizes this as a decision record (architectural decision) from conversation context",
"Uses Decision structure: Context → Decision → Rationale → Options Considered (with Pros/Cons) → Consequences → Implementation",
"Extracts from context: decision made, options considered (PostgreSQL vs MongoDB), rationale, trade-offs",
"Creates document with proper structure including Date, Status (Accepted), and Deciders",
"Includes both positive and negative consequences (trade-offs) in Consequences section",
"Uses Notion:notion-search to check if decision log database exists",
"If database exists, asks whether to add there or create standalone page",
"If creating in database, fetches schema using Notion:notion-fetch and sets properties: Decision title, Date, Status, Domain (Architecture), Deciders, Impact",
"Uses Notion:notion-create-pages with parent: { data_source_id } for database or { page_id } for parent page",
"Applies proper Notion markdown formatting with sections",
"Suggests linking from architecture docs or project pages"
],
"success_criteria": [
"Document follows Decision structure from SKILL.md content types",
"All key sections present: Context, Decision, Rationale, Options Considered (with Pros/Cons for each), Consequences, Implementation",
"Decision is clearly stated (PostgreSQL chosen over MongoDB)",
"Options that were considered are documented with pros/cons structure",
"Rationale explains why PostgreSQL was chosen based on conversation context",
"Consequences include both positive (benefits) and negative (trade-offs)",
"If in database, properties are set correctly from schema (Decision, Date, Status: Accepted, Domain: Architecture, Impact)",
"Document is dated and has status 'Accepted'",
"Uses correct tool names (Notion:notion-search, Notion:notion-fetch, Notion:notion-create-pages)"
]
}