Initial commit: Claude Skills Factory with 8 refined custom skills
Custom Skills (ourdigital-custom-skills/): - 00-ourdigital-visual-storytelling: Blog featured image prompt generator - 01-ourdigital-research-publisher: Research-to-publication workflow - 02-notion-organizer: Notion workspace management - 03-research-to-presentation: Notion research to PPT/Figma - 04-seo-gateway-strategist: SEO gateway page strategy planning - 05-gateway-page-content-builder: Gateway page content generation - 20-jamie-brand-editor: Jamie Clinic branded content GENERATION - 21-jamie-brand-guardian: Jamie Clinic content REVIEW & evaluation Refinements applied: - All skills converted to SKILL.md format with YAML frontmatter - Added version fields to all skills - Flattened nested folder structures - Removed packaging artifacts (.zip, .skill files) - Reorganized file structures (scripts/, references/, etc.) - Differentiated Jamie skills with clear roles 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,120 @@
|
||||
# Spec to Implementation Skill Evaluations
|
||||
|
||||
Evaluation scenarios for testing the Spec to Implementation skill across different Claude models.
|
||||
|
||||
## Purpose
|
||||
|
||||
These evaluations ensure the Spec to Implementation skill:
|
||||
- Finds and parses specification pages accurately
|
||||
- Breaks down specs into actionable implementation plans
|
||||
- Creates tasks that Claude can implement with clear acceptance criteria
|
||||
- Tracks progress and updates implementation status
|
||||
- Works consistently across Haiku, Sonnet, and Opus
|
||||
|
||||
## Evaluation Files
|
||||
|
||||
### basic-spec-implementation.json
|
||||
Tests basic workflow of turning a spec into an implementation plan.
|
||||
|
||||
**Scenario**: Implement user authentication feature from spec
|
||||
**Key Behaviors**:
|
||||
- Searches for and finds the authentication spec page
|
||||
- Fetches spec and extracts requirements
|
||||
- Parses requirements into phases (setup, core features, polish)
|
||||
- Creates implementation plan page linked to original spec
|
||||
- Breaks down into clear phases with deliverables
|
||||
- Includes timeline and dependencies
|
||||
|
||||
### spec-to-tasks.json
|
||||
Tests creating concrete tasks from a specification in a task database.
|
||||
|
||||
**Scenario**: Create tasks from API redesign spec
|
||||
**Key Behaviors**:
|
||||
- Finds spec page in Notion
|
||||
- Extracts specific requirements and acceptance criteria
|
||||
- Searches for or creates task database
|
||||
- Fetches task database schema
|
||||
- Creates multiple tasks with proper properties (Status, Priority, Sprint, etc.)
|
||||
- Each task has clear title, description, and acceptance criteria
|
||||
- Tasks have dependencies where appropriate
|
||||
- Links all tasks back to original spec
|
||||
|
||||
## Running Evaluations
|
||||
|
||||
1. Enable the `spec-to-implementation` skill
|
||||
2. Submit the query from the evaluation file
|
||||
3. Verify the skill finds the spec page via search
|
||||
4. Check that requirements are accurately parsed
|
||||
5. Confirm implementation plan is created with phases
|
||||
6. Verify tasks have clear, implementable acceptance criteria
|
||||
7. Check that tasks link back to spec
|
||||
8. Test with Haiku, Sonnet, and Opus
|
||||
|
||||
## Expected Skill Behaviors
|
||||
|
||||
Spec to Implementation evaluations should verify:
|
||||
|
||||
### Spec Discovery & Parsing
|
||||
- Searches Notion for specification pages
|
||||
- Fetches complete spec content
|
||||
- Extracts all requirements accurately
|
||||
- Identifies technical dependencies
|
||||
- Understands acceptance criteria
|
||||
- Notes any ambiguities or missing details
|
||||
|
||||
### Implementation Planning
|
||||
- Creates implementation plan page
|
||||
- Breaks work into logical phases:
|
||||
- Phase 1: Foundation/Setup
|
||||
- Phase 2: Core Implementation
|
||||
- Phase 3: Testing & Polish
|
||||
- Includes timeline estimates
|
||||
- Identifies dependencies between phases
|
||||
- Links back to original spec
|
||||
|
||||
### Task Creation
|
||||
- Finds or identifies task database
|
||||
- Fetches database schema for property names
|
||||
- Creates tasks with correct properties
|
||||
- Each task has:
|
||||
- Clear, specific title
|
||||
- Context and description
|
||||
- Acceptance criteria (checklist format)
|
||||
- Appropriate priority and status
|
||||
- Link to spec page
|
||||
- Tasks are right-sized (not too big, not too small)
|
||||
- Dependencies between tasks are noted
|
||||
|
||||
### Progress Tracking
|
||||
- Implementation plan includes progress markers
|
||||
- Tasks can be updated as work progresses
|
||||
- Status updates link to completed work
|
||||
- Blockers or changes are noted
|
||||
|
||||
## Creating New Evaluations
|
||||
|
||||
When adding Spec to Implementation evaluations:
|
||||
|
||||
1. **Test different spec types** - Features, migrations, refactors, API changes, UI components
|
||||
2. **Vary complexity** - Simple 1-phase vs. complex multi-phase implementations
|
||||
3. **Test task granularity** - Does it create appropriately-sized tasks?
|
||||
4. **Include edge cases** - Vague specs, conflicting requirements, missing details
|
||||
5. **Test database integration** - Creating tasks in existing task databases with various schemas
|
||||
6. **Progress tracking** - Updating implementation plans as tasks complete
|
||||
|
||||
## Example Success Criteria
|
||||
|
||||
**Good** (specific, testable):
|
||||
- "Searches Notion for spec page using feature name"
|
||||
- "Creates implementation plan with 3 phases: Setup → Core → Polish"
|
||||
- "Creates 5-8 tasks in task database with properties: Task (title), Status, Priority, Sprint"
|
||||
- "Each task has acceptance criteria in checklist format (- [ ] ...)"
|
||||
- "Tasks link back to spec using mention-page tag"
|
||||
- "Task titles are specific and actionable (e.g., 'Create login API endpoint' not 'Authentication')"
|
||||
|
||||
**Bad** (vague, untestable):
|
||||
- "Creates good implementation plan"
|
||||
- "Tasks are well-structured"
|
||||
- "Breaks down spec appropriately"
|
||||
- "Links to spec"
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"name": "Create Implementation Plan from Spec",
|
||||
"skills": ["spec-to-implementation"],
|
||||
"query": "Create an implementation plan for the User Authentication spec page",
|
||||
"expected_behavior": [
|
||||
"Step 1: Uses Notion:notion-search to find 'User Authentication spec' with keywords like 'User Authentication' or 'auth spec'",
|
||||
"Step 2: If not found or ambiguous, asks user for spec page URL/ID",
|
||||
"Step 3: Fetches spec page using Notion:notion-fetch with URL/ID from search results",
|
||||
"Step 4: Parses spec using patterns from reference/spec-parsing.md to extract requirements, acceptance criteria, constraints",
|
||||
"Step 5: Identifies functional requirements (user stories, features, workflows) and non-functional requirements (performance, security)",
|
||||
"Step 6: Creates implementation plan following structure from reference/templates.md",
|
||||
"Step 7: Includes sections: Overview, Linked Spec, Requirements Summary, Technical Approach, Implementation Phases",
|
||||
"Step 8: Breaks work into logical phases with Goal, Tasks checklist, Estimated effort per phase",
|
||||
"Step 9: Identifies dependencies and risks from spec content",
|
||||
"Step 10: Links plan back to original spec page using <mention-page url='...'>",
|
||||
"Step 11: Creates plan page using Notion:notion-create-pages with appropriate title (e.g., 'Implementation Plan: User Authentication')",
|
||||
"Step 12: Places plan appropriately (asks user or suggests under project/spec parent)"
|
||||
],
|
||||
"success_criteria": [
|
||||
"Spec is found using Notion:notion-search before attempting to fetch (or user is asked for URL if not found)",
|
||||
"Spec is fetched using Notion:notion-fetch with correct URL/ID from search results",
|
||||
"Plan includes clear overview and spec link with mention-page tag",
|
||||
"Requirements are extracted from actual spec content (not generic) using spec-parsing patterns",
|
||||
"Work is broken into multiple phases (typically 3-5) following template structure",
|
||||
"Each phase has Goal, Tasks (as checkboxes), and Estimated effort",
|
||||
"Dependencies and risks sections are included with specific details from spec",
|
||||
"Plan follows Implementation Plan structure from reference/templates.md",
|
||||
"Success criteria or acceptance criteria from spec are referenced in plan",
|
||||
"Uses correct tool names and sequence: Notion:notion-search → Notion:notion-fetch → Notion:notion-create-pages"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "Create Tasks from Specification",
|
||||
"skills": ["spec-to-implementation", "task-manager"],
|
||||
"query": "Read the Payment Integration spec and create implementation tasks in our Tasks database",
|
||||
"expected_behavior": [
|
||||
"Step 1: Uses Notion:notion-search to find Payment Integration spec or asks for URL",
|
||||
"Step 2: Fetches spec page using Notion:notion-fetch to read full content",
|
||||
"Step 3: Parses spec using reference/spec-parsing.md patterns to identify work items",
|
||||
"Step 4: Breaks down into appropriately-sized tasks using breakdown patterns from reference/task-creation.md",
|
||||
"Step 5: Uses Notion:notion-search to find Tasks database location",
|
||||
"Step 6: Fetches Tasks database using Notion:notion-fetch to get schema, property names, and data sources",
|
||||
"Step 7: Identifies correct data source from <data-source> tags in fetch results",
|
||||
"Step 8: Optionally creates implementation plan page first (recommended per workflow)",
|
||||
"Step 9: For each task - creates task page using Notion:notion-create-pages with parent: { data_source_id: 'collection://...' }",
|
||||
"Step 10: Sets task properties from schema: Title, Status (To Do), Priority, Related Tasks (link to spec)",
|
||||
"Step 11: Includes task description with context, acceptance criteria from spec, dependencies",
|
||||
"Step 12: Links tasks to spec page using <mention-page> and to each other for dependencies",
|
||||
"Step 13: Sequences tasks appropriately (setup → implementation → testing per reference/task-creation.md)",
|
||||
"Step 14: Reports summary: 'Created X tasks for Payment Integration: [task list with links]'"
|
||||
],
|
||||
"success_criteria": [
|
||||
"Spec is found using Notion:notion-search before attempting to fetch",
|
||||
"Task database is found using Notion:notion-search before attempting to fetch schema",
|
||||
"Database schema is fetched and data source identified from <data-source> tags",
|
||||
"Multiple tasks created (at least 3-5) covering spec scope",
|
||||
"Tasks are right-sized per reference/task-creation.md (1-2 days each)",
|
||||
"Each task has clear acceptance criteria extracted from spec",
|
||||
"Tasks are properly sequenced with dependencies using relation properties",
|
||||
"All tasks link back to original spec using mention-page tags",
|
||||
"Task properties match exact schema from fetched database",
|
||||
"Tasks use correct parent (data_source_id: 'collection://...') for database placement",
|
||||
"Uses correct tool names and sequence: Notion:notion-search (2x) → Notion:notion-fetch (2x) → Notion:notion-create-pages (Nx)"
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user