our-claude-skills/docs/superpowers/specs/2026-04-28-notion-semantic-search-design.md

Notion Semantic Search (Skill 31) — Design

Date: 2026-04-28 Status: Approved (brainstorming) Scope: Phase 3b-i — semantic search across Notion workspaces with query expansion + LLM rerank Sequence: First of two sub-projects derived from the original Phase 3b "Notion-as-RAG export". The push-to-NotebookLM skill is the second sub-project, gets its own design after this ships. Predecessor: Phase 3c (commit 6446f00 — extended block coverage in 32-notion-writer)

---

Goal

Build a CLI skill that searches the user's Notion workspace semantically. Output is either a terminal-friendly table for browsing or JSON for piping into the (future) notion-to-notebooklm push skill.

The load-bearing problem: Notion's native search is keyword-only and weak — "AI agents" doesn't surface a page titled "Multi-agent orchestration", and Korean ↔ English queries don't cross-match. We close the gap with two LLM stages over Notion's API search: query expansion (Claude generates synonym/cross-language variants) and rerank (Claude scores candidates against the original query).

This is the foundation for the rest of the 31-notion-organizer vision — aggregation, move/reorganize, and exports all operate on search results, so getting search right unblocks everything else.

---

Non-goals

• Block-level result granularity. Page-level only in v1. Surfacing matching block snippets within pages adds complexity and most retrieval is page-level anyway. Defer until a real use case demands it.
• Cross-workspace search. Notion's integration model is per-workspace; abstracting over multiple workspaces means installing the integration in each. Document the limitation, don't try to hide it.
• Embedding-based search. Vector store + sync pipeline pushes this back into "build a tool" territory. Out of scope for the skill model.
• The push-to-NotebookLM skill. Separate spec, separate plan. Search outputs JSON; the push skill consumes it.
• Automatic pagination of search results. Cap at ~30 candidates pre-rerank; this is enough for top-10 reranked results. If a query genuinely has 100+ relevant matches, the user can constrain with --databases or --filter.

---

Architectural decisions

Decision	Choice	Rationale
Search strategy	Strategy C — query expansion + rerank	Notion's keyword search alone is the gap we're filling. Rerank-only catches synonyms but misses cross-language. Expansion + rerank covers both.
Workflow shape	Standalone search skill (browse-friendly), JSON output for downstream chaining	User searches frequently for memory-refresh; pushing to NotebookLM is occasional. Two independent skills, composable via JSON.
LLM model	Claude Haiku 4.5 for both expansion and rerank	Plenty good for "rank these 30 titles + properties by relevance"; ~$0.005/query, 8-15s total latency.
LLM client	anthropic SDK preferred, claude -p CLI fallback	SDK gives structured output; CLI works without separate API key for users on Claude Code subscriptions.
Caching	SHA256(query + sorted_candidate_ids) → JSON file in ~/.cache/notion-search/	Cheap, no infra, invalidates naturally when Notion content changes (different candidates → cache miss). 1-day TTL.
Property filter syntax	JSON object matching 32-notion-writer's --properties form	Consistency across the skill suite.
Failure mode	Permissive degradation	Rerank fail → return raw expanded-search; expansion fail → original query only. Match the parser philosophy from Phase 3c.

---

CLI surface

# Default — rerank + expand, terminal output
notion-search "AI agents in 2026"

# Pipe to JSON for the future push skill
notion-search "AI agents" --json | jq '.[].id'

# Constrain to specific databases
notion-search "AI agents" --databases f8f19ede-32bd-43ac-9f60-0651f6f40afe,abc-def-...

# Property filter (same JSON form as notion-writer's --properties)
notion-search "MCP" --filter '{"Status": "Done", "Topic": "AI"}'

# Fast paths
notion-search "exact term" --no-rerank   # Notion API only, no LLM
notion-search "exact term" --no-expand   # Skip variant generation
notion-search "AI agents" --limit 5      # Default 10
notion-search "AI agents" --no-cache     # Bypass result cache

Flags:

Flag	Default	Effect
--json	off	Emit JSON array instead of terminal table
--databases <ids>	all accessible	Comma-separated database/data_source IDs to constrain search
--filter <json>	none	Property filter applied per-database (skipped for workspace-wide search)
--no-rerank	off	Skip Claude rerank stage
--no-expand	off	Skip query-variant generation
--limit <n>	10	Number of results to return after rerank
--no-cache	off	Bypass cache lookup AND don't write cache

---

Pipeline

1. Query expansion (skipped if --no-expand)

Claude Haiku takes the original query and produces 3-5 variants covering:

• Synonyms ("AI agents" → "multi-agent orchestration", "autonomous LLM agents")
• Cross-language KR↔EN ("AI agents" → "AI 에이전트")
• Related concepts ("AI agents" → "agent SDK", "tool use")

The original query is always included. Total variants ≤ 5 to bound API calls.

Prompt approach: ask Claude for a JSON array of variants, parse with json.loads. On parse failure, fall back to original query only.

2. Search execution

For each variant, hit Notion API:

• Workspace-wide (client.search) when no --databases flag
• Per-database (client.data_sources.query) when --databases is provided. The existing _notion_compat.resolve_data_source_id resolves DB IDs.

Calls are made sequentially (no concurrent dispatch) to stay well under Notion's 3 req/sec average rate limit. With ≤5 variants × ≤2 DBs (typical case), that's at most 10 sequential calls — under 4 seconds total even at the slower end.

Each call returns up to 100 results; we cap candidates at 30 total (after dedup across all variants and DBs) to keep rerank costs predictable.

Dedupe by page ID. Preserve the highest position-rank if the same page appears for multiple variants (used as fallback ordering when rerank is skipped).

3. Property + excerpt fetch

For each candidate, gather:

• Title (from page.properties[title_prop])
• Key properties (Status, Topic, Type, Account Code, etc. — whatever exists)
• 200-char excerpt: fetch the first text-bearing block via client.blocks.children.list with page_size=5, walk the returned blocks and pick the first paragraph/heading/quote whose rich-text concatenates to non-empty content

Excerpt fallback: if none of the first 5 blocks have text content (e.g., page leads with an image, table-only, or empty), excerpt is the empty string. Rerank still runs — title + properties usually carry enough signal.

Properties often arrive inline with client.search results; we only re-fetch when properties are stripped (data source queries return full properties; workspace search doesn't).

4. Rerank (skipped if --no-rerank)

Cache lookup first: SHA256(original_query + sorted_candidate_ids).

Cache miss → call Claude Haiku with:

• Original (un-expanded) query
• Numbered list of candidates: [N] Title — Properties — Excerpt
• Asks for a JSON array of objects: {index, score, why} ordered by score descending

Parse, map back to candidate page objects, take top --limit.

Failure modes:

• LLM call error → return raw expanded-search results in candidate order, warn on stderr
• JSON parse error → same as above
• Rerank returns fewer than requested → take what's there, warn

5. Output

Terminal (default): formatted table with title, relevance, snippet, key properties, URL. ANSI color via rich library if available, plain text otherwise.

JSON (--json): array of objects with stable schema:

[
  {
    "id": "abc-def-...",                    // dashed UUID
    "url": "https://notion.so/...",
    "title": "MCP server architecture...",
    "relevance": 0.94,                       // 0.0-1.0; null if --no-rerank
    "snippet": "Direct match — covers...",   // null if --no-rerank
    "excerpt": "First paragraph text...",    // 200-char from page body
    "properties": {                          // only properties present on the page
      "Status": "Done",
      "Topic": ["AI", "MCP"],
      "Account Code": "D.intelligence"
    }
  }
]

The schema is stable: any future tool consuming search output (notion-to-notebooklm push, aggregation, etc.) reads this format.

---

File structure

File	Purpose	LOC
custom-skills/31-notion-organizer/code/scripts/notion_search.py	Main CLI + pipeline	~400
custom-skills/31-notion-organizer/code/scripts/test_notion_search.py	Unit tests with mocked Claude/Notion	~250
custom-skills/31-notion-organizer/code/scripts/_search_llm.py	LLM client abstraction (SDK + CLI fallback, prompt construction)	~150
custom-skills/31-notion-organizer/commands/notion-search.md	Slash command definition	~50
custom-skills/31-notion-organizer/code/CLAUDE.md	Add usage section	+60 lines
custom-skills/31-notion-organizer/code/scripts/requirements.txt	Add anthropic (optional, falls back to CLI)	+1 line

Total: ~850 LOC + tests + docs. ~3 days focused work.

The _notion_compat helper from Phase 2 is reused (client factory, error explanation, ID resolution). No new compatibility layer needed.

---

LLM client (_search_llm.py)

Abstraction so callers don't care whether SDK or CLI is in use.

def call_claude(prompt: str, model: str = "claude-haiku-4-5", max_tokens: int = 1000) -> str:
    """Return Claude's text response. Raises on failure."""
    if _have_anthropic_sdk() and os.getenv("ANTHROPIC_API_KEY"):
        return _call_via_sdk(prompt, model, max_tokens)
    if _have_claude_cli():
        return _call_via_cli(prompt, model)
    raise RuntimeError(
        "No Claude client available. Install `anthropic` and set "
        "ANTHROPIC_API_KEY, or install Claude Code CLI."
    )

Two thin implementations behind it:

• _call_via_sdk: standard anthropic.Anthropic().messages.create(...)
• _call_via_cli: subprocess.run(["claude", "-p", prompt, "--model", model], ...), capture stdout

Both return the assistant's text content as a single string. Callers (expand_query, rerank_candidates) parse JSON out of it.

---

Caching

Key: SHA256 hex digest of f"{query}|{','.join(sorted(candidate_ids))}".

Storage: ~/.cache/notion-search/<hash>.json. Each file contains:

{
  "query": "AI agents",
  "candidate_ids": ["abc...", "def..."],
  "results": [...],         // the reranked output
  "cached_at": "2026-04-28T14:23:00Z"
}

Lookup: if file exists and now - cached_at < TTL (default 24h), return cached results. Else run rerank and write fresh.

Invalidation: TTL-only for simplicity. --no-cache bypasses both read and write.

Why not invalidate on Notion content changes? That requires tracking last-edited timestamps and computing a content hash — significant complexity for marginal benefit. With a 1-day TTL, stale results are bounded; user can --no-cache for fresh runs.

---

Error handling

Failure	Behavior
Notion API: ObjectNotFound on --databases ID	Error out with friendly message via _notion_compat.explain_api_error
Notion API: Unauthorized	Same
Query expansion fails (LLM error or JSON parse)	Use original query only, warn on stderr, continue
Rerank fails (LLM error or JSON parse)	Return raw expanded-search results in candidate order, warn on stderr, continue
Cache file corrupted	Delete cache file, treat as miss
Empty Notion results	Print "No matches for ''" and exit 0 (not an error)
LLM client unavailable (no SDK + no CLI)	Error out with setup instructions if --no-rerank and --no-expand aren't both set; otherwise proceed
--filter JSON parse error	Error out before any API calls

---

Tests

10 tests covering the public surface, all using mocked Claude and Notion clients (no real API calls in tests):

Test	Verifies
test_expand_query_returns_variants	LLM mock returns variant list including original; expansion produces 3-5 unique strings
test_search_unions_and_dedupes	Two variants returning overlapping pages produce deduped candidate set
test_rerank_orders_by_relevance	Mock rerank returns scores; output sorted score-descending
test_no_rerank_returns_raw	--no-rerank skips LLM, returns Notion's native order, no relevance scores
test_no_expand_uses_single_query	--no-expand calls Notion once with original query, no LLM expansion
test_json_output_parseable	--json emits valid JSON conforming to the schema in Section 5
test_cache_hit_on_repeat	Second identical query skips LLM; cache file exists
test_cache_miss_on_different_candidates	Different candidate set hashes different → fresh rerank
test_property_filter_applied	--filter JSON passed to data_sources.query verbatim
test_database_scope_constrains	--databases causes per-DB queries instead of workspace search

Tests live in test_notion_search.py. Run via python3 test_notion_search.py (matching the existing test_parser.py convention in 32).

---

Out-of-scope follow-ups

• 3b-ii — notion-to-notebooklm push skill: consumes search-output JSON, pushes pages as NotebookLM sources. Separate spec after this ships.
• Block-level granularity: surface matching blocks within pages. Adds chunking logic and richer rerank prompts. Defer.
• Aggregation skill: "find pages on X, write a summary page with citations." Builds on this. Future Phase 3 sub-project.
• Multi-workspace search: requires per-workspace integrations and credential management. Future.

---

Implementation transition

After user approval, transition to superpowers:writing-plans to break this into bite-sized TDD tasks. Likely shape:

1. _search_llm.py skeleton (SDK + CLI client abstraction) + unit test
2. Query expansion module + tests
3. Search execution module (workspace + per-DB) + tests
4. Property + excerpt fetch + tests
5. Rerank module (with prompt + JSON parsing) + tests
6. Cache layer + tests
7. CLI assembly (argparse + output formatting) + tests
8. Slash command + 31's CLAUDE.md update + final integration smoke test

Each step independently testable. Total ~3 days estimated, mirrors Phase 3c's pacing.