our-claude-skills/docs/superpowers/specs/2026-04-28-notion-semantic-search-design.md

# Notion Semantic Search (Skill 31) — Design

> **Date**: 2026-04-28
> **Status**: Approved (brainstorming)
> **Scope**: Phase 3b-i — semantic search across Notion workspaces with query expansion + LLM rerank
> **Sequence**: First of two sub-projects derived from the original Phase 3b "Notion-as-RAG export". The push-to-NotebookLM skill is the second sub-project, gets its own design after this ships.
> **Predecessor**: Phase 3c (commit `6446f00` — extended block coverage in `32-notion-writer`)

---

## Goal

Build a CLI skill that searches the user's Notion workspace semantically. Output is either a terminal-friendly table for browsing or JSON for piping into the (future) `notion-to-notebooklm` push skill.

The load-bearing problem: Notion's native search is keyword-only and weak — "AI agents" doesn't surface a page titled "Multi-agent orchestration", and Korean ↔ English queries don't cross-match. We close the gap with two LLM stages over Notion's API search: query expansion (Claude generates synonym/cross-language variants) and rerank (Claude scores candidates against the original query).

This is the foundation for the rest of the 31-notion-organizer vision — aggregation, move/reorganize, and exports all operate on search results, so getting search right unblocks everything else.

---

## Non-goals

- **Block-level result granularity.** Page-level only in v1. Surfacing matching block snippets within pages adds complexity and most retrieval is page-level anyway. Defer until a real use case demands it.
- **Cross-workspace search.** Notion's integration model is per-workspace; abstracting over multiple workspaces means installing the integration in each. Document the limitation, don't try to hide it.
- **Embedding-based search.** Vector store + sync pipeline pushes this back into "build a tool" territory. Out of scope for the skill model.
- **The push-to-NotebookLM skill.** Separate spec, separate plan. Search outputs JSON; the push skill consumes it.
- **Automatic pagination of search results.** Cap at ~30 candidates pre-rerank; this is enough for top-10 reranked results. If a query genuinely has 100+ relevant matches, the user can constrain with `--databases` or `--filter`.

---

## Architectural decisions

| Decision | Choice | Rationale |
|---|---|---|
| Search strategy | **Strategy C — query expansion + rerank** | Notion's keyword search alone is the gap we're filling. Rerank-only catches synonyms but misses cross-language. Expansion + rerank covers both. |
| Workflow shape | **Standalone search skill** (browse-friendly), JSON output for downstream chaining | User searches frequently for memory-refresh; pushing to NotebookLM is occasional. Two independent skills, composable via JSON. |
| LLM model | **Claude Haiku 4.5** for both expansion and rerank | Plenty good for "rank these 30 titles + properties by relevance"; ~$0.005/query, 8-15s total latency. |
| LLM client | **`anthropic` SDK preferred, `claude -p` CLI fallback** | SDK gives structured output; CLI works without separate API key for users on Claude Code subscriptions. |
| Caching | **SHA256(query + sorted_candidate_ids) → JSON file** in `~/.cache/notion-search/` | Cheap, no infra, invalidates naturally when Notion content changes (different candidates → cache miss). 1-day TTL. |
| Property filter syntax | **JSON object** matching `32-notion-writer`'s `--properties` form | Consistency across the skill suite. |
| Failure mode | **Permissive degradation** | Rerank fail → return raw expanded-search; expansion fail → original query only. Match the parser philosophy from Phase 3c. |

---

## CLI surface

```bash
# Default — rerank + expand, terminal output
notion-search "AI agents in 2026"

# Pipe to JSON for the future push skill
notion-search "AI agents" --json | jq '.[].id'

# Constrain to specific databases
notion-search "AI agents" --databases f8f19ede-32bd-43ac-9f60-0651f6f40afe,abc-def-...

# Property filter (same JSON form as notion-writer's --properties)
notion-search "MCP" --filter '{"Status": "Done", "Topic": "AI"}'

# Fast paths
notion-search "exact term" --no-rerank   # Notion API only, no LLM
notion-search "exact term" --no-expand   # Skip variant generation
notion-search "AI agents" --limit 5      # Default 10
notion-search "AI agents" --no-cache     # Bypass result cache
```

**Flags:**

| Flag | Default | Effect |
|---|---|---|
| `--json` | off | Emit JSON array instead of terminal table |
| `--databases <ids>` | all accessible | Comma-separated database/data_source IDs to constrain search |
| `--filter <json>` | none | Property filter applied per-database (skipped for workspace-wide search) |
| `--no-rerank` | off | Skip Claude rerank stage |
| `--no-expand` | off | Skip query-variant generation |
| `--limit <n>` | 10 | Number of results to return after rerank |
| `--no-cache` | off | Bypass cache lookup AND don't write cache |

---

## Pipeline

### 1. Query expansion (skipped if `--no-expand`)

Claude Haiku takes the original query and produces 3-5 variants covering:
- Synonyms ("AI agents" → "multi-agent orchestration", "autonomous LLM agents")
- Cross-language KR↔EN ("AI agents" → "AI 에이전트")
- Related concepts ("AI agents" → "agent SDK", "tool use")

The original query is always included. Total variants ≤ 5 to bound API calls.

Prompt approach: ask Claude for a JSON array of variants, parse with `json.loads`. On parse failure, fall back to original query only.

### 2. Search execution

For each variant, hit Notion API:
- Workspace-wide (`client.search`) when no `--databases` flag
- Per-database (`client.data_sources.query`) when `--databases` is provided. The existing `_notion_compat.resolve_data_source_id` resolves DB IDs.

Calls are made sequentially (no concurrent dispatch) to stay well under Notion's 3 req/sec average rate limit. With ≤5 variants × ≤2 DBs (typical case), that's at most 10 sequential calls — under 4 seconds total even at the slower end.

Each call returns up to 100 results; we cap candidates at 30 total (after dedup across all variants and DBs) to keep rerank costs predictable.

Dedupe by page ID. Preserve the highest position-rank if the same page appears for multiple variants (used as fallback ordering when rerank is skipped).

### 3. Property + excerpt fetch

For each candidate, gather:
- Title (from page.properties[title_prop])
- Key properties (Status, Topic, Type, Account Code, etc. — whatever exists)
- 200-char excerpt: fetch the first text-bearing block via `client.blocks.children.list` with `page_size=5`, walk the returned blocks and pick the first paragraph/heading/quote whose rich-text concatenates to non-empty content

Excerpt fallback: if none of the first 5 blocks have text content (e.g., page leads with an image, table-only, or empty), excerpt is the empty string. Rerank still runs — title + properties usually carry enough signal.

Properties often arrive inline with `client.search` results; we only re-fetch when properties are stripped (data source queries return full properties; workspace search doesn't).

### 4. Rerank (skipped if `--no-rerank`)

Cache lookup first: SHA256(original_query + sorted_candidate_ids).

Cache miss → call Claude Haiku with:
- Original (un-expanded) query
- Numbered list of candidates: `[N] Title — Properties — Excerpt`
- Asks for a JSON array of objects: `{index, score, why}` ordered by score descending

Parse, map back to candidate page objects, take top `--limit`.

Failure modes:
- LLM call error → return raw expanded-search results in candidate order, warn on stderr
- JSON parse error → same as above
- Rerank returns fewer than requested → take what's there, warn

### 5. Output

**Terminal** (default): formatted table with title, relevance, snippet, key properties, URL. ANSI color via `rich` library if available, plain text otherwise.

**JSON** (`--json`): array of objects with stable schema:
```json
[
  {
    "id": "abc-def-...",                    // dashed UUID
    "url": "https://notion.so/...",
    "title": "MCP server architecture...",
    "relevance": 0.94,                       // 0.0-1.0; null if --no-rerank
    "snippet": "Direct match — covers...",   // null if --no-rerank
    "excerpt": "First paragraph text...",    // 200-char from page body
    "properties": {                          // only properties present on the page
      "Status": "Done",
      "Topic": ["AI", "MCP"],
      "Account Code": "D.intelligence"
    }
  }
]
```

The schema is stable: any future tool consuming search output (notion-to-notebooklm push, aggregation, etc.) reads this format.

---

## File structure

| File | Purpose | LOC |
|---|---|---|
| `custom-skills/31-notion-organizer/code/scripts/notion_search.py` | Main CLI + pipeline | ~400 |
| `custom-skills/31-notion-organizer/code/scripts/test_notion_search.py` | Unit tests with mocked Claude/Notion | ~250 |
| `custom-skills/31-notion-organizer/code/scripts/_search_llm.py` | LLM client abstraction (SDK + CLI fallback, prompt construction) | ~150 |
| `custom-skills/31-notion-organizer/commands/notion-search.md` | Slash command definition | ~50 |
| `custom-skills/31-notion-organizer/code/CLAUDE.md` | Add usage section | +60 lines |
| `custom-skills/31-notion-organizer/code/scripts/requirements.txt` | Add `anthropic` (optional, falls back to CLI) | +1 line |

**Total: ~850 LOC + tests + docs.** ~3 days focused work.

The `_notion_compat` helper from Phase 2 is reused (client factory, error explanation, ID resolution). No new compatibility layer needed.

---

## LLM client (`_search_llm.py`)

Abstraction so callers don't care whether SDK or CLI is in use.

```python
def call_claude(prompt: str, model: str = "claude-haiku-4-5", max_tokens: int = 1000) -> str:
    """Return Claude's text response. Raises on failure."""
    if _have_anthropic_sdk() and os.getenv("ANTHROPIC_API_KEY"):
        return _call_via_sdk(prompt, model, max_tokens)
    if _have_claude_cli():
        return _call_via_cli(prompt, model)
    raise RuntimeError(
        "No Claude client available. Install `anthropic` and set "
        "ANTHROPIC_API_KEY, or install Claude Code CLI."
    )
```

Two thin implementations behind it:
- `_call_via_sdk`: standard `anthropic.Anthropic().messages.create(...)`
- `_call_via_cli`: `subprocess.run(["claude", "-p", prompt, "--model", model], ...)`, capture stdout

Both return the assistant's text content as a single string. Callers (`expand_query`, `rerank_candidates`) parse JSON out of it.

---

## Caching

**Key:** SHA256 hex digest of `f"{query}|{','.join(sorted(candidate_ids))}"`.

**Storage:** `~/.cache/notion-search/<hash>.json`. Each file contains:
```json
{
  "query": "AI agents",
  "candidate_ids": ["abc...", "def..."],
  "results": [...],         // the reranked output
  "cached_at": "2026-04-28T14:23:00Z"
}
```

**Lookup:** if file exists and `now - cached_at < TTL` (default 24h), return cached results. Else run rerank and write fresh.

**Invalidation:** TTL-only for simplicity. `--no-cache` bypasses both read and write.

**Why not invalidate on Notion content changes?** That requires tracking last-edited timestamps and computing a content hash — significant complexity for marginal benefit. With a 1-day TTL, stale results are bounded; user can `--no-cache` for fresh runs.

---

## Error handling

| Failure | Behavior |
|---|---|
| Notion API: `ObjectNotFound` on `--databases` ID | Error out with friendly message via `_notion_compat.explain_api_error` |
| Notion API: `Unauthorized` | Same |
| Query expansion fails (LLM error or JSON parse) | Use original query only, warn on stderr, continue |
| Rerank fails (LLM error or JSON parse) | Return raw expanded-search results in candidate order, warn on stderr, continue |
| Cache file corrupted | Delete cache file, treat as miss |
| Empty Notion results | Print "No matches for '<query>'" and exit 0 (not an error) |
| LLM client unavailable (no SDK + no CLI) | Error out with setup instructions if `--no-rerank` and `--no-expand` aren't both set; otherwise proceed |
| `--filter` JSON parse error | Error out before any API calls |

---

## Tests

10 tests covering the public surface, all using mocked Claude and Notion clients (no real API calls in tests):

| Test | Verifies |
|---|---|
| `test_expand_query_returns_variants` | LLM mock returns variant list including original; expansion produces 3-5 unique strings |
| `test_search_unions_and_dedupes` | Two variants returning overlapping pages produce deduped candidate set |
| `test_rerank_orders_by_relevance` | Mock rerank returns scores; output sorted score-descending |
| `test_no_rerank_returns_raw` | `--no-rerank` skips LLM, returns Notion's native order, no relevance scores |
| `test_no_expand_uses_single_query` | `--no-expand` calls Notion once with original query, no LLM expansion |
| `test_json_output_parseable` | `--json` emits valid JSON conforming to the schema in Section 5 |
| `test_cache_hit_on_repeat` | Second identical query skips LLM; cache file exists |
| `test_cache_miss_on_different_candidates` | Different candidate set hashes different → fresh rerank |
| `test_property_filter_applied` | `--filter` JSON passed to data_sources.query verbatim |
| `test_database_scope_constrains` | `--databases` causes per-DB queries instead of workspace search |

Tests live in `test_notion_search.py`. Run via `python3 test_notion_search.py` (matching the existing test_parser.py convention in 32).

---

## Out-of-scope follow-ups

- **3b-ii — `notion-to-notebooklm` push skill**: consumes search-output JSON, pushes pages as NotebookLM sources. Separate spec after this ships.
- **Block-level granularity**: surface matching blocks within pages. Adds chunking logic and richer rerank prompts. Defer.
- **Aggregation skill**: "find pages on X, write a summary page with citations." Builds on this. Future Phase 3 sub-project.
- **Multi-workspace search**: requires per-workspace integrations and credential management. Future.

---

## Implementation transition

After user approval, transition to `superpowers:writing-plans` to break this into bite-sized TDD tasks. Likely shape:

1. `_search_llm.py` skeleton (SDK + CLI client abstraction) + unit test
2. Query expansion module + tests
3. Search execution module (workspace + per-DB) + tests
4. Property + excerpt fetch + tests
5. Rerank module (with prompt + JSON parsing) + tests
6. Cache layer + tests
7. CLI assembly (argparse + output formatting) + tests
8. Slash command + 31's CLAUDE.md update + final integration smoke test

Each step independently testable. Total ~3 days estimated, mirrors Phase 3c's pacing.