From 7f20c08edf67095986be0627c0f8cfe2815d8e62 Mon Sep 17 00:00:00 2001
From: Andrew Yim <andrew.yim@dintelligence.co.kr>
Date: Sat, 27 Jun 2026 07:39:36 +0900
Subject: [PATCH] docs(notion-writer): design spec for CLI file uploads +
 markdown engine
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Tier 1: local ![](path) images uploaded via `ntn files create` and
embedded as file_upload image blocks (parser stays pure; upload happens
in an isolated post-parse walk).

Tier 2: opt-in `--engine markdown` writes through Notion's native
enhanced-markdown endpoints (POST /pages, PATCH .../markdown) with a
dialect translator (callouts/columns/toggles/mentions) so one source
doc works in both engines. Default engine stays `blocks` (no regression).

Adds ntn_files.py + md_translate.py, version-aware client (2026-03-11
for markdown writes only), new test suites, venv setup, docs → v1.3.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 ...7-notion-writer-cli-enhancements-design.md | 280 ++++++++++++++++++
 1 file changed, 280 insertions(+)
 create mode 100644 docs/superpowers/specs/2026-06-27-notion-writer-cli-enhancements-design.md

diff --git a/docs/superpowers/specs/2026-06-27-notion-writer-cli-enhancements-design.md b/docs/superpowers/specs/2026-06-27-notion-writer-cli-enhancements-design.md
new file mode 100644
index 0000000..f0dc6d0
--- /dev/null
+++ b/docs/superpowers/specs/2026-06-27-notion-writer-cli-enhancements-design.md
@@ -0,0 +1,280 @@
+# Notion Writer (Skill 32) — CLI Enhancements: File Uploads + Markdown Engine
+
+> **Date**: 2026-06-27
+> **Status**: Approved (brainstorming)
+> **Scope**: Two additive features — (Tier 1) local file/image upload via the `ntn` CLI, and (Tier 2) an opt-in "markdown engine" that writes through Notion's native enhanced-markdown endpoints.
+> **Predecessor**: 2026-04-27 Extended Block Coverage (callouts/toggles/columns/mentions), CLAUDE.md v1.2.0
+> **Target version**: bump CLAUDE.md footer to v1.3.0
+
+---
+
+## Goal
+
+Add two capabilities to `custom-skills/32-notion-writer/code/scripts/notion_writer.py`:
+
+1. **Tier 1 — Local media uploads.** Today `![alt](path)` is effectively broken (the inline link regex leaves a stray `!` and renders the image as a text link). Make standalone image references work: local files are uploaded via `ntn files create` and embedded as Notion `image` blocks with a `file_upload` reference; remote URLs become `external` image blocks. This fills a genuine capability gap — the parser previously had no image support at all.
+
+2. **Tier 2 — Markdown engine.** Add `--engine markdown` to write content through Notion's native enhanced-markdown endpoints (`POST /v1/pages` with `markdown`, `PATCH /v1/pages/:id/markdown`) instead of the local block converter. A dialect translator converts the skill's existing authoring syntax (GitHub alerts, Pandoc columns, `@[mentions]`) into Notion-flavored markdown so a single source document works in both engines.
+
+The default engine stays `blocks` — every existing call and downstream script behaves identically. Both engines share one media-upload path.
+
+---
+
+## Non-goals
+
+- **Re-platforming onto `ntn`** (Tier 3, rejected). The skill keeps its token-based SDK core; `ntn` is used only as a file-upload subprocess.
+- **Reading pages back as markdown** (`GET /v1/pages/:id/markdown`). Not needed for a writer; parked.
+- **Workspace-mismatch hard guard.** User chose plain shell-out; preflight surfaces the `ntn` target workspace as an informational note but does not block on mismatch.
+- **Generic local attachments** (`<file>`, `<pdf>`, `<video>`, `<audio>`) in v1. The upload mechanism is media-type agnostic, but v1 wires only `![]()` images. Other media use the same walk later — parked.
+- **Legacy markdown update commands** (`insert_content` for arbitrary positions, `replace_content_range`). Only `insert_content{position:end}` (append) and `replace_content` (replace) are used.
+- **Bidirectional dialect translation** (Notion enhanced markdown → skill dialect). One direction only.
+
+---
+
+## Architectural decisions (locked during brainstorming)
+
+| Decision | Choice | Rationale |
+|---|---|---|
+| File-upload transport | **Shell out to `ntn files create --plain`** | User choice. The CLI does the full upload lifecycle in one command. |
+| Upload-failure visibility | **Loud, not silent** — preflight checks `ntn` is installed + logged in and prints the target workspace; upload errors abort with the file path + `ntn` stderr | File uploads are workspace-scoped; a mismatch must surface, not silently misfire. |
+| Markdown dialect handling | **Translate** skill dialect → Notion enhanced markdown | User choice — one authoring document works in both engines. |
+| Engine selection | **`--engine {blocks,markdown}`, default `blocks`** | Zero regression for existing callers; opt-in to the new path. |
+| Local-media trigger | **Auto-detect** local paths in `![]()`; remote URLs left external | User choice — most ergonomic; replaces the silent-strip behavior. |
+| Parser purity | **Preserve it** — uploads happen in a post-parse block walk, never inside the parser | The 2026-04-27 design locked "parser makes no API calls"; honored by isolating impurity. |
+| Markdown transport | **`client.request(...)` low-level calls**, not typed SDK methods | The markdown endpoints have no stable typed SDK methods; `request` is version-safe and avoids SDK coupling. |
+| Notion API version | **`2026-03-11` for the markdown engine**; blocks engine keeps the SDK default (`2025-09-03`) | Markdown endpoints require the newer version; blocks path stays untouched. |
+| Python env | **Create venv + `requirements.txt`** at the documented path | The documented venv doesn't exist; the skill can't currently run. |
+
+---
+
+## Module structure
+
+Two new modules keep `notion_writer.py` from bloating; each has one purpose and is testable in isolation.
+
+| File | Status | Role |
+|---|---|---|
+| `scripts/notion_writer.py` | modified | CLI surface, engine routing, post-parse media materialization, two-phase markdown writes |
+| `scripts/_notion_compat.py` | modified | Version-aware `make_client`; markdown-endpoint helpers (`create_page_markdown`, `append_markdown`, `replace_markdown`) |
+| `scripts/ntn_files.py` | **new** | `ntn files create` subprocess wrapper; preflight; `upload(path) -> file_upload_id` |
+| `scripts/md_translate.py` | **new** | Pure functions: skill dialect → Notion enhanced markdown |
+| `scripts/requirements.txt` | modified | Pin `notion-client`, `python-dotenv` |
+| `scripts/venv/` | **new** | Created at documented path; not committed (gitignored) |
+
+Dependencies between units:
+- `notion_writer` depends on `_notion_compat`, `ntn_files`, `md_translate`.
+- `md_translate` and the parser are pure (deterministic, no I/O).
+- `ntn_files` owns all subprocess I/O.
+- `_notion_compat` owns all Notion HTTP I/O.
+
+---
+
+## Tier 1 — Local media uploads (blocks engine)
+
+### Parsing (pure)
+Add an image detector to `_parse_lines`, ordered with the other block detectors. A **standalone** image line matches:
+
+```python
+IMAGE_RE = re.compile(r'^\s*!\[([^\]]*)\]\(([^)\s]+)\)\s*$')
+```
+
+`create_image_block(alt, target)` always emits the **external shape** regardless of whether `target` is local or remote:
+
+```python
+{"object":"block","type":"image",
+ "image":{"type":"external","external":{"url": target},
+          "caption": parse_rich_text(alt)}}
+```
+
+The parser stays pure and upload-unaware. Inline images inside a paragraph are out of scope (rare); only standalone image lines become image blocks.
+
+### Materialization (impure, isolated)
+After parsing and before sending, `notion_writer` runs a recursive block walk:
+
+```python
+materialize_local_media(blocks, base_dir, uploader) -> blocks
+```
+
+- Walks `blocks` and every nested `children` list (so images inside toggles/columns are covered).
+- For each `image` block whose `external.url` is **not** `http(s):` and resolves to an existing file under `base_dir`:
+  - `file_upload_id = uploader.upload(resolved_path)`
+  - rewrite to `{"image":{"type":"file_upload","file_upload":{"id": file_upload_id}, "caption": ...}}`
+- Remote/external images are left unchanged.
+- A local path that does not exist on disk → abort with a clear error (`image references missing file: <path>`), since the user clearly intended a local embed.
+
+`base_dir` = the `--file` argument's parent directory; for `--stdin`, the current working directory.
+
+### Upload wrapper — `ntn_files.py`
+
+```python
+def preflight() -> WorkspaceInfo        # cached; once per process
+def upload(path: Path) -> str           # returns file_upload_id
+```
+
+- `preflight()`:
+  - `shutil.which("ntn")` → if missing, raise with install hint (`curl -fsSL https://ntn.dev | bash`).
+  - `ntn api v1/users/me` (JSON) → parse `workspace_name` / `workspace_id`; print one informational line: `ntn → workspace "<name>"`. Does not block on mismatch (per locked decision). If the call fails (not logged in), raise with `ntn login` hint.
+- `upload(path)`:
+  - `subprocess.run(["ntn","files","create","--plain"], stdin=open(path,"rb"), capture_output=True, text=True)`
+  - on non-zero exit → raise `NtnUploadError(path, stderr)`.
+  - parse stdout: the upload ID is the first tab-separated field of the first line.
+  - returns the ID. (Notion requires attaching within ~1 hour; the immediate block append in the same run is well inside that window.)
+
+---
+
+## Tier 2 — Markdown engine
+
+### Dialect translator — `md_translate.py`
+
+`translate(content: str) -> str` converts the skill dialect to Notion enhanced markdown. Only three constructs differ; everything else passes through verbatim.
+
+| Skill dialect | → Notion enhanced markdown |
+|---|---|
+| `> [!NOTE]` + contiguous `>` body | `<callout icon="ℹ️" color="blue_bg">\n\t<body>\n</callout>` |
+| `::: columns` / `::: column` / `:::` | `<columns>\n\t<column>…</column>\n\t<column>…</column>\n</columns>` |
+| `<details>` / `<summary>` toggle | `<details>\n<summary>…</summary>\n\t<children>\n</details>` — children **re-indented one tab** |
+| `@[Title](id\|url)` | `<mention-page url="<resolved>">Title</mention-page>` |
+| headings, lists, to-dos, quotes, fenced code, `---`, **bold**/*italic*/`code`/`~~strike~~`/links, pipe tables | **pass through unchanged** |
+
+Notes:
+- Callout icon/color reuse the existing `ALERT_TYPES` map; Notion color names use the `_bg` suffix (`blue_bg`), not the block-API `blue_background`. A small adapter maps one to the other.
+- Children inside `<callout>`, `<columns>`, and `<details>` are **tab-indented** one level (enhanced-markdown requirement — toggle and callout children "must be indented"). The translator emits tabs; this is why `<details>` is translated rather than passed through verbatim.
+- Pipe tables pass through — Notion's own enhanced-markdown example renders a standard pipe table, so no `<table>` rewrite is needed in v1. (If live testing shows pipe tables are rejected, the fallback is a `<table>` rewrite — noted as the one open verification item.)
+- Mention resolution reuses `extract_notion_id`; an unresolvable target degrades to plain `@Title` text (same posture as the blocks engine).
+- The translator is line-oriented and reentrant for the two container constructs, mirroring `_parse_lines`.
+
+### Transport helpers — `_notion_compat.py`
+
+All via `client.request(...)` so no SDK typed-method dependency:
+
+```python
+def make_client(api_key, notion_version=None) -> Client   # version override added
+def create_page_markdown(client, parent, properties, markdown) -> dict
+    # POST v1/pages  body={parent, properties, markdown}
+def append_markdown(client, page_id, markdown) -> dict
+    # PATCH v1/pages/{id}/markdown
+    # body={"type":"insert_content","insert_content":{"content":md,"position":{"type":"end"}}}
+def replace_markdown(client, page_id, markdown, allow_deleting=False) -> dict
+    # PATCH v1/pages/{id}/markdown
+    # body={"type":"replace_content","replace_content":{"new_str":md,"allow_deleting_content":allow_deleting}}
+```
+
+**Two-client split for version safety.** The markdown *write* helpers use a client built with `notion_version="2026-03-11"`. All schema/property/upsert-lookup operations (`resolve_data_source_id`, `get_schema`, `coerce_properties`, `find_existing_page`) continue on the **default-version** client, so the 2025-09-03 data-source behavior the skill already relies on is unchanged. Only the three markdown write calls cross to the newer version. The data-source parent (`{type:data_source_id,...}`) is expected to remain valid under `2026-03-11` (confirmed in the live smoke test).
+
+### Routing by operation
+
+| CLI op | blocks engine (default) | markdown engine |
+|---|---|---|
+| Create DB row (`-d -t`) | `pages.create(parent, properties, children=blocks)` | `create_page_markdown(parent, properties, translate(content))` |
+| Append to page (`-p`) | `blocks.children.append` | `append_markdown(page_id, translate(content))` |
+| Replace page (`-p -r`) | delete-all-blocks + append | `replace_markdown(page_id, translate(content))` |
+| Upsert (`--upsert-by`) | unchanged (property update + body) | property update via SDK + body via `replace_markdown` |
+
+The markdown `replace_content` path replaces the brittle paginated delete-every-block logic for that engine. `--upsert-by` lookup/property-coercion logic is shared unchanged; only the body write differs.
+
+### Markdown engine + local images (two-phase write)
+
+The markdown endpoints take a URL for `![](url)`, and a `file_upload` id is not a URL, so local images cannot be inlined into the markdown string. Handling, reusing the Tier 1 upload + append helpers:
+
+1. Split standalone **local** image refs out of the content (same `IMAGE_RE`, local targets only; remote `![](http…)` stay inline in the markdown); translate the remainder; write it (create / append / replace).
+2. Upload the local images and append them as `image` blocks to the resulting page via `blocks.children.append`.
+
+**Known limitation (documented):** in the markdown engine, local images land at the **end** of the page, not their original position. When image position matters, use the blocks engine. Remote images keep their position (they stay inline in the markdown).
+
+---
+
+## Flags & versioning
+
+New CLI flags:
+
+| Flag | Default | Meaning |
+|---|---|---|
+| `--engine {blocks,markdown}` | `blocks` | Select the write path |
+| `--notion-version VERSION` | engine-dependent | Override the API version (markdown defaults to `2026-03-11`) |
+| `--allow-deleting-content` | off | Permit markdown `replace_content` to delete child pages/databases |
+
+Validation: `--engine markdown` with `--upsert-by` on a property the markdown path can't update falls back to the shared coercion logic (no new restriction). `--allow-deleting-content` is ignored by the blocks engine (warn if combined).
+
+---
+
+## Error handling
+
+Permissive where the user's intent is ambiguous; loud where they clearly intended an action that failed.
+
+| Failure | Behavior |
+|---|---|
+| `ntn` not installed | Abort before any write; install hint |
+| `ntn` not logged in (`users/me` fails) | Abort; `ntn login` hint |
+| `ntn files create` non-zero exit | Abort; print file path + `ntn` stderr |
+| Local image path missing on disk | Abort; `image references missing file: <path>` |
+| Markdown `replace`/selection no match | Route `validation_error` through `explain_api_error` with a markdown-specific hint |
+| `replace_content` would delete child pages | Surface the API's affected-items list; suggest `--allow-deleting-content` |
+| Synced-page update rejected | Clear message (synced pages can't be updated) |
+| Unresolvable `@[mention]` | Degrade to plain `@Title` (both engines) |
+| Two-phase image append fails after text write | Warn that text was written but images were not; non-zero exit |
+
+`explain_api_error` gains markdown-endpoint cases; all other paths reuse existing handling.
+
+---
+
+## Testing
+
+TDD. New unit suites alongside the existing `test_parser.py` (28 tests, must stay green).
+
+| Suite | Verifies |
+|---|---|
+| `test_parser.py` (extended) | `![alt](url)` → external image block; `![alt](./local.png)` → external-shape with local URL pre-materialization; non-image `!` text unaffected; existing 28 stay green |
+| `test_md_translate.py` (new) | callout (each alert type → icon+`_bg` color, tab-indented body); columns → `<columns>/<column>`; mention id+url+invalid; passthrough of headings/lists/code/tables/`<details>`; escaping |
+| `test_ntn_files.py` (new) | `subprocess.run` mocked: command shape, `--plain` first-field parse, non-zero → `NtnUploadError`, preflight missing-`ntn`, preflight not-logged-in; local-vs-remote branch in `materialize_local_media` |
+| `test_engine_routing.py` (new) | client `request` mocked: each op×engine calls the right path/method/body; markdown client built with `2026-03-11`; two-phase image append fires after text write |
+
+**Live smoke test** (manual, end of implementation) against the "Working with AI" data source `f8f19ede-32bd-43ac-9f60-0651f6f40afe`, both engines:
+1. `--test` connection.
+2. Blocks engine: create a row with a local image → confirm `file_upload` image block renders.
+3. Markdown engine: create a row from a doc using `[!NOTE]` + `:::columns` → confirm callout + columns render; confirm a local image appended at end.
+4. Markdown engine `--replace` on the same page → confirm `replace_content` works.
+
+This verifies the one open item (pipe-table passthrough) and the data-source parent under `2026-03-11`.
+
+---
+
+## File changes
+
+| File | Change |
+|---|---|
+| `scripts/notion_writer.py` | `--engine`/`--notion-version`/`--allow-deleting-content` flags; image detector + `create_image_block`; `materialize_local_media` walk; markdown-engine routing; two-phase image append |
+| `scripts/_notion_compat.py` | `make_client(notion_version=…)`; `create_page_markdown`/`append_markdown`/`replace_markdown`; markdown cases in `explain_api_error` |
+| `scripts/ntn_files.py` | **new** — preflight + `upload` |
+| `scripts/md_translate.py` | **new** — `translate` + construct translators |
+| `scripts/test_md_translate.py`, `scripts/test_ntn_files.py`, `scripts/test_engine_routing.py` | **new** test suites |
+| `scripts/test_parser.py` | image tests |
+| `scripts/requirements.txt` | pin deps |
+| `code/CLAUDE.md`, `SKILL.md` | document engines, file uploads, flags, the markdown-engine image limitation; v1.3.0 changelog |
+| `.gitignore` | ensure `venv/` ignored |
+
+---
+
+## Out of scope (parking lot)
+
+- Generic local `<file>`/`<pdf>`/`<video>`/`<audio>` upload (mechanism ready; not wired in v1).
+- `GET /v1/pages/:id/markdown` round-trip read.
+- Workspace-mismatch hard guard (escalate preflight from warn → block) if it proves necessary in practice.
+- Notion enhanced markdown → skill dialect (reverse translation), which would also serve the parked Phase 3b Notion-as-RAG export.
+- `<table>` rewrite for pipe tables — only if live testing shows pipe tables are rejected by the markdown endpoint.
+- Inline (non-standalone) images.
+
+---
+
+## Implementation transition
+
+After user approval of this spec, invoke `superpowers:writing-plans`. The plan will likely sequence:
+
+1. Create venv + `requirements.txt`; confirm existing 28 tests run green (baseline).
+2. `ntn_files.py` + `test_ntn_files.py` (subprocess mocked).
+3. Image detector + `create_image_block` + `materialize_local_media` + parser image tests (blocks engine Tier 1 complete; live single-image check).
+4. `md_translate.py` + `test_md_translate.py` (pure, no API).
+5. `_notion_compat` markdown helpers + version-aware client + `test_engine_routing.py`.
+6. Wire `--engine`/flags + routing + two-phase image append in `notion_writer.py`.
+7. Docs (CLAUDE.md/SKILL.md) + v1.3.0 bump.
+8. Live smoke test (both engines) against the "Working with AI" DB; resolve the pipe-table verification item.
+
+Each step is independently testable and revertable.