our-claude-skills/docs/superpowers/specs/2026-06-27-notion-writer-cli-enhancements-design.md

Notion Writer (Skill 32) — CLI Enhancements: File Uploads + Markdown Engine

Date: 2026-06-27 Status: Approved (brainstorming) Scope: Two additive features — (Tier 1) local file/image upload via the ntn CLI, and (Tier 2) an opt-in "markdown engine" that writes through Notion's native enhanced-markdown endpoints. Predecessor: 2026-04-27 Extended Block Coverage (callouts/toggles/columns/mentions), CLAUDE.md v1.2.0 Target version: bump CLAUDE.md footer to v1.3.0

---

Goal

Add two capabilities to custom-skills/32-notion-writer/code/scripts/notion_writer.py:

1. Tier 1 — Local media uploads. Today ![alt](path) is effectively broken (the inline link regex leaves a stray ! and renders the image as a text link). Make standalone image references work: local files are uploaded via ntn files create and embedded as Notion image blocks with a file_upload reference; remote URLs become external image blocks. This fills a genuine capability gap — the parser previously had no image support at all.

2. Tier 2 — Markdown engine. Add --engine markdown to write content through Notion's native enhanced-markdown endpoints (POST /v1/pages with markdown, PATCH /v1/pages/:id/markdown) instead of the local block converter. A dialect translator converts the skill's existing authoring syntax (GitHub alerts, Pandoc columns, @[mentions]) into Notion-flavored markdown so a single source document works in both engines.

The default engine stays blocks — every existing call and downstream script behaves identically. Both engines share one media-upload path.

---

Non-goals

• Re-platforming onto ntn (Tier 3, rejected). The skill keeps its token-based SDK core; ntn is used only as a file-upload subprocess.
• Reading pages back as markdown (GET /v1/pages/:id/markdown). Not needed for a writer; parked.
• Workspace-mismatch hard guard. User chose plain shell-out; preflight surfaces the ntn target workspace as an informational note but does not block on mismatch.
• Generic local attachments (<file>, <pdf>, <video>, <audio>) in v1. The upload mechanism is media-type agnostic, but v1 wires only ![]() images. Other media use the same walk later — parked.
• Legacy markdown update commands (insert_content for arbitrary positions, replace_content_range). Only insert_content{position:end} (append) and replace_content (replace) are used.
• Bidirectional dialect translation (Notion enhanced markdown → skill dialect). One direction only.

---

Architectural decisions (locked during brainstorming)

Decision	Choice	Rationale
File-upload transport	Shell out to ntn files create --plain	User choice. The CLI does the full upload lifecycle in one command.
Upload-failure visibility	Loud, not silent — preflight checks ntn is installed + logged in and prints the target workspace; upload errors abort with the file path + ntn stderr	File uploads are workspace-scoped; a mismatch must surface, not silently misfire.
Markdown dialect handling	Translate skill dialect → Notion enhanced markdown	User choice — one authoring document works in both engines.
Engine selection	--engine {blocks,markdown}, default blocks	Zero regression for existing callers; opt-in to the new path.
Local-media trigger	Auto-detect local paths in ![](); remote URLs left external	User choice — most ergonomic; replaces the silent-strip behavior.
Parser purity	Preserve it — uploads happen in a post-parse block walk, never inside the parser	The 2026-04-27 design locked "parser makes no API calls"; honored by isolating impurity.
Markdown transport	client.request(...) low-level calls, not typed SDK methods	The markdown endpoints have no stable typed SDK methods; request is version-safe and avoids SDK coupling.
Notion API version	2026-03-11 for the markdown engine; blocks engine keeps the SDK default (2025-09-03)	Markdown endpoints require the newer version; blocks path stays untouched.
Python env	Create venv + requirements.txt at the documented path	The documented venv doesn't exist; the skill can't currently run.

---

Module structure

Two new modules keep notion_writer.py from bloating; each has one purpose and is testable in isolation.

File	Status	Role
scripts/notion_writer.py	modified	CLI surface, engine routing, post-parse media materialization, two-phase markdown writes
scripts/_notion_compat.py	modified	Version-aware make_client; markdown-endpoint helpers (create_page_markdown, append_markdown, replace_markdown)
scripts/ntn_files.py	new	ntn files create subprocess wrapper; preflight; upload(path) -> file_upload_id
scripts/md_translate.py	new	Pure functions: skill dialect → Notion enhanced markdown
scripts/requirements.txt	modified	Pin notion-client, python-dotenv
scripts/venv/	new	Created at documented path; not committed (gitignored)

Dependencies between units:

• notion_writer depends on _notion_compat, ntn_files, md_translate.
• md_translate and the parser are pure (deterministic, no I/O).
• ntn_files owns all subprocess I/O.
• _notion_compat owns all Notion HTTP I/O.

---

Tier 1 — Local media uploads (blocks engine)

Parsing (pure)

Add an image detector to _parse_lines, ordered with the other block detectors. A standalone image line matches:

IMAGE_RE = re.compile(r'^\s*!\[([^\]]*)\]\(([^)\s]+)\)\s*$')

create_image_block(alt, target) always emits the external shape regardless of whether target is local or remote:

{"object":"block","type":"image",
 "image":{"type":"external","external":{"url": target},
          "caption": parse_rich_text(alt)}}

The parser stays pure and upload-unaware. Inline images inside a paragraph are out of scope (rare); only standalone image lines become image blocks.

Materialization (impure, isolated)

After parsing and before sending, notion_writer runs a recursive block walk:

materialize_local_media(blocks, base_dir, uploader) -> blocks

• Walks blocks and every nested children list (so images inside toggles/columns are covered).
• For each image block whose external.url is not http(s): and resolves to an existing file under base_dir:
  • file_upload_id = uploader.upload(resolved_path)
  • rewrite to {"image":{"type":"file_upload","file_upload":{"id": file_upload_id}, "caption": ...}}
• Remote/external images are left unchanged.
• A local path that does not exist on disk → abort with a clear error (image references missing file: <path>), since the user clearly intended a local embed.

base_dir = the --file argument's parent directory; for --stdin, the current working directory.

Upload wrapper — ntn_files.py

def preflight() -> WorkspaceInfo        # cached; once per process
def upload(path: Path) -> str           # returns file_upload_id

• preflight():
  • shutil.which("ntn") → if missing, raise with install hint (curl -fsSL https://ntn.dev | bash).
  • ntn api v1/users/me (JSON) → parse workspace_name / workspace_id; print one informational line: ntn → workspace "<name>". Does not block on mismatch (per locked decision). If the call fails (not logged in), raise with ntn login hint.
• upload(path):
  • subprocess.run(["ntn","files","create","--plain"], stdin=open(path,"rb"), capture_output=True, text=True)
  • on non-zero exit → raise NtnUploadError(path, stderr).
  • parse stdout: the upload ID is the first tab-separated field of the first line.
  • returns the ID. (Notion requires attaching within ~1 hour; the immediate block append in the same run is well inside that window.)

---

Tier 2 — Markdown engine

Dialect translator — md_translate.py

translate(content: str) -> str converts the skill dialect to Notion enhanced markdown. Only three constructs differ; everything else passes through verbatim.

Skill dialect	→ Notion enhanced markdown
> [!NOTE] + contiguous > body	<callout icon="ℹ️" color="blue_bg">\n\t<body>\n</callout>
::: columns / ::: column / :::	<columns>\n\t<column>…</column>\n\t<column>…</column>\n</columns>
<details> / <summary> toggle	<details>\n<summary>…</summary>\n\t<children>\n</details> — children re-indented one tab
@[Title](id|url)	<mention-page url="<resolved>">Title</mention-page>
headings, lists, to-dos, quotes, fenced code, ---, bold/italic/code/~~strike~~/links, pipe tables	pass through unchanged

Notes:

• Callout icon/color reuse the existing ALERT_TYPES map; Notion color names use the _bg suffix (blue_bg), not the block-API blue_background. A small adapter maps one to the other.
• Children inside <callout>, <columns>, and <details> are tab-indented one level (enhanced-markdown requirement — toggle and callout children "must be indented"). The translator emits tabs; this is why <details> is translated rather than passed through verbatim.
• Pipe tables pass through — Notion's own enhanced-markdown example renders a standard pipe table, so no <table> rewrite is needed in v1. (If live testing shows pipe tables are rejected, the fallback is a <table> rewrite — noted as the one open verification item.)
• Mention resolution reuses extract_notion_id; an unresolvable target degrades to plain @Title text (same posture as the blocks engine).
• The translator is line-oriented and reentrant for the two container constructs, mirroring _parse_lines.

Transport helpers — _notion_compat.py

All via client.request(...) so no SDK typed-method dependency:

def make_client(api_key, notion_version=None) -> Client   # version override added
def create_page_markdown(client, parent, properties, markdown) -> dict
    # POST v1/pages  body={parent, properties, markdown}
def append_markdown(client, page_id, markdown) -> dict
    # PATCH v1/pages/{id}/markdown
    # body={"type":"insert_content","insert_content":{"content":md,"position":{"type":"end"}}}
def replace_markdown(client, page_id, markdown, allow_deleting=False) -> dict
    # PATCH v1/pages/{id}/markdown
    # body={"type":"replace_content","replace_content":{"new_str":md,"allow_deleting_content":allow_deleting}}

Two-client split for version safety. The markdown write helpers use a client built with notion_version="2026-03-11". All schema/property/upsert-lookup operations (resolve_data_source_id, get_schema, coerce_properties, find_existing_page) continue on the default-version client, so the 2025-09-03 data-source behavior the skill already relies on is unchanged. Only the three markdown write calls cross to the newer version. The data-source parent ({type:data_source_id,...}) is expected to remain valid under 2026-03-11 (confirmed in the live smoke test).

Routing by operation

CLI op	blocks engine (default)	markdown engine
Create DB row (-d -t)	pages.create(parent, properties, children=blocks)	create_page_markdown(parent, properties, translate(content))
Append to page (-p)	blocks.children.append	append_markdown(page_id, translate(content))
Replace page (-p -r)	delete-all-blocks + append	replace_markdown(page_id, translate(content))
Upsert (--upsert-by)	unchanged (property update + body)	property update via SDK + body via replace_markdown

The markdown replace_content path replaces the brittle paginated delete-every-block logic for that engine. --upsert-by lookup/property-coercion logic is shared unchanged; only the body write differs.

Markdown engine + local images (two-phase write)

The markdown endpoints take a URL for ![](url), and a file_upload id is not a URL, so local images cannot be inlined into the markdown string. Handling, reusing the Tier 1 upload + append helpers:

1. Split standalone local image refs out of the content (same IMAGE_RE, local targets only; remote ![](http…) stay inline in the markdown); translate the remainder; write it (create / append / replace).
2. Upload the local images and append them as image blocks to the resulting page via blocks.children.append.

Known limitation (documented): in the markdown engine, local images land at the end of the page, not their original position. When image position matters, use the blocks engine. Remote images keep their position (they stay inline in the markdown).

---

Flags & versioning

New CLI flags:

Flag	Default	Meaning
--engine {blocks,markdown}	blocks	Select the write path
--notion-version VERSION	engine-dependent	Override the API version (markdown defaults to 2026-03-11)
--allow-deleting-content	off	Permit markdown replace_content to delete child pages/databases

Validation: --engine markdown with --upsert-by on a property the markdown path can't update falls back to the shared coercion logic (no new restriction). --allow-deleting-content is ignored by the blocks engine (warn if combined).

---

Error handling

Permissive where the user's intent is ambiguous; loud where they clearly intended an action that failed.

Failure	Behavior
ntn not installed	Abort before any write; install hint
ntn not logged in (users/me fails)	Abort; ntn login hint
ntn files create non-zero exit	Abort; print file path + ntn stderr
Local image path missing on disk	Abort; image references missing file: <path>
Markdown replace/selection no match	Route validation_error through explain_api_error with a markdown-specific hint
replace_content would delete child pages	Surface the API's affected-items list; suggest --allow-deleting-content
Synced-page update rejected	Clear message (synced pages can't be updated)
Unresolvable @[mention]	Degrade to plain @Title (both engines)
Two-phase image append fails after text write	Warn that text was written but images were not; non-zero exit

explain_api_error gains markdown-endpoint cases; all other paths reuse existing handling.

---

Testing

TDD. New unit suites alongside the existing test_parser.py (28 tests, must stay green).

Suite	Verifies
test_parser.py (extended)	 → external image block;  → external-shape with local URL pre-materialization; non-image ! text unaffected; existing 28 stay green
test_md_translate.py (new)	callout (each alert type → icon+_bg color, tab-indented body); columns → <columns>/<column>; mention id+url+invalid; passthrough of headings/lists/code/tables/<details>; escaping
test_ntn_files.py (new)	subprocess.run mocked: command shape, --plain first-field parse, non-zero → NtnUploadError, preflight missing-ntn, preflight not-logged-in; local-vs-remote branch in materialize_local_media
test_engine_routing.py (new)	client request mocked: each op×engine calls the right path/method/body; markdown client built with 2026-03-11; two-phase image append fires after text write

Live smoke test (manual, end of implementation) against the "Working with AI" data source f8f19ede-32bd-43ac-9f60-0651f6f40afe, both engines:

1. --test connection.
2. Blocks engine: create a row with a local image → confirm file_upload image block renders.
3. Markdown engine: create a row from a doc using [!NOTE] + :::columns → confirm callout + columns render; confirm a local image appended at end.
4. Markdown engine --replace on the same page → confirm replace_content works.

This verifies the one open item (pipe-table passthrough) and the data-source parent under 2026-03-11.

---

File changes

File	Change
scripts/notion_writer.py	--engine/--notion-version/--allow-deleting-content flags; image detector + create_image_block; materialize_local_media walk; markdown-engine routing; two-phase image append
scripts/_notion_compat.py	make_client(notion_version=…); create_page_markdown/append_markdown/replace_markdown; markdown cases in explain_api_error
scripts/ntn_files.py	new — preflight + upload
scripts/md_translate.py	new — translate + construct translators
scripts/test_md_translate.py, scripts/test_ntn_files.py, scripts/test_engine_routing.py	new test suites
scripts/test_parser.py	image tests
scripts/requirements.txt	pin deps
code/CLAUDE.md, SKILL.md	document engines, file uploads, flags, the markdown-engine image limitation; v1.3.0 changelog
.gitignore	ensure venv/ ignored

---

Out of scope (parking lot)

• Generic local <file>/<pdf>/<video>/<audio> upload (mechanism ready; not wired in v1).
• GET /v1/pages/:id/markdown round-trip read.
• Workspace-mismatch hard guard (escalate preflight from warn → block) if it proves necessary in practice.
• Notion enhanced markdown → skill dialect (reverse translation), which would also serve the parked Phase 3b Notion-as-RAG export.
• <table> rewrite for pipe tables — only if live testing shows pipe tables are rejected by the markdown endpoint.
• Inline (non-standalone) images.

---

Implementation transition

After user approval of this spec, invoke superpowers:writing-plans. The plan will likely sequence:

1. Create venv + requirements.txt; confirm existing 28 tests run green (baseline).
2. ntn_files.py + test_ntn_files.py (subprocess mocked).
3. Image detector + create_image_block + materialize_local_media + parser image tests (blocks engine Tier 1 complete; live single-image check).
4. md_translate.py + test_md_translate.py (pure, no API).
5. _notion_compat markdown helpers + version-aware client + test_engine_routing.py.
6. Wire --engine/flags + routing + two-phase image append in notion_writer.py.
7. Docs (CLAUDE.md/SKILL.md) + v1.3.0 bump.
8. Live smoke test (both engines) against the "Working with AI" DB; resolve the pipe-table verification item.

Each step is independently testable and revertable.