Compare commits
16 Commits
main
...
50c6741c76
| Author | SHA1 | Date | |
|---|---|---|---|
| 50c6741c76 | |||
| 759bb20911 | |||
| cf7e649284 | |||
| 5e898f81e7 | |||
| b14701d92c | |||
| aeba10cea4 | |||
| c5a6c3cda0 | |||
| 4bb2b2d918 | |||
| 2ee5809f57 | |||
| 80c9efa282 | |||
| 0a44526761 | |||
| 4426920037 | |||
| 6ad857ae20 | |||
| 3214cdde59 | |||
| 7f20c08edf | |||
| 43824075dd |
162
.claude-plugin/marketplace.json
Normal file
162
.claude-plugin/marketplace.json
Normal file
@@ -0,0 +1,162 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/claude-code-marketplace-schema.json",
|
||||
"name": "ourdigital-skills",
|
||||
"owner": {
|
||||
"name": "OurDigital",
|
||||
"email": "andrew.yim@ourdigital.org"
|
||||
},
|
||||
"metadata": {
|
||||
"description": "OurDigital custom Claude skills, grouped into domain plugins (core, SEO, Jamie, D.intelligence, Notion, GTM, NotebookLM, utilities). Enable only the domains you need.",
|
||||
"version": "1.0.0"
|
||||
},
|
||||
"plugins": [
|
||||
{
|
||||
"name": "ourdigital-core",
|
||||
"description": "OurDigital core workflows — brand guide, Korean blog & English journal, research-to-blog, Notion-to-deck, visual design, ad copy, training material, back-office docs, skill creator, estimate engine.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/01-ourdigital-brand-guide",
|
||||
"./custom-skills/02-ourdigital-blog",
|
||||
"./custom-skills/03-ourdigital-journal",
|
||||
"./custom-skills/04-ourdigital-research",
|
||||
"./custom-skills/05-ourdigital-document",
|
||||
"./custom-skills/06-ourdigital-designer",
|
||||
"./custom-skills/07-ourdigital-ad-manager",
|
||||
"./custom-skills/08-ourdigital-trainer",
|
||||
"./custom-skills/09-ourdigital-backoffice",
|
||||
"./custom-skills/10-ourdigital-skill-creator",
|
||||
"./custom-skills/96-ourdigital-estimate-engine"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-seo",
|
||||
"description": "Full SEO suite — technical/on-page audits, Core Web Vitals, Search Console, schema validate/generate, local, keyword, SERP, rank tracking, links, content, e-commerce, KPIs, international, AI visibility, knowledge graph, gateway pages, competitor intel, crawl budget, migration, reporting, presales.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/11-seo-comprehensive-audit",
|
||||
"./custom-skills/12-seo-technical-audit",
|
||||
"./custom-skills/13-seo-on-page-audit",
|
||||
"./custom-skills/14-seo-core-web-vitals",
|
||||
"./custom-skills/15-seo-search-console",
|
||||
"./custom-skills/16-seo-schema-validator",
|
||||
"./custom-skills/17-seo-schema-generator",
|
||||
"./custom-skills/18-seo-local-audit",
|
||||
"./custom-skills/19-seo-keyword-strategy",
|
||||
"./custom-skills/20-seo-serp-analysis",
|
||||
"./custom-skills/21-seo-position-tracking",
|
||||
"./custom-skills/22-seo-link-building",
|
||||
"./custom-skills/23-seo-content-strategy",
|
||||
"./custom-skills/24-seo-ecommerce",
|
||||
"./custom-skills/25-seo-kpi-framework",
|
||||
"./custom-skills/26-seo-international",
|
||||
"./custom-skills/27-seo-ai-visibility",
|
||||
"./custom-skills/28-seo-knowledge-graph",
|
||||
"./custom-skills/29-seo-gateway-architect",
|
||||
"./custom-skills/30-seo-gateway-builder",
|
||||
"./custom-skills/31-seo-competitor-intel",
|
||||
"./custom-skills/32-seo-crawl-budget",
|
||||
"./custom-skills/33-seo-migration-planner",
|
||||
"./custom-skills/34-seo-reporting-dashboard",
|
||||
"./custom-skills/95-ourdigital-presales-seo"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-notion",
|
||||
"description": "Notion workspace tools — organize/clean up a workspace and write/export content into Notion.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/31-notion-organizer",
|
||||
"./custom-skills/32-notion-writer"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-jamie",
|
||||
"description": "Jamie Plastic Surgery Clinic brand suite — branded content, brand audit, KakaoTalk Kanana FAQ, YouTube SEO + subtitle QA, Instagram management, journal editor, multi-channel marketing.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/40-jamie-brand-editor",
|
||||
"./custom-skills/41-jamie-brand-audit",
|
||||
"./custom-skills/42-jamie-faq-entry",
|
||||
"./custom-skills/43-jamie-youtube-manager",
|
||||
"./custom-skills/44-jamie-youtube-subtitle-checker",
|
||||
"./custom-skills/45-jamie-instagram-manager",
|
||||
"./custom-skills/46-jamie-journal-editor",
|
||||
"./custom-skills/47-jamie-marketing-editor"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-notebooklm",
|
||||
"description": "NotebookLM automation — Q&A with citations, notebook/source/artifact management, studio content generation (podcasts, videos, quizzes), and research/source discovery. Requires the notebooklm-py CLI.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/50-notebooklm-agent",
|
||||
"./custom-skills/51-notebooklm-automation",
|
||||
"./custom-skills/52-notebooklm-studio",
|
||||
"./custom-skills/53-notebooklm-research"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-gtm",
|
||||
"description": "Google Tag Manager tooling — container audit/gap analysis, tag/trigger/variable editor (API + ES5 Custom HTML + dataLayer), and QA/validation.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/60-gtm-audit",
|
||||
"./custom-skills/61-gtm-editor",
|
||||
"./custom-skills/62-gtm-validator"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-dintel",
|
||||
"description": "D.intelligence Agent Corps — brand guardian/editor, document secretary, quotation manager, service architect, marketing manager, back-office manager, account manager, and cross-skill update meta-agent.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/70-dintel-brand-guardian",
|
||||
"./custom-skills/71-dintel-brand-editor",
|
||||
"./custom-skills/72-dintel-doc-secretary",
|
||||
"./custom-skills/73-dintel-quotation-mgr",
|
||||
"./custom-skills/74-dintel-service-architect",
|
||||
"./custom-skills/75-dintel-marketing-mgr",
|
||||
"./custom-skills/76-dintel-backoffice-mgr",
|
||||
"./custom-skills/77-dintel-account-mgr",
|
||||
"./custom-skills/79-dintel-skill-update"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "ourdigital-utils",
|
||||
"description": "Utility skills — Claude settings/token optimizer, Google Drive organizer, reference-documentation curator suite, and TUI wizard design template.",
|
||||
"source": "./",
|
||||
"strict": false,
|
||||
"skills": [
|
||||
"./custom-skills/80-claude-settings-optimizer",
|
||||
"./custom-skills/82-our-gdrive-organizer",
|
||||
"./custom-skills/90-reference-curator",
|
||||
"./custom-skills/92-tui-design-template"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "mac-optimizer",
|
||||
"description": "macOS system health toolkit — read-only audits, cleanup, and security checks. Commands: mac-doctor, mac-packages, mac-environment, mac-security, mac-cleanup, mac-resources (+ mac-optimizer skill).",
|
||||
"source": "./custom-skills/81-mac-optimizer",
|
||||
"strict": false
|
||||
},
|
||||
{
|
||||
"name": "multi-agent-guide",
|
||||
"description": "Multi-agent collaboration framework — agent hierarchies, ownership rules, guardrails, handoff protocols, and CI/CD integration for Claude, Gemini, Codex, and human agents. Commands: quick-setup, setup-agents (+ multi-agent-guide skill).",
|
||||
"source": "./custom-skills/91-multi-agent-guide",
|
||||
"strict": false
|
||||
},
|
||||
{
|
||||
"name": "dintel-bootstrap",
|
||||
"description": "Install and verify D.intelligence custom MCP agents (DTM, D.DA, OurSEO) in settings.json; bootstrap a new machine or diagnose a broken install (dintel-bootstrap skill).",
|
||||
"source": "./custom-skills/94-dintel-bootstrap",
|
||||
"strict": false
|
||||
}
|
||||
]
|
||||
}
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -99,3 +99,6 @@ build/
|
||||
# Temporary files
|
||||
output/
|
||||
keyword_analysis_*.json
|
||||
|
||||
# notion-writer skill venv
|
||||
custom-skills/32-notion-writer/code/scripts/venv/
|
||||
|
||||
20
README.md
20
README.md
@@ -12,6 +12,11 @@ cd our-claude-skills/custom-skills/_ourdigital-shared
|
||||
./install.sh
|
||||
```
|
||||
|
||||
This symlinks the global slash commands into `~/.claude/commands/`, sets up the
|
||||
Python virtual environment, and configures credentials. It does **not** register
|
||||
skills natively — to load a skill as a Claude Code skill, also symlink it into
|
||||
`~/.claude/skills/` (see [Usage → Claude Code](#claude-code)).
|
||||
|
||||
## Custom Skills Overview
|
||||
|
||||
### OurDigital Core (01-10)
|
||||
@@ -215,16 +220,23 @@ The `_ourdigital-shared/` directory provides:
|
||||
|
||||
### Claude Code
|
||||
|
||||
Skills are auto-detected via symlinks in `~/.claude/skills/`:
|
||||
The Quick Install symlinks the **slash commands** into `~/.claude/commands/`. To
|
||||
also load a skill natively, symlink its **root** directory (which holds the
|
||||
loadable `SKILL.md`) into `~/.claude/skills/`, using the clean name without the
|
||||
`NN-` prefix:
|
||||
|
||||
```bash
|
||||
# Install skill symlink
|
||||
ln -sf /path/to/skill/desktop ~/.claude/skills/skill-name
|
||||
# From the repo root — symlink a skill into Claude Code
|
||||
ln -sf "$PWD/custom-skills/16-seo-schema-validator" ~/.claude/skills/seo-schema-validator
|
||||
```
|
||||
|
||||
> Legacy skills that don't yet have a root `SKILL.md` expose it under
|
||||
> `code/SKILL.md` instead — symlink `.../<skill>/code` for those.
|
||||
|
||||
### Claude Desktop
|
||||
|
||||
Copy the `desktop/SKILL.md` file to your Claude Desktop skills folder.
|
||||
Import the skill's `desktop/` folder (containing `SKILL.md` + `skill.yaml`) via
|
||||
your Claude Desktop skills settings.
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
@@ -139,6 +139,19 @@ python notion_writer.py -d DATABASE_URL -t "Entry Title" -f content.md
|
||||
| `---` | Divider |
|
||||
| Paragraphs | Paragraph |
|
||||
|
||||
### Engines and image uploads
|
||||
|
||||
Two write engines via `--engine {blocks,markdown}` (default: `blocks`).
|
||||
|
||||
The **blocks engine** (default) converts markdown locally to Notion block objects. Local images (``) are auto-uploaded via the `ntn` CLI and embedded at their original position in the page. Requires `ntn` installed and `ntn login`.
|
||||
|
||||
The **markdown engine** (`--engine markdown`) posts the document through Notion's native enhanced-markdown API (`Notion-Version: 2026-03-11`, set automatically; override with `--notion-version`). The skill's authoring dialect — GitHub alerts (`[!NOTE]`), Pandoc columns (`::: columns`), `<details>` toggles, and `@[mention]` — is auto-translated before posting. Note: local images are appended at the end of the page rather than inline with this engine; use `--engine blocks` when image position matters. Pass `--allow-deleting-content` when `--replace` needs to remove child pages or databases.
|
||||
|
||||
```bash
|
||||
# Markdown engine — create a DB row from a doc with callouts or columns
|
||||
python notion_writer.py -d DB_URL -t "Notes" --engine markdown -f notes.md
|
||||
```
|
||||
|
||||
## Workflow Example
|
||||
|
||||
Integrate with Jamie YouTube Manager to log video info:
|
||||
|
||||
@@ -189,6 +189,52 @@ print("Hello")
|
||||
|
||||
Notion's URL validator requires absolute URLs for link annotations. The parser converts TOC-style anchor links to bold to preserve navigation intent and silently strips relative paths.
|
||||
|
||||
### File uploads
|
||||
|
||||
Standalone local-image lines (``) are auto-uploaded to Notion and embedded as `file_upload` image blocks. Remote images (``) are left as external links unchanged.
|
||||
|
||||
**Requirements:**
|
||||
- `ntn` CLI installed: `curl -fsSL https://ntn.dev | bash`
|
||||
|
||||
Uploads run as the **same integration** that writes the page (`NOTION_API_KEY`). The script injects `NOTION_API_TOKEN=$NOTION_API_KEY` into every `ntn` subprocess, so the file upload and the page share one identity — no separate `ntn login` or workspace matching is needed. `ntn` only needs to be installed, not logged in.
|
||||
|
||||
### Engines
|
||||
|
||||
Two write engines, selected with `--engine {blocks,markdown}`. Default is `blocks`.
|
||||
|
||||
| Engine | Flag | When to use |
|
||||
|--------|------|-------------|
|
||||
| **blocks** (default) | `--engine blocks` | General use; images embed at their exact position; full table + container support |
|
||||
| **markdown** | `--engine markdown` | Richer Notion-native formatting via enhanced-markdown endpoints |
|
||||
|
||||
**blocks engine** converts markdown to Notion block objects locally (via `markdown_to_notion_blocks`). Local images are uploaded via `ntn` and embedded at their exact position in the document.
|
||||
|
||||
**markdown engine** posts the document through Notion's native enhanced-markdown endpoints (`Notion-Version: 2026-03-11`, set automatically). The skill's authoring dialect is auto-translated before posting:
|
||||
|
||||
| Skill dialect | Translated to |
|
||||
|---------------|---------------|
|
||||
| `> [!NOTE]` / `[!TIP]` / `[!IMPORTANT]` / `[!WARNING]` / `[!CAUTION]` | `<callout icon="..." color="...">` |
|
||||
| `::: columns` / `::: column` / `:::` | `<columns><column>...</column></columns>` |
|
||||
| `<details><summary>...</summary>` | `<details>` (Notion toggle) |
|
||||
| `@[Title](id-or-url)` | `<mention-page url="...">` |
|
||||
|
||||
**Local image limitation with `--engine markdown`**: local images cannot be placed inline via the enhanced-markdown API. They are appended at the end of the page as a second write pass. Use `--engine blocks` when image placement matters.
|
||||
|
||||
**`--notion-version`**: Override the API version for the markdown engine (default: `2026-03-11`).
|
||||
|
||||
**`--allow-deleting-content`**: Required when `--replace` needs to delete child pages or databases under the target page; the Notion API refuses such deletions unless this flag is present. Applies only to the markdown engine's `--replace` path (`--engine markdown --replace`); has no effect with `--engine blocks`.
|
||||
|
||||
```bash
|
||||
# Markdown engine, create a row from a doc with callouts/columns
|
||||
python notion_writer.py -d DB_URL -t "Notes" --engine markdown -f notes.md
|
||||
|
||||
# Blocks engine with a local image (auto-uploaded via ntn)
|
||||
python notion_writer.py -p PAGE_URL -f post.md # post.md contains 
|
||||
|
||||
# Markdown engine, replace page allowing child deletion
|
||||
python notion_writer.py -p PAGE_URL -f doc.md --replace --engine markdown --allow-deleting-content
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
@@ -398,9 +444,10 @@ python notion_writer.py -d DB_URL -t "Title" --upsert-by "Name" -f content.md
|
||||
|
||||
---
|
||||
|
||||
*Version 1.2.0 | Claude Code | 2026-04-27*
|
||||
*Version 1.3.0 | Claude Code | 2026-06-27*
|
||||
|
||||
Changelog:
|
||||
- 1.3.0 — Local file/image uploads via the `ntn` CLI (`` → file_upload image blocks). New `--engine markdown` path writing through Notion's native enhanced-markdown endpoints with a dialect translator. Added `--notion-version` and `--allow-deleting-content`.
|
||||
- 1.2.0 — Extended block coverage: GitHub-alert callouts, HTML5 `<details>` toggles, Pandoc `::: columns` fenced div, inline `@[Title](id-or-url)` page mentions. Parser made reentrant to support full recursion inside container blocks.
|
||||
- 1.1.0 — Migrated to Notion API 2025-09-03 (multi-source databases). Added `--properties` JSON flag, `--upsert-by` for idempotency, anchor-link parser fix, friendlier API error messages.
|
||||
- 1.0.0 — Initial release with markdown→Notion block conversion.
|
||||
|
||||
@@ -16,8 +16,11 @@ from notion_client import Client
|
||||
from notion_client.errors import APIErrorCode, APIResponseError
|
||||
|
||||
|
||||
def make_client(api_key: str) -> Client:
|
||||
"""Build a sync Notion client on the SDK default API version (2025-09-03+)."""
|
||||
def make_client(api_key: str, notion_version: str = None) -> Client:
|
||||
"""Build a sync Notion client. Pass notion_version to override the SDK
|
||||
default (needed: 2026-03-11 for the markdown content endpoints)."""
|
||||
if notion_version:
|
||||
return Client(auth=api_key, notion_version=notion_version)
|
||||
return Client(auth=api_key)
|
||||
|
||||
|
||||
@@ -210,7 +213,10 @@ def explain_api_error(exc: APIResponseError, context: str = "") -> str:
|
||||
"https://www.notion.so/my-integrations."
|
||||
)
|
||||
if code == APIErrorCode.ValidationError:
|
||||
return f"Validation error{suffix}: {exc.body.get('message', str(exc))}"
|
||||
msg = exc.body.get('message', str(exc))
|
||||
if 'delet' in msg.lower():
|
||||
msg += " — re-run with --allow-deleting-content to permit this."
|
||||
return f"Validation error{suffix}: {msg}"
|
||||
if code == APIErrorCode.RateLimited:
|
||||
return f"Rate limited{suffix}. Back off and retry."
|
||||
return f"Notion API error [{code}]{suffix}: {exc}"
|
||||
@@ -253,3 +259,33 @@ def find_existing_page(
|
||||
)
|
||||
results = response.get("results") or []
|
||||
return results[0] if results else None
|
||||
|
||||
|
||||
def create_page_markdown(client, parent, properties, markdown):
|
||||
"""POST /v1/pages with the enhanced-markdown body param.
|
||||
`markdown` is only included in the body when non-empty to avoid sending
|
||||
a blank markdown field when no --file/--stdin was supplied."""
|
||||
body: Dict[str, Any] = {"parent": parent, "properties": properties}
|
||||
if markdown:
|
||||
body["markdown"] = markdown
|
||||
return client.request(path="pages", method="POST", body=body)
|
||||
|
||||
|
||||
def append_markdown(client, page_id, markdown):
|
||||
"""PATCH /v1/pages/:id/markdown — append at end (insert_content)."""
|
||||
return client.request(
|
||||
path=f"pages/{page_id}/markdown", method="PATCH",
|
||||
body={"type": "insert_content",
|
||||
"insert_content": {"content": markdown,
|
||||
"position": {"type": "end"}}},
|
||||
)
|
||||
|
||||
|
||||
def replace_markdown(client, page_id, markdown, allow_deleting=False):
|
||||
"""PATCH /v1/pages/:id/markdown — replace all content."""
|
||||
return client.request(
|
||||
path=f"pages/{page_id}/markdown", method="PATCH",
|
||||
body={"type": "replace_content",
|
||||
"replace_content": {"new_str": markdown,
|
||||
"allow_deleting_content": allow_deleting}},
|
||||
)
|
||||
|
||||
135
custom-skills/32-notion-writer/code/scripts/md_translate.py
Normal file
135
custom-skills/32-notion-writer/code/scripts/md_translate.py
Normal file
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Translate the notion-writer markdown dialect into Notion enhanced
|
||||
markdown for the markdown write engine. Pure functions, no I/O."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import List
|
||||
|
||||
# NOTE: `notion_writer` is imported lazily inside _translate_mentions to avoid
|
||||
# a circular import (notion_writer imports this module at its top level).
|
||||
|
||||
# Skill alert type -> (emoji, Notion enhanced-markdown background color)
|
||||
CALLOUT_MAP = {
|
||||
"NOTE": ("ℹ️", "blue_bg"),
|
||||
"TIP": ("💡", "green_bg"),
|
||||
"IMPORTANT": ("☝️", "purple_bg"),
|
||||
"WARNING": ("⚠️", "yellow_bg"),
|
||||
"CAUTION": ("🚨", "red_bg"),
|
||||
}
|
||||
_ALERT_RE = re.compile(r'^\s*\[!(NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]\s*$')
|
||||
_MENTION_RE = re.compile(r'@\[([^\]]+)\]\(([^)\s]+)\)')
|
||||
|
||||
|
||||
def _translate_mentions(text: str) -> str:
|
||||
from notion_writer import extract_notion_id, format_id_with_dashes # lazy: breaks import cycle
|
||||
|
||||
def repl(m: "re.Match") -> str:
|
||||
title, target = m.group(1), m.group(2)
|
||||
page_id = extract_notion_id(target)
|
||||
if page_id:
|
||||
return (f'<mention-page url="{format_id_with_dashes(page_id)}">'
|
||||
f'{title}</mention-page>')
|
||||
return f"@{title}"
|
||||
return _MENTION_RE.sub(repl, text)
|
||||
|
||||
|
||||
def _indent(lines: List[str]) -> List[str]:
|
||||
return ["\t" + ln if ln.strip() else ln for ln in lines]
|
||||
|
||||
|
||||
def translate(content: str) -> str:
|
||||
lines = content.split("\n")
|
||||
out: List[str] = []
|
||||
i = 0
|
||||
in_fence = False
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
|
||||
# Fence tracking: pass through unchanged, toggle state
|
||||
if line.strip().startswith("```"):
|
||||
in_fence = not in_fence
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Inside fence: pass through verbatim, no transformation
|
||||
if in_fence:
|
||||
out.append(line)
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Callout: > [!TYPE] then contiguous > body lines
|
||||
if line.lstrip().startswith(">"):
|
||||
body_first = line.lstrip()[1:].strip()
|
||||
alert = _ALERT_RE.match(body_first)
|
||||
if alert:
|
||||
emoji, color = CALLOUT_MAP[alert.group(1)]
|
||||
i += 1
|
||||
body: List[str] = []
|
||||
while i < len(lines) and lines[i].lstrip().startswith(">"):
|
||||
body.append(lines[i].lstrip()[1:].lstrip())
|
||||
i += 1
|
||||
out.append(f'<callout icon="{emoji}" color="{color}">')
|
||||
out.extend(_indent([_translate_mentions(b) for b in body]))
|
||||
out.append("</callout>")
|
||||
continue
|
||||
|
||||
# Columns: ::: columns / ::: column / :::
|
||||
# State machine: "::: column" opens a column; ":::" closes a column
|
||||
# when one is open, otherwise closes the wrapper. The Pandoc layout
|
||||
# emits N "::: column" opens, N ":::" column-closes, then one final
|
||||
# ":::" wrapper-close.
|
||||
if line.strip() == "::: columns":
|
||||
i += 1
|
||||
cols: List[List[str]] = []
|
||||
cur: List[str] = None
|
||||
while i < len(lines):
|
||||
s = lines[i].strip()
|
||||
if s == "::: column":
|
||||
if cur is not None:
|
||||
cols.append(cur)
|
||||
cur = []
|
||||
i += 1
|
||||
elif s == ":::":
|
||||
if cur is not None: # close the open column
|
||||
cols.append(cur)
|
||||
cur = None
|
||||
i += 1
|
||||
else: # close the wrapper
|
||||
i += 1
|
||||
break
|
||||
else:
|
||||
if cur is not None:
|
||||
cur.append(lines[i])
|
||||
i += 1
|
||||
out.append("<columns>")
|
||||
for col in cols:
|
||||
out.append("\t<column>")
|
||||
out.extend(_indent(_indent(
|
||||
[_translate_mentions(c) for c in col])))
|
||||
out.append("\t</column>")
|
||||
out.append("</columns>")
|
||||
continue
|
||||
|
||||
# Toggle: <details><summary>..</summary> body </details>
|
||||
if line.strip() == "<details>":
|
||||
out.append("<details>")
|
||||
i += 1
|
||||
if i < len(lines) and lines[i].lstrip().startswith("<summary>"):
|
||||
out.append(lines[i].strip())
|
||||
i += 1
|
||||
body = []
|
||||
while i < len(lines) and lines[i].strip() != "</details>":
|
||||
body.append(_translate_mentions(lines[i]))
|
||||
i += 1
|
||||
out.extend(_indent(body))
|
||||
out.append("</details>")
|
||||
if i < len(lines): # skip closing </details>
|
||||
i += 1
|
||||
continue
|
||||
|
||||
out.append(_translate_mentions(line))
|
||||
i += 1
|
||||
return "\n".join(out)
|
||||
@@ -17,6 +17,11 @@ from notion_client import Client
|
||||
from notion_client.errors import APIResponseError
|
||||
|
||||
import _notion_compat as compat
|
||||
import ntn_files
|
||||
from ntn_files import NtnUploadError
|
||||
import md_translate
|
||||
|
||||
MARKDOWN_NOTION_VERSION = "2026-03-11"
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv(Path(__file__).parent / '.env')
|
||||
@@ -56,6 +61,7 @@ def format_id_with_dashes(raw_id: str) -> str:
|
||||
|
||||
|
||||
TABLE_SEPARATOR_RE = re.compile(r'^\s*\|?(\s*:?-{3,}:?\s*\|)+\s*:?-{3,}:?\s*\|?\s*$')
|
||||
IMAGE_RE = re.compile(r'^\s*!\[([^\]]*)\]\(([^)\s]+)\)\s*$')
|
||||
|
||||
|
||||
def _is_table_row(line: str) -> bool:
|
||||
@@ -210,6 +216,13 @@ def _parse_lines(lines: List[str]) -> List[Dict[str, Any]]:
|
||||
blocks.append(create_column_list_block(column_blocks))
|
||||
continue
|
||||
|
||||
image_match = IMAGE_RE.match(line)
|
||||
if image_match:
|
||||
blocks.append(create_image_block(
|
||||
image_match.group(1), image_match.group(2)))
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if _is_table_row(line) and i + 1 < len(lines) and TABLE_SEPARATOR_RE.match(lines[i + 1]):
|
||||
header_cells = _split_table_row(line)
|
||||
i += 2
|
||||
@@ -507,6 +520,55 @@ def create_divider_block() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def create_image_block(alt: str, target: str) -> Dict[str, Any]:
|
||||
"""Image block in external shape. Local targets are converted to
|
||||
file_upload shape later by ntn_files.materialize_local_media."""
|
||||
block: Dict[str, Any] = {
|
||||
"object": "block",
|
||||
"type": "image",
|
||||
"image": {"type": "external", "external": {"url": target}},
|
||||
}
|
||||
if alt:
|
||||
block["image"]["caption"] = parse_rich_text(alt)
|
||||
return block
|
||||
|
||||
|
||||
def _content_has_local_images(content: str) -> bool:
|
||||
in_fence = False
|
||||
for line in content.split('\n'):
|
||||
if line.strip().startswith('```'):
|
||||
in_fence = not in_fence
|
||||
continue # fence delimiter lines are never image lines
|
||||
if in_fence:
|
||||
continue
|
||||
m = IMAGE_RE.match(line)
|
||||
if m and not m.group(2).startswith(('http://', 'https://')):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def extract_local_images(content: str):
|
||||
"""Remove standalone LOCAL image lines; keep remote ones inline.
|
||||
Returns (content_without_local_images, [(alt, target), ...]).
|
||||
Lines inside fenced code blocks are passed through unchanged."""
|
||||
kept, imgs = [], []
|
||||
in_fence = False
|
||||
for line in content.split('\n'):
|
||||
if line.strip().startswith('```'):
|
||||
in_fence = not in_fence
|
||||
kept.append(line)
|
||||
continue
|
||||
if in_fence:
|
||||
kept.append(line)
|
||||
continue
|
||||
m = IMAGE_RE.match(line)
|
||||
if m and not m.group(2).startswith(('http://', 'https://')):
|
||||
imgs.append((m.group(1), m.group(2)))
|
||||
continue
|
||||
kept.append(line)
|
||||
return "\n".join(kept), imgs
|
||||
|
||||
|
||||
def create_table_block(header_cells: List[str], body_rows: List[List[str]]) -> Dict[str, Any]:
|
||||
"""Build a Notion `table` block with header + body rows.
|
||||
|
||||
@@ -746,21 +808,6 @@ def update_page_properties(notion: Client, page_id: str, properties: Dict) -> bo
|
||||
return False
|
||||
|
||||
|
||||
def write_to_page(notion: Client, page_id: str, markdown_content: str, mode: str = 'append') -> bool:
|
||||
"""Write markdown content to a Notion page."""
|
||||
blocks = markdown_to_notion_blocks(markdown_content)
|
||||
|
||||
if not blocks:
|
||||
print("No content to write")
|
||||
return False
|
||||
|
||||
if mode == 'replace':
|
||||
if not clear_page_content(notion, page_id):
|
||||
return False
|
||||
|
||||
return append_to_page(notion, page_id, blocks)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Push markdown content to Notion pages or databases',
|
||||
@@ -796,9 +843,28 @@ Examples:
|
||||
parser.add_argument('--list', '-l', nargs='?', const='all', choices=['all', 'pages', 'databases'],
|
||||
help='List accessible pages and/or databases (default: all)')
|
||||
parser.add_argument('--info', action='store_true', help='Show page/database info')
|
||||
parser.add_argument('--engine', choices=['blocks', 'markdown'],
|
||||
default='blocks',
|
||||
help='Write engine: blocks (default) or markdown')
|
||||
parser.add_argument('--notion-version', dest='notion_version',
|
||||
help='Override Notion API version')
|
||||
parser.add_argument('--allow-deleting-content', dest='allow_deleting',
|
||||
action='store_true',
|
||||
help='Markdown replace may delete child pages/dbs')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
_main_body(parser, args)
|
||||
except NtnUploadError as e:
|
||||
print(f"Error: {e}")
|
||||
if e.stderr:
|
||||
print(e.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def _main_body(parser, args):
|
||||
"""Body of main() after argparse; separated so NtnUploadError can be caught cleanly."""
|
||||
if not NOTION_TOKEN:
|
||||
print("Error: NOTION_API_KEY not set in environment.")
|
||||
print("Preferred: fetch from 1Password at runtime —")
|
||||
@@ -885,23 +951,64 @@ Examples:
|
||||
sys.exit(1)
|
||||
content = file_path.read_text(encoding='utf-8')
|
||||
|
||||
base_dir = Path(args.file).parent if args.file else Path.cwd()
|
||||
|
||||
def _md_client():
|
||||
version = args.notion_version or MARKDOWN_NOTION_VERSION
|
||||
return compat.make_client(NOTION_TOKEN, notion_version=version)
|
||||
|
||||
def _upload_blocks_for(images):
|
||||
"""Build + materialize image blocks for two-phase markdown writes."""
|
||||
blocks = [create_image_block(alt, target) for alt, target in images]
|
||||
return ntn_files.materialize_local_media(blocks, base_dir, ntn_files.upload)
|
||||
|
||||
# Write to page
|
||||
if args.page:
|
||||
if not content:
|
||||
print("Error: No content provided. Use --file or --stdin")
|
||||
sys.exit(1)
|
||||
|
||||
page_id = extract_notion_id(args.page)
|
||||
if not page_id:
|
||||
print(f"Error: Invalid Notion page URL/ID: {args.page}")
|
||||
sys.exit(1)
|
||||
|
||||
mode = 'replace' if args.replace else 'append'
|
||||
print(f"{'Replacing' if mode == 'replace' else 'Appending'} content to page...")
|
||||
|
||||
if write_to_page(notion, page_id, content, mode):
|
||||
print(f"✅ Successfully wrote content to page")
|
||||
formatted_id = format_id_with_dashes(page_id)
|
||||
|
||||
if args.engine == 'markdown':
|
||||
if _content_has_local_images(content):
|
||||
ntn_files.preflight()
|
||||
md_body, local_imgs = extract_local_images(content)
|
||||
md_body = md_translate.translate(md_body)
|
||||
mc = _md_client()
|
||||
try:
|
||||
if args.replace:
|
||||
compat.replace_markdown(mc, formatted_id, md_body,
|
||||
allow_deleting=args.allow_deleting)
|
||||
else:
|
||||
compat.append_markdown(mc, formatted_id, md_body)
|
||||
except APIResponseError as exc:
|
||||
print(f"Error: {compat.explain_api_error(exc, formatted_id)}")
|
||||
sys.exit(1)
|
||||
if local_imgs and not append_to_page(notion, page_id, _upload_blocks_for(local_imgs)):
|
||||
print("❌ Text was written but image append failed")
|
||||
sys.exit(1)
|
||||
print("✅ Successfully wrote content to page (markdown engine)")
|
||||
print(f" https://notion.so/{formatted_id.replace('-', '')}")
|
||||
return
|
||||
|
||||
# blocks engine (default)
|
||||
blocks = markdown_to_notion_blocks(content)
|
||||
if _content_has_local_images(content):
|
||||
ntn_files.preflight()
|
||||
blocks = ntn_files.materialize_local_media(blocks, base_dir, ntn_files.upload)
|
||||
if not blocks:
|
||||
print("No content to write")
|
||||
sys.exit(1)
|
||||
print(f"{'Replacing' if args.replace else 'Appending'} content to page...")
|
||||
if args.replace and not clear_page_content(notion, page_id):
|
||||
print("❌ Failed to write content")
|
||||
sys.exit(1)
|
||||
if append_to_page(notion, page_id, blocks):
|
||||
print("✅ Successfully wrote content to page")
|
||||
print(f" https://notion.so/{formatted_id.replace('-', '')}")
|
||||
else:
|
||||
print("❌ Failed to write content")
|
||||
@@ -960,6 +1067,8 @@ Examples:
|
||||
|
||||
content_blocks = markdown_to_notion_blocks(content) if content else None
|
||||
|
||||
existing = None
|
||||
|
||||
# Upsert path: look for existing row by the named property
|
||||
if args.upsert_by:
|
||||
if args.upsert_by not in schema_props:
|
||||
@@ -980,6 +1089,39 @@ Examples:
|
||||
print(f"Error during upsert lookup: {compat.explain_api_error(exc)}")
|
||||
sys.exit(1)
|
||||
|
||||
if args.engine == 'markdown':
|
||||
md_content = content or ""
|
||||
if _content_has_local_images(md_content):
|
||||
ntn_files.preflight()
|
||||
md_body, local_imgs = extract_local_images(md_content)
|
||||
md_body = md_translate.translate(md_body)
|
||||
mc = _md_client()
|
||||
try:
|
||||
if args.upsert_by and existing:
|
||||
compat.replace_markdown(mc, existing['id'], md_body,
|
||||
allow_deleting=args.allow_deleting)
|
||||
if not update_page_properties(notion, existing['id'], properties):
|
||||
sys.exit(1)
|
||||
new_id = existing['id']
|
||||
else:
|
||||
parent = compat.build_data_source_parent(data_source_id)
|
||||
result = compat.create_page_markdown(mc, parent, properties, md_body)
|
||||
new_id = result['id']
|
||||
except APIResponseError as exc:
|
||||
print(f"Error: {compat.explain_api_error(exc)}")
|
||||
sys.exit(1)
|
||||
if local_imgs and not append_to_page(notion, new_id, _upload_blocks_for(local_imgs)):
|
||||
print("❌ Text was written but image append failed")
|
||||
sys.exit(1)
|
||||
print("✅ Successfully wrote database row (markdown engine)")
|
||||
print(f" https://notion.so/{new_id.replace('-', '')}")
|
||||
return
|
||||
|
||||
if content_blocks and _content_has_local_images(content or ""):
|
||||
ntn_files.preflight()
|
||||
content_blocks = ntn_files.materialize_local_media(
|
||||
content_blocks, base_dir, ntn_files.upload)
|
||||
|
||||
if existing:
|
||||
page_id = existing['id']
|
||||
print(f"Updating existing row (matched on {args.upsert_by}={lookup_value!r})...")
|
||||
|
||||
154
custom-skills/32-notion-writer/code/scripts/ntn_files.py
Normal file
154
custom-skills/32-notion-writer/code/scripts/ntn_files.py
Normal file
@@ -0,0 +1,154 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Local file uploads to Notion via the `ntn` CLI.
|
||||
|
||||
Owns all subprocess I/O for the skill. The CLI handles the full File Upload
|
||||
lifecycle (create -> send bytes -> complete) and prints the upload ID.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Callable, Any
|
||||
|
||||
_WORKSPACE: Optional[Dict[str, str]] = None
|
||||
|
||||
|
||||
def _ntn_env():
|
||||
"""Env for ntn subprocesses: force ntn to authenticate as the SAME
|
||||
integration the script uses (NOTION_API_KEY/NOTION_TOKEN), so an uploaded
|
||||
file and the page that references it share one identity. Notion scopes
|
||||
file uploads to the creating integration, so a mismatch makes the upload
|
||||
un-attachable."""
|
||||
env = dict(os.environ)
|
||||
token = os.environ.get("NOTION_API_KEY") or os.environ.get("NOTION_TOKEN")
|
||||
if token:
|
||||
env["NOTION_API_TOKEN"] = token
|
||||
return env
|
||||
|
||||
|
||||
class NtnUploadError(Exception):
|
||||
"""Raised when `ntn` is unavailable or a file upload fails."""
|
||||
|
||||
def __init__(self, message: str, path: str = "", stderr: str = ""):
|
||||
super().__init__(message)
|
||||
self.path = path
|
||||
self.stderr = stderr
|
||||
|
||||
|
||||
def preflight() -> Dict[str, str]:
|
||||
"""Verify `ntn` is installed and logged in; return its target workspace.
|
||||
|
||||
Cached for the process. Raises NtnUploadError with an actionable hint
|
||||
on failure. Prints one informational line naming the workspace `ntn`
|
||||
targets (file uploads are workspace-scoped).
|
||||
"""
|
||||
global _WORKSPACE
|
||||
if _WORKSPACE is not None:
|
||||
return _WORKSPACE
|
||||
|
||||
if shutil.which("ntn") is None:
|
||||
raise NtnUploadError(
|
||||
"ntn CLI not found. Install it with: "
|
||||
"curl -fsSL https://ntn.dev | bash"
|
||||
)
|
||||
|
||||
result = subprocess.run(
|
||||
["ntn", "api", "v1/users/me"],
|
||||
capture_output=True, text=True, env=_ntn_env(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise NtnUploadError(
|
||||
"ntn is not logged in. Run: ntn login\n" + result.stderr.strip()
|
||||
)
|
||||
|
||||
try:
|
||||
me = json.loads(result.stdout)
|
||||
bot = me.get("bot", {})
|
||||
info = {
|
||||
"workspace_name": bot.get("workspace_name", "unknown"),
|
||||
"workspace_id": bot.get("workspace_id", "unknown"),
|
||||
}
|
||||
except (ValueError, AttributeError):
|
||||
info = {"workspace_name": "unknown", "workspace_id": "unknown"}
|
||||
|
||||
print(f'ntn -> workspace "{info["workspace_name"]}"', file=sys.stderr)
|
||||
_WORKSPACE = info
|
||||
return info
|
||||
|
||||
|
||||
def upload(path: Path) -> str:
|
||||
"""Upload a local file via `ntn files create --plain`; return its id."""
|
||||
path = Path(path)
|
||||
with open(path, "rb") as fh:
|
||||
result = subprocess.run(
|
||||
["ntn", "files", "create", "--plain"],
|
||||
stdin=fh, capture_output=True, text=True, env=_ntn_env(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise NtnUploadError(
|
||||
f"ntn upload failed for {path}", path=str(path),
|
||||
stderr=result.stderr.strip(),
|
||||
)
|
||||
if not result.stdout.strip():
|
||||
raise NtnUploadError(
|
||||
"ntn returned no output", path=str(path), stderr=result.stderr.strip()
|
||||
)
|
||||
first_line = result.stdout.strip().splitlines()[0]
|
||||
upload_id = first_line.split("\t")[0].strip()
|
||||
return upload_id
|
||||
|
||||
|
||||
_REMOTE_PREFIXES = ("http://", "https://")
|
||||
|
||||
|
||||
def _resolve_local(url: str, base_dir: Path) -> Optional[Path]:
|
||||
"""Return the resolved path for a local media url, or None if remote."""
|
||||
if url.startswith(_REMOTE_PREFIXES):
|
||||
return None
|
||||
p = Path(url)
|
||||
return p if p.is_absolute() else Path(base_dir) / p
|
||||
|
||||
|
||||
def materialize_local_media(
|
||||
blocks: List[Dict[str, Any]],
|
||||
base_dir: Path,
|
||||
upload_fn: Callable[[Path], str] = upload,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Walk blocks (and nested children); upload local image files and
|
||||
rewrite them to file_upload shape. Remote images are left as-is."""
|
||||
for block in blocks:
|
||||
if block.get("type") == "image":
|
||||
img = block["image"]
|
||||
if img.get("type") == "external":
|
||||
url = img.get("external", {}).get("url", "")
|
||||
resolved = _resolve_local(url, base_dir)
|
||||
if resolved is not None:
|
||||
if not resolved.is_file():
|
||||
raise NtnUploadError(
|
||||
f"image references missing file: {url}",
|
||||
path=str(resolved),
|
||||
)
|
||||
upload_id = upload_fn(resolved)
|
||||
caption = img.get("caption")
|
||||
new_img: Dict[str, Any] = {
|
||||
"type": "file_upload",
|
||||
"file_upload": {"id": upload_id},
|
||||
}
|
||||
if caption:
|
||||
new_img["caption"] = caption
|
||||
block["image"] = new_img
|
||||
# Recurse into any nested children list.
|
||||
btype = block.get("type")
|
||||
nested = block.get(btype, {}) if isinstance(block.get(btype), dict) else {}
|
||||
if isinstance(nested.get("children"), list):
|
||||
materialize_local_media(nested["children"], base_dir, upload_fn)
|
||||
if btype == "column_list":
|
||||
for col in nested.get("children", []):
|
||||
col_children = col.get("column", {}).get("children", [])
|
||||
materialize_local_media(col_children, base_dir, upload_fn)
|
||||
return blocks
|
||||
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for markdown transport helpers — run with `python test_engine_routing.py`."""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
import _notion_compat as compat
|
||||
import notion_writer
|
||||
|
||||
|
||||
def _fake_client():
|
||||
c = mock.MagicMock()
|
||||
c.request.return_value = {"object": "page", "id": "p1"}
|
||||
return c
|
||||
|
||||
|
||||
def test_create_page_markdown():
|
||||
c = _fake_client()
|
||||
parent = {"type": "data_source_id", "data_source_id": "ds1"}
|
||||
compat.create_page_markdown(c, parent, {"Name": {"title": []}}, "# Hi")
|
||||
_, kwargs = c.request.call_args
|
||||
assert kwargs["path"] == "pages"
|
||||
assert kwargs["method"] == "POST"
|
||||
assert kwargs["body"]["markdown"] == "# Hi"
|
||||
assert kwargs["body"]["parent"] == parent
|
||||
|
||||
|
||||
def test_append_markdown():
|
||||
c = _fake_client()
|
||||
compat.append_markdown(c, "page-123", "## More")
|
||||
_, kwargs = c.request.call_args
|
||||
assert kwargs["path"] == "pages/page-123/markdown"
|
||||
assert kwargs["method"] == "PATCH"
|
||||
assert kwargs["body"]["type"] == "insert_content"
|
||||
assert kwargs["body"]["insert_content"]["content"] == "## More"
|
||||
assert kwargs["body"]["insert_content"]["position"] == {"type": "end"}
|
||||
|
||||
|
||||
def test_replace_markdown():
|
||||
c = _fake_client()
|
||||
compat.replace_markdown(c, "page-123", "# Fresh", allow_deleting=True)
|
||||
_, kwargs = c.request.call_args
|
||||
assert kwargs["path"] == "pages/page-123/markdown"
|
||||
assert kwargs["body"]["type"] == "replace_content"
|
||||
assert kwargs["body"]["replace_content"]["new_str"] == "# Fresh"
|
||||
assert kwargs["body"]["replace_content"]["allow_deleting_content"] is True
|
||||
|
||||
|
||||
def test_extract_local_images_splits():
|
||||
content = ("# Title\n\n\n\n"
|
||||
"\n\nbody")
|
||||
without, imgs = notion_writer.extract_local_images(content)
|
||||
assert imgs == [("local", "./b.png")]
|
||||
assert "" not in without
|
||||
assert "" in without # remote stays
|
||||
|
||||
|
||||
def test_content_has_local_images():
|
||||
assert notion_writer._content_has_local_images("") is True
|
||||
assert notion_writer._content_has_local_images("") is False
|
||||
assert notion_writer._content_has_local_images("no images") is False
|
||||
|
||||
|
||||
def test_local_image_inside_fence_ignored():
|
||||
"""A local image reference inside a fenced code block must NOT be detected
|
||||
as a real image by _content_has_local_images or extracted by extract_local_images."""
|
||||
fenced = "```markdown\n\n```"
|
||||
assert notion_writer._content_has_local_images(fenced) is False, \
|
||||
"_content_has_local_images should return False for image inside fence"
|
||||
without, imgs = notion_writer.extract_local_images(fenced)
|
||||
assert len(imgs) == 0, "extract_local_images should not extract image inside fence"
|
||||
# The fenced lines (including the image line) should be preserved verbatim
|
||||
assert "" in without, "fence content must be preserved in output"
|
||||
|
||||
|
||||
def run_all():
|
||||
tests = [
|
||||
test_create_page_markdown,
|
||||
test_append_markdown,
|
||||
test_replace_markdown,
|
||||
test_extract_local_images_splits,
|
||||
test_content_has_local_images,
|
||||
test_local_image_inside_fence_ignored,
|
||||
]
|
||||
for t in tests:
|
||||
print(f"\n{t.__name__}")
|
||||
t()
|
||||
print("\n" + "=" * 50)
|
||||
print(f"✅ All {len(tests)} tests passed")
|
||||
print("=" * 50)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_all()
|
||||
101
custom-skills/32-notion-writer/code/scripts/test_md_translate.py
Normal file
101
custom-skills/32-notion-writer/code/scripts/test_md_translate.py
Normal file
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for md_translate.py — run with `python test_md_translate.py`."""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
import md_translate
|
||||
|
||||
|
||||
def test_passthrough_basics():
|
||||
src = "# Heading\n\n- item\n\n```python\nx=1\n```"
|
||||
assert md_translate.translate(src) == src
|
||||
|
||||
|
||||
def test_callout_note():
|
||||
out = md_translate.translate("> [!NOTE]\n> Be careful")
|
||||
assert '<callout icon="ℹ️" color="blue_bg">' in out
|
||||
assert "\tBe careful" in out
|
||||
assert "</callout>" in out
|
||||
|
||||
|
||||
def test_callout_warning_color():
|
||||
out = md_translate.translate("> [!WARNING]\n> Danger")
|
||||
assert 'color="yellow_bg"' in out
|
||||
assert 'icon="⚠️"' in out
|
||||
|
||||
|
||||
def test_columns():
|
||||
src = "::: columns\n::: column\nLeft\n:::\n::: column\nRight\n:::\n:::"
|
||||
out = md_translate.translate(src)
|
||||
assert "<columns>" in out
|
||||
assert out.count("<column>") == 2
|
||||
assert "</columns>" in out
|
||||
assert "\tLeft" in out or "\t\tLeft" in out
|
||||
|
||||
|
||||
def test_toggle_children_indented():
|
||||
src = "<details>\n<summary>More</summary>\nBody line\n</details>"
|
||||
out = md_translate.translate(src)
|
||||
assert "<summary>More</summary>" in out
|
||||
assert "\tBody line" in out
|
||||
|
||||
|
||||
def test_mention_url():
|
||||
out = md_translate.translate(
|
||||
"See @[ADR](https://notion.so/X-abcdef0123456789abcdef0123456789).")
|
||||
assert "<mention-page url=" in out
|
||||
assert ">ADR</mention-page>" in out
|
||||
|
||||
|
||||
def test_mention_invalid_plain():
|
||||
out = md_translate.translate("ping @[Bob](not-an-id)")
|
||||
assert "@Bob" in out
|
||||
assert "mention-page" not in out
|
||||
|
||||
|
||||
def test_fence_passthrough_no_transform():
|
||||
"""Lines inside fenced code blocks must pass through verbatim — no callout,
|
||||
columns, or mention transformation applied."""
|
||||
raw_id = "abcdef0123456789abcdef0123456789"
|
||||
src = (
|
||||
"```\n"
|
||||
"> [!NOTE]\n"
|
||||
"::: columns\n"
|
||||
f"@[Page]({raw_id})\n"
|
||||
"```"
|
||||
)
|
||||
out = md_translate.translate(src)
|
||||
# No transformation should have occurred
|
||||
assert "<callout" not in out, "callout must not be emitted inside fence"
|
||||
assert "<columns>" not in out, "columns must not be emitted inside fence"
|
||||
assert "<mention-page" not in out, "mention must not be emitted inside fence"
|
||||
# Original lines must be present verbatim
|
||||
assert "> [!NOTE]" in out
|
||||
assert "::: columns" in out
|
||||
assert f"@[Page]({raw_id})" in out
|
||||
|
||||
|
||||
def run_all():
|
||||
tests = [
|
||||
test_passthrough_basics,
|
||||
test_callout_note,
|
||||
test_callout_warning_color,
|
||||
test_columns,
|
||||
test_toggle_children_indented,
|
||||
test_mention_url,
|
||||
test_mention_invalid_plain,
|
||||
test_fence_passthrough_no_transform,
|
||||
]
|
||||
for t in tests:
|
||||
print(f"\n{t.__name__}")
|
||||
t()
|
||||
print("\n" + "=" * 50)
|
||||
print(f"✅ All {len(tests)} tests passed")
|
||||
print("=" * 50)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_all()
|
||||
152
custom-skills/32-notion-writer/code/scripts/test_ntn_files.py
Normal file
152
custom-skills/32-notion-writer/code/scripts/test_ntn_files.py
Normal file
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for ntn_files.py — run with `python test_ntn_files.py`."""
|
||||
|
||||
import sys
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
import ntn_files
|
||||
from ntn_files import NtnUploadError
|
||||
|
||||
|
||||
def _reset_cache():
|
||||
ntn_files._WORKSPACE = None
|
||||
|
||||
|
||||
def test_preflight_missing_ntn():
|
||||
_reset_cache()
|
||||
with mock.patch("shutil.which", return_value=None):
|
||||
try:
|
||||
ntn_files.preflight()
|
||||
assert False, "expected NtnUploadError"
|
||||
except NtnUploadError as e:
|
||||
assert "ntn" in str(e).lower()
|
||||
|
||||
|
||||
def test_preflight_returns_workspace():
|
||||
_reset_cache()
|
||||
fake = subprocess.CompletedProcess(
|
||||
args=[], returncode=0,
|
||||
stdout='{"bot":{"workspace_name":"D.Intelligence",'
|
||||
'"workspace_id":"ws-123"}}',
|
||||
stderr="",
|
||||
)
|
||||
with mock.patch("shutil.which", return_value="/usr/bin/ntn"), \
|
||||
mock.patch("subprocess.run", return_value=fake):
|
||||
info = ntn_files.preflight()
|
||||
assert info["workspace_name"] == "D.Intelligence"
|
||||
assert info["workspace_id"] == "ws-123"
|
||||
|
||||
|
||||
def test_upload_returns_id():
|
||||
_reset_cache()
|
||||
fake = subprocess.CompletedProcess(
|
||||
args=[], returncode=0,
|
||||
stdout="43833259-72ae-404e-8441-b6577f3159b4\tphoto.png\tuploaded\n",
|
||||
stderr="",
|
||||
)
|
||||
# upload() opens the file before subprocess.run; mock open so the file
|
||||
# need not exist (subprocess.run is mocked and never reads the handle).
|
||||
with mock.patch("builtins.open", mock.mock_open(read_data=b"x")), \
|
||||
mock.patch("subprocess.run", return_value=fake):
|
||||
upload_id = ntn_files.upload(Path("/tmp/photo.png"))
|
||||
assert upload_id == "43833259-72ae-404e-8441-b6577f3159b4"
|
||||
|
||||
|
||||
def test_upload_failure_raises():
|
||||
_reset_cache()
|
||||
fake = subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr="boom: invalid file",
|
||||
)
|
||||
with mock.patch("builtins.open", mock.mock_open(read_data=b"x")), \
|
||||
mock.patch("subprocess.run", return_value=fake):
|
||||
try:
|
||||
ntn_files.upload(Path("/tmp/bad.png"))
|
||||
assert False, "expected NtnUploadError"
|
||||
except NtnUploadError as e:
|
||||
assert e.path == "/tmp/bad.png"
|
||||
assert "boom" in e.stderr
|
||||
|
||||
|
||||
def _img(url):
|
||||
return {"object": "block", "type": "image",
|
||||
"image": {"type": "external", "external": {"url": url}}}
|
||||
|
||||
|
||||
def test_materialize_remote_untouched():
|
||||
blocks = [_img("https://ex.com/a.png")]
|
||||
out = ntn_files.materialize_local_media(
|
||||
blocks, Path("/tmp"), upload_fn=lambda p: "SHOULD_NOT_RUN")
|
||||
assert out[0]["image"]["type"] == "external"
|
||||
|
||||
|
||||
def test_materialize_local_uploaded(tmp_path=None):
|
||||
import tempfile, os
|
||||
d = Path(tempfile.mkdtemp())
|
||||
(d / "x.png").write_bytes(b"fake")
|
||||
blocks = [_img("x.png")]
|
||||
out = ntn_files.materialize_local_media(
|
||||
blocks, d, upload_fn=lambda p: "UP123")
|
||||
assert out[0]["image"]["type"] == "file_upload"
|
||||
assert out[0]["image"]["file_upload"]["id"] == "UP123"
|
||||
|
||||
|
||||
def test_materialize_nested_children():
|
||||
import tempfile
|
||||
d = Path(tempfile.mkdtemp())
|
||||
(d / "y.png").write_bytes(b"fake")
|
||||
toggle = {"object": "block", "type": "toggle",
|
||||
"toggle": {"rich_text": [], "children": [_img("y.png")]}}
|
||||
out = ntn_files.materialize_local_media(
|
||||
[toggle], d, upload_fn=lambda p: "UPNESTED")
|
||||
child = out[0]["toggle"]["children"][0]
|
||||
assert child["image"]["file_upload"]["id"] == "UPNESTED"
|
||||
|
||||
|
||||
def test_materialize_missing_file_raises():
|
||||
blocks = [_img("nope.png")]
|
||||
try:
|
||||
ntn_files.materialize_local_media(
|
||||
blocks, Path("/tmp"), upload_fn=lambda p: "x")
|
||||
assert False, "expected NtnUploadError"
|
||||
except NtnUploadError as e:
|
||||
assert "nope.png" in str(e) or "nope.png" in e.path
|
||||
|
||||
|
||||
def test_upload_passes_token_env():
|
||||
_reset_cache()
|
||||
fake = subprocess.CompletedProcess(args=[], returncode=0,
|
||||
stdout="ID123\tphoto.png\tuploaded\n", stderr="")
|
||||
with mock.patch.dict("os.environ", {"NOTION_API_KEY": "tok-abc"}, clear=False), \
|
||||
mock.patch("builtins.open", mock.mock_open(read_data=b"x")), \
|
||||
mock.patch("subprocess.run", return_value=fake) as m:
|
||||
ntn_files.upload(Path("/tmp/p.png"))
|
||||
passed_env = m.call_args.kwargs["env"]
|
||||
assert passed_env["NOTION_API_TOKEN"] == "tok-abc"
|
||||
|
||||
|
||||
def run_all():
|
||||
tests = [
|
||||
test_preflight_missing_ntn,
|
||||
test_preflight_returns_workspace,
|
||||
test_upload_returns_id,
|
||||
test_upload_failure_raises,
|
||||
test_materialize_remote_untouched,
|
||||
test_materialize_local_uploaded,
|
||||
test_materialize_nested_children,
|
||||
test_materialize_missing_file_raises,
|
||||
test_upload_passes_token_env,
|
||||
]
|
||||
for t in tests:
|
||||
print(f"\n{t.__name__}")
|
||||
t()
|
||||
print("\n" + "=" * 50)
|
||||
print(f"✅ All {len(tests)} tests passed")
|
||||
print("=" * 50)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_all()
|
||||
@@ -413,6 +413,28 @@ def test_no_literal_markers_leak():
|
||||
_assert("bold" in joined and "link" in joined, "visible words preserved")
|
||||
|
||||
|
||||
def test_image_remote_external():
|
||||
blocks = markdown_to_notion_blocks("")
|
||||
assert len(blocks) == 1
|
||||
b = blocks[0]
|
||||
assert b["type"] == "image"
|
||||
assert b["image"]["type"] == "external"
|
||||
assert b["image"]["external"]["url"] == "https://ex.com/c.png"
|
||||
assert b["image"]["caption"][0]["text"]["content"] == "a chart"
|
||||
|
||||
|
||||
def test_image_local_external_shape_preupload():
|
||||
blocks = markdown_to_notion_blocks("")
|
||||
assert len(blocks) == 1
|
||||
assert blocks[0]["image"]["external"]["url"] == "./pics/x.png"
|
||||
|
||||
|
||||
def test_image_only_when_standalone():
|
||||
# An inline bang-bracket inside prose is NOT an image block.
|
||||
blocks = markdown_to_notion_blocks("see  inline")
|
||||
assert blocks[0]["type"] == "paragraph"
|
||||
|
||||
|
||||
def run_all():
|
||||
tests = [
|
||||
test_rich_text_plain,
|
||||
@@ -447,6 +469,9 @@ def run_all():
|
||||
test_rich_text_relative_link_becomes_plain,
|
||||
test_rich_text_absolute_link_preserved,
|
||||
test_no_literal_markers_leak,
|
||||
test_image_remote_external,
|
||||
test_image_local_external_shape_preupload,
|
||||
test_image_only_when_standalone,
|
||||
]
|
||||
for t in tests:
|
||||
print(f"\n{t.__name__}")
|
||||
|
||||
@@ -15,31 +15,5 @@
|
||||
"packages",
|
||||
"disk-space"
|
||||
],
|
||||
"license": "MIT",
|
||||
"commands": [
|
||||
{
|
||||
"name": "mac-doctor",
|
||||
"description": "Full macOS system health check — runs all 5 audit modules and presents unified findings"
|
||||
},
|
||||
{
|
||||
"name": "mac-packages",
|
||||
"description": "Audit package managers (Homebrew, npm, pip, pyenv) for outdated packages and issues"
|
||||
},
|
||||
{
|
||||
"name": "mac-environment",
|
||||
"description": "Audit shell environment — PATH, symlinks, shell configs, and startup time"
|
||||
},
|
||||
{
|
||||
"name": "mac-security",
|
||||
"description": "Security posture assessment — SIP, Gatekeeper, Firewall, FileVault, SSH, ports"
|
||||
},
|
||||
{
|
||||
"name": "mac-cleanup",
|
||||
"description": "Scan and clean caches, logs, and clutter — shows sizes first, cleans only with consent"
|
||||
},
|
||||
{
|
||||
"name": "mac-resources",
|
||||
"description": "Monitor CPU, memory, disk, battery, and identify resource-hungry processes"
|
||||
}
|
||||
]
|
||||
"license": "MIT"
|
||||
}
|
||||
|
||||
@@ -18,15 +18,5 @@
|
||||
"gemini",
|
||||
"codex"
|
||||
],
|
||||
"license": "MIT",
|
||||
"commands": [
|
||||
{
|
||||
"name": "multi-agent-setup",
|
||||
"description": "Quick setup for multi-agent collaboration"
|
||||
},
|
||||
{
|
||||
"name": "setup-agents",
|
||||
"description": "Full interactive multi-agent setup"
|
||||
}
|
||||
]
|
||||
"license": "MIT"
|
||||
}
|
||||
|
||||
1335
docs/superpowers/plans/2026-06-27-notion-writer-cli-enhancements.md
Normal file
1335
docs/superpowers/plans/2026-06-27-notion-writer-cli-enhancements.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,280 @@
|
||||
# Notion Writer (Skill 32) — CLI Enhancements: File Uploads + Markdown Engine
|
||||
|
||||
> **Date**: 2026-06-27
|
||||
> **Status**: Approved (brainstorming)
|
||||
> **Scope**: Two additive features — (Tier 1) local file/image upload via the `ntn` CLI, and (Tier 2) an opt-in "markdown engine" that writes through Notion's native enhanced-markdown endpoints.
|
||||
> **Predecessor**: 2026-04-27 Extended Block Coverage (callouts/toggles/columns/mentions), CLAUDE.md v1.2.0
|
||||
> **Target version**: bump CLAUDE.md footer to v1.3.0
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
Add two capabilities to `custom-skills/32-notion-writer/code/scripts/notion_writer.py`:
|
||||
|
||||
1. **Tier 1 — Local media uploads.** Today `` is effectively broken (the inline link regex leaves a stray `!` and renders the image as a text link). Make standalone image references work: local files are uploaded via `ntn files create` and embedded as Notion `image` blocks with a `file_upload` reference; remote URLs become `external` image blocks. This fills a genuine capability gap — the parser previously had no image support at all.
|
||||
|
||||
2. **Tier 2 — Markdown engine.** Add `--engine markdown` to write content through Notion's native enhanced-markdown endpoints (`POST /v1/pages` with `markdown`, `PATCH /v1/pages/:id/markdown`) instead of the local block converter. A dialect translator converts the skill's existing authoring syntax (GitHub alerts, Pandoc columns, `@[mentions]`) into Notion-flavored markdown so a single source document works in both engines.
|
||||
|
||||
The default engine stays `blocks` — every existing call and downstream script behaves identically. Both engines share one media-upload path.
|
||||
|
||||
---
|
||||
|
||||
## Non-goals
|
||||
|
||||
- **Re-platforming onto `ntn`** (Tier 3, rejected). The skill keeps its token-based SDK core; `ntn` is used only as a file-upload subprocess.
|
||||
- **Reading pages back as markdown** (`GET /v1/pages/:id/markdown`). Not needed for a writer; parked.
|
||||
- **Workspace-mismatch hard guard.** User chose plain shell-out; preflight surfaces the `ntn` target workspace as an informational note but does not block on mismatch.
|
||||
- **Generic local attachments** (`<file>`, `<pdf>`, `<video>`, `<audio>`) in v1. The upload mechanism is media-type agnostic, but v1 wires only `![]()` images. Other media use the same walk later — parked.
|
||||
- **Legacy markdown update commands** (`insert_content` for arbitrary positions, `replace_content_range`). Only `insert_content{position:end}` (append) and `replace_content` (replace) are used.
|
||||
- **Bidirectional dialect translation** (Notion enhanced markdown → skill dialect). One direction only.
|
||||
|
||||
---
|
||||
|
||||
## Architectural decisions (locked during brainstorming)
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
|---|---|---|
|
||||
| File-upload transport | **Shell out to `ntn files create --plain`** | User choice. The CLI does the full upload lifecycle in one command. |
|
||||
| Upload-failure visibility | **Loud, not silent** — preflight checks `ntn` is installed + logged in and prints the target workspace; upload errors abort with the file path + `ntn` stderr | File uploads are workspace-scoped; a mismatch must surface, not silently misfire. |
|
||||
| Markdown dialect handling | **Translate** skill dialect → Notion enhanced markdown | User choice — one authoring document works in both engines. |
|
||||
| Engine selection | **`--engine {blocks,markdown}`, default `blocks`** | Zero regression for existing callers; opt-in to the new path. |
|
||||
| Local-media trigger | **Auto-detect** local paths in `![]()`; remote URLs left external | User choice — most ergonomic; replaces the silent-strip behavior. |
|
||||
| Parser purity | **Preserve it** — uploads happen in a post-parse block walk, never inside the parser | The 2026-04-27 design locked "parser makes no API calls"; honored by isolating impurity. |
|
||||
| Markdown transport | **`client.request(...)` low-level calls**, not typed SDK methods | The markdown endpoints have no stable typed SDK methods; `request` is version-safe and avoids SDK coupling. |
|
||||
| Notion API version | **`2026-03-11` for the markdown engine**; blocks engine keeps the SDK default (`2025-09-03`) | Markdown endpoints require the newer version; blocks path stays untouched. |
|
||||
| Python env | **Create venv + `requirements.txt`** at the documented path | The documented venv doesn't exist; the skill can't currently run. |
|
||||
|
||||
---
|
||||
|
||||
## Module structure
|
||||
|
||||
Two new modules keep `notion_writer.py` from bloating; each has one purpose and is testable in isolation.
|
||||
|
||||
| File | Status | Role |
|
||||
|---|---|---|
|
||||
| `scripts/notion_writer.py` | modified | CLI surface, engine routing, post-parse media materialization, two-phase markdown writes |
|
||||
| `scripts/_notion_compat.py` | modified | Version-aware `make_client`; markdown-endpoint helpers (`create_page_markdown`, `append_markdown`, `replace_markdown`) |
|
||||
| `scripts/ntn_files.py` | **new** | `ntn files create` subprocess wrapper; preflight; `upload(path) -> file_upload_id` |
|
||||
| `scripts/md_translate.py` | **new** | Pure functions: skill dialect → Notion enhanced markdown |
|
||||
| `scripts/requirements.txt` | modified | Pin `notion-client`, `python-dotenv` |
|
||||
| `scripts/venv/` | **new** | Created at documented path; not committed (gitignored) |
|
||||
|
||||
Dependencies between units:
|
||||
- `notion_writer` depends on `_notion_compat`, `ntn_files`, `md_translate`.
|
||||
- `md_translate` and the parser are pure (deterministic, no I/O).
|
||||
- `ntn_files` owns all subprocess I/O.
|
||||
- `_notion_compat` owns all Notion HTTP I/O.
|
||||
|
||||
---
|
||||
|
||||
## Tier 1 — Local media uploads (blocks engine)
|
||||
|
||||
### Parsing (pure)
|
||||
Add an image detector to `_parse_lines`, ordered with the other block detectors. A **standalone** image line matches:
|
||||
|
||||
```python
|
||||
IMAGE_RE = re.compile(r'^\s*!\[([^\]]*)\]\(([^)\s]+)\)\s*$')
|
||||
```
|
||||
|
||||
`create_image_block(alt, target)` always emits the **external shape** regardless of whether `target` is local or remote:
|
||||
|
||||
```python
|
||||
{"object":"block","type":"image",
|
||||
"image":{"type":"external","external":{"url": target},
|
||||
"caption": parse_rich_text(alt)}}
|
||||
```
|
||||
|
||||
The parser stays pure and upload-unaware. Inline images inside a paragraph are out of scope (rare); only standalone image lines become image blocks.
|
||||
|
||||
### Materialization (impure, isolated)
|
||||
After parsing and before sending, `notion_writer` runs a recursive block walk:
|
||||
|
||||
```python
|
||||
materialize_local_media(blocks, base_dir, uploader) -> blocks
|
||||
```
|
||||
|
||||
- Walks `blocks` and every nested `children` list (so images inside toggles/columns are covered).
|
||||
- For each `image` block whose `external.url` is **not** `http(s):` and resolves to an existing file under `base_dir`:
|
||||
- `file_upload_id = uploader.upload(resolved_path)`
|
||||
- rewrite to `{"image":{"type":"file_upload","file_upload":{"id": file_upload_id}, "caption": ...}}`
|
||||
- Remote/external images are left unchanged.
|
||||
- A local path that does not exist on disk → abort with a clear error (`image references missing file: <path>`), since the user clearly intended a local embed.
|
||||
|
||||
`base_dir` = the `--file` argument's parent directory; for `--stdin`, the current working directory.
|
||||
|
||||
### Upload wrapper — `ntn_files.py`
|
||||
|
||||
```python
|
||||
def preflight() -> WorkspaceInfo # cached; once per process
|
||||
def upload(path: Path) -> str # returns file_upload_id
|
||||
```
|
||||
|
||||
- `preflight()`:
|
||||
- `shutil.which("ntn")` → if missing, raise with install hint (`curl -fsSL https://ntn.dev | bash`).
|
||||
- `ntn api v1/users/me` (JSON) → parse `workspace_name` / `workspace_id`; print one informational line: `ntn → workspace "<name>"`. Does not block on mismatch (per locked decision). If the call fails (not logged in), raise with `ntn login` hint.
|
||||
- `upload(path)`:
|
||||
- `subprocess.run(["ntn","files","create","--plain"], stdin=open(path,"rb"), capture_output=True, text=True)`
|
||||
- on non-zero exit → raise `NtnUploadError(path, stderr)`.
|
||||
- parse stdout: the upload ID is the first tab-separated field of the first line.
|
||||
- returns the ID. (Notion requires attaching within ~1 hour; the immediate block append in the same run is well inside that window.)
|
||||
|
||||
---
|
||||
|
||||
## Tier 2 — Markdown engine
|
||||
|
||||
### Dialect translator — `md_translate.py`
|
||||
|
||||
`translate(content: str) -> str` converts the skill dialect to Notion enhanced markdown. Only three constructs differ; everything else passes through verbatim.
|
||||
|
||||
| Skill dialect | → Notion enhanced markdown |
|
||||
|---|---|
|
||||
| `> [!NOTE]` + contiguous `>` body | `<callout icon="ℹ️" color="blue_bg">\n\t<body>\n</callout>` |
|
||||
| `::: columns` / `::: column` / `:::` | `<columns>\n\t<column>…</column>\n\t<column>…</column>\n</columns>` |
|
||||
| `<details>` / `<summary>` toggle | `<details>\n<summary>…</summary>\n\t<children>\n</details>` — children **re-indented one tab** |
|
||||
| `@[Title](id\|url)` | `<mention-page url="<resolved>">Title</mention-page>` |
|
||||
| headings, lists, to-dos, quotes, fenced code, `---`, **bold**/*italic*/`code`/`~~strike~~`/links, pipe tables | **pass through unchanged** |
|
||||
|
||||
Notes:
|
||||
- Callout icon/color reuse the existing `ALERT_TYPES` map; Notion color names use the `_bg` suffix (`blue_bg`), not the block-API `blue_background`. A small adapter maps one to the other.
|
||||
- Children inside `<callout>`, `<columns>`, and `<details>` are **tab-indented** one level (enhanced-markdown requirement — toggle and callout children "must be indented"). The translator emits tabs; this is why `<details>` is translated rather than passed through verbatim.
|
||||
- Pipe tables pass through — Notion's own enhanced-markdown example renders a standard pipe table, so no `<table>` rewrite is needed in v1. (If live testing shows pipe tables are rejected, the fallback is a `<table>` rewrite — noted as the one open verification item.)
|
||||
- Mention resolution reuses `extract_notion_id`; an unresolvable target degrades to plain `@Title` text (same posture as the blocks engine).
|
||||
- The translator is line-oriented and reentrant for the two container constructs, mirroring `_parse_lines`.
|
||||
|
||||
### Transport helpers — `_notion_compat.py`
|
||||
|
||||
All via `client.request(...)` so no SDK typed-method dependency:
|
||||
|
||||
```python
|
||||
def make_client(api_key, notion_version=None) -> Client # version override added
|
||||
def create_page_markdown(client, parent, properties, markdown) -> dict
|
||||
# POST v1/pages body={parent, properties, markdown}
|
||||
def append_markdown(client, page_id, markdown) -> dict
|
||||
# PATCH v1/pages/{id}/markdown
|
||||
# body={"type":"insert_content","insert_content":{"content":md,"position":{"type":"end"}}}
|
||||
def replace_markdown(client, page_id, markdown, allow_deleting=False) -> dict
|
||||
# PATCH v1/pages/{id}/markdown
|
||||
# body={"type":"replace_content","replace_content":{"new_str":md,"allow_deleting_content":allow_deleting}}
|
||||
```
|
||||
|
||||
**Two-client split for version safety.** The markdown *write* helpers use a client built with `notion_version="2026-03-11"`. All schema/property/upsert-lookup operations (`resolve_data_source_id`, `get_schema`, `coerce_properties`, `find_existing_page`) continue on the **default-version** client, so the 2025-09-03 data-source behavior the skill already relies on is unchanged. Only the three markdown write calls cross to the newer version. The data-source parent (`{type:data_source_id,...}`) is expected to remain valid under `2026-03-11` (confirmed in the live smoke test).
|
||||
|
||||
### Routing by operation
|
||||
|
||||
| CLI op | blocks engine (default) | markdown engine |
|
||||
|---|---|---|
|
||||
| Create DB row (`-d -t`) | `pages.create(parent, properties, children=blocks)` | `create_page_markdown(parent, properties, translate(content))` |
|
||||
| Append to page (`-p`) | `blocks.children.append` | `append_markdown(page_id, translate(content))` |
|
||||
| Replace page (`-p -r`) | delete-all-blocks + append | `replace_markdown(page_id, translate(content))` |
|
||||
| Upsert (`--upsert-by`) | unchanged (property update + body) | property update via SDK + body via `replace_markdown` |
|
||||
|
||||
The markdown `replace_content` path replaces the brittle paginated delete-every-block logic for that engine. `--upsert-by` lookup/property-coercion logic is shared unchanged; only the body write differs.
|
||||
|
||||
### Markdown engine + local images (two-phase write)
|
||||
|
||||
The markdown endpoints take a URL for ``, and a `file_upload` id is not a URL, so local images cannot be inlined into the markdown string. Handling, reusing the Tier 1 upload + append helpers:
|
||||
|
||||
1. Split standalone **local** image refs out of the content (same `IMAGE_RE`, local targets only; remote `` stay inline in the markdown); translate the remainder; write it (create / append / replace).
|
||||
2. Upload the local images and append them as `image` blocks to the resulting page via `blocks.children.append`.
|
||||
|
||||
**Known limitation (documented):** in the markdown engine, local images land at the **end** of the page, not their original position. When image position matters, use the blocks engine. Remote images keep their position (they stay inline in the markdown).
|
||||
|
||||
---
|
||||
|
||||
## Flags & versioning
|
||||
|
||||
New CLI flags:
|
||||
|
||||
| Flag | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `--engine {blocks,markdown}` | `blocks` | Select the write path |
|
||||
| `--notion-version VERSION` | engine-dependent | Override the API version (markdown defaults to `2026-03-11`) |
|
||||
| `--allow-deleting-content` | off | Permit markdown `replace_content` to delete child pages/databases |
|
||||
|
||||
Validation: `--engine markdown` with `--upsert-by` on a property the markdown path can't update falls back to the shared coercion logic (no new restriction). `--allow-deleting-content` is ignored by the blocks engine (warn if combined).
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
Permissive where the user's intent is ambiguous; loud where they clearly intended an action that failed.
|
||||
|
||||
| Failure | Behavior |
|
||||
|---|---|
|
||||
| `ntn` not installed | Abort before any write; install hint |
|
||||
| `ntn` not logged in (`users/me` fails) | Abort; `ntn login` hint |
|
||||
| `ntn files create` non-zero exit | Abort; print file path + `ntn` stderr |
|
||||
| Local image path missing on disk | Abort; `image references missing file: <path>` |
|
||||
| Markdown `replace`/selection no match | Route `validation_error` through `explain_api_error` with a markdown-specific hint |
|
||||
| `replace_content` would delete child pages | Surface the API's affected-items list; suggest `--allow-deleting-content` |
|
||||
| Synced-page update rejected | Clear message (synced pages can't be updated) |
|
||||
| Unresolvable `@[mention]` | Degrade to plain `@Title` (both engines) |
|
||||
| Two-phase image append fails after text write | Warn that text was written but images were not; non-zero exit |
|
||||
|
||||
`explain_api_error` gains markdown-endpoint cases; all other paths reuse existing handling.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
TDD. New unit suites alongside the existing `test_parser.py` (28 tests, must stay green).
|
||||
|
||||
| Suite | Verifies |
|
||||
|---|---|
|
||||
| `test_parser.py` (extended) | `` → external image block; `` → external-shape with local URL pre-materialization; non-image `!` text unaffected; existing 28 stay green |
|
||||
| `test_md_translate.py` (new) | callout (each alert type → icon+`_bg` color, tab-indented body); columns → `<columns>/<column>`; mention id+url+invalid; passthrough of headings/lists/code/tables/`<details>`; escaping |
|
||||
| `test_ntn_files.py` (new) | `subprocess.run` mocked: command shape, `--plain` first-field parse, non-zero → `NtnUploadError`, preflight missing-`ntn`, preflight not-logged-in; local-vs-remote branch in `materialize_local_media` |
|
||||
| `test_engine_routing.py` (new) | client `request` mocked: each op×engine calls the right path/method/body; markdown client built with `2026-03-11`; two-phase image append fires after text write |
|
||||
|
||||
**Live smoke test** (manual, end of implementation) against the "Working with AI" data source `f8f19ede-32bd-43ac-9f60-0651f6f40afe`, both engines:
|
||||
1. `--test` connection.
|
||||
2. Blocks engine: create a row with a local image → confirm `file_upload` image block renders.
|
||||
3. Markdown engine: create a row from a doc using `[!NOTE]` + `:::columns` → confirm callout + columns render; confirm a local image appended at end.
|
||||
4. Markdown engine `--replace` on the same page → confirm `replace_content` works.
|
||||
|
||||
This verifies the one open item (pipe-table passthrough) and the data-source parent under `2026-03-11`.
|
||||
|
||||
---
|
||||
|
||||
## File changes
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `scripts/notion_writer.py` | `--engine`/`--notion-version`/`--allow-deleting-content` flags; image detector + `create_image_block`; `materialize_local_media` walk; markdown-engine routing; two-phase image append |
|
||||
| `scripts/_notion_compat.py` | `make_client(notion_version=…)`; `create_page_markdown`/`append_markdown`/`replace_markdown`; markdown cases in `explain_api_error` |
|
||||
| `scripts/ntn_files.py` | **new** — preflight + `upload` |
|
||||
| `scripts/md_translate.py` | **new** — `translate` + construct translators |
|
||||
| `scripts/test_md_translate.py`, `scripts/test_ntn_files.py`, `scripts/test_engine_routing.py` | **new** test suites |
|
||||
| `scripts/test_parser.py` | image tests |
|
||||
| `scripts/requirements.txt` | pin deps |
|
||||
| `code/CLAUDE.md`, `SKILL.md` | document engines, file uploads, flags, the markdown-engine image limitation; v1.3.0 changelog |
|
||||
| `.gitignore` | ensure `venv/` ignored |
|
||||
|
||||
---
|
||||
|
||||
## Out of scope (parking lot)
|
||||
|
||||
- Generic local `<file>`/`<pdf>`/`<video>`/`<audio>` upload (mechanism ready; not wired in v1).
|
||||
- `GET /v1/pages/:id/markdown` round-trip read.
|
||||
- Workspace-mismatch hard guard (escalate preflight from warn → block) if it proves necessary in practice.
|
||||
- Notion enhanced markdown → skill dialect (reverse translation), which would also serve the parked Phase 3b Notion-as-RAG export.
|
||||
- `<table>` rewrite for pipe tables — only if live testing shows pipe tables are rejected by the markdown endpoint.
|
||||
- Inline (non-standalone) images.
|
||||
|
||||
---
|
||||
|
||||
## Implementation transition
|
||||
|
||||
After user approval of this spec, invoke `superpowers:writing-plans`. The plan will likely sequence:
|
||||
|
||||
1. Create venv + `requirements.txt`; confirm existing 28 tests run green (baseline).
|
||||
2. `ntn_files.py` + `test_ntn_files.py` (subprocess mocked).
|
||||
3. Image detector + `create_image_block` + `materialize_local_media` + parser image tests (blocks engine Tier 1 complete; live single-image check).
|
||||
4. `md_translate.py` + `test_md_translate.py` (pure, no API).
|
||||
5. `_notion_compat` markdown helpers + version-aware client + `test_engine_routing.py`.
|
||||
6. Wire `--engine`/flags + routing + two-phase image append in `notion_writer.py`.
|
||||
7. Docs (CLAUDE.md/SKILL.md) + v1.3.0 bump.
|
||||
8. Live smoke test (both engines) against the "Working with AI" DB; resolve the pipe-table verification item.
|
||||
|
||||
Each step is independently testable and revertable.
|
||||
Reference in New Issue
Block a user