Compare commits
11 Commits
144a17c88d
...
6446f00522
| Author | SHA1 | Date | |
|---|---|---|---|
| 6446f00522 | |||
| 707405b940 | |||
| 328de7d052 | |||
| 0421a65327 | |||
| 0f8d7b2df1 | |||
| aac1082bb7 | |||
| 4e692d0e9a | |||
| e0f93e6add | |||
| c318df8551 | |||
| e2ae8aad94 | |||
| c66b5e12cc |
@@ -157,8 +157,12 @@ When upserting, `--file` (or `--stdin`) replaces the page's body blocks; `--prop
|
||||
| `- [ ] task` | To-do (unchecked) |
|
||||
| `- [x] task` | To-do (checked) |
|
||||
| `> quote` | Quote |
|
||||
| `> [!NOTE]` / `[!TIP]` / `[!IMPORTANT]` / `[!WARNING]` / `[!CAUTION]` | Callout (with icon + colored background) |
|
||||
| `<details><summary>X</summary> ... </details>` | Toggle (multi-line form, recursive children) |
|
||||
| `::: columns / ::: column / :::` | Column list (Pandoc fenced div, ≥2 columns required) |
|
||||
| `` ```code``` `` | Code block |
|
||||
| `---` | Divider |
|
||||
| Tables (`\| col \|`) | Table |
|
||||
| Paragraphs | Paragraph |
|
||||
|
||||
### Code Block Languages
|
||||
@@ -181,6 +185,7 @@ print("Hello")
|
||||
| `[text](https://...)` | link (absolute URLs only) |
|
||||
| `[text](#anchor)` | bold (Notion rejects fragment URLs) |
|
||||
| `[text](relative/path.md)` | plain text (Notion rejects relative URLs) |
|
||||
| `@[Title](page-id-or-notion-url)` | page mention (resolves via Notion URL or raw 32-hex ID) |
|
||||
|
||||
Notion's URL validator requires absolute URLs for link annotations. The parser converts TOC-style anchor links to bold to preserve navigation intent and silently strips relative paths.
|
||||
|
||||
@@ -205,6 +210,65 @@ python notion_writer.py \
|
||||
--file meeting_notes.md
|
||||
```
|
||||
|
||||
### Callouts (GitHub alerts)
|
||||
|
||||
````markdown
|
||||
> [!NOTE]
|
||||
> Just FYI: this method is idempotent.
|
||||
|
||||
> [!WARNING]
|
||||
> Don't run this in production without a backup.
|
||||
````
|
||||
|
||||
Renders as Notion callout blocks with corresponding emoji icon and colored background.
|
||||
|
||||
### Toggles (HTML5 `<details>`)
|
||||
|
||||
````markdown
|
||||
<details>
|
||||
<summary>Click to expand: full debug log</summary>
|
||||
|
||||
```bash
|
||||
$ python notion_writer.py --test
|
||||
✅ Connected
|
||||
```
|
||||
|
||||
Lists, code blocks, and even nested `<details>` work inside.
|
||||
</details>
|
||||
````
|
||||
|
||||
Multi-line form required: `<details>` and `</details>` must be on their own lines.
|
||||
|
||||
### Columns (Pandoc fenced div)
|
||||
|
||||
````markdown
|
||||
::: columns
|
||||
::: column
|
||||
**Column 1**
|
||||
|
||||
- item a
|
||||
- item b
|
||||
:::
|
||||
::: column
|
||||
**Column 2**
|
||||
|
||||
```python
|
||||
print("hello")
|
||||
```
|
||||
:::
|
||||
:::
|
||||
````
|
||||
|
||||
≥2 columns required by Notion. Single-column blocks degrade to plain paragraphs.
|
||||
|
||||
### Page mentions
|
||||
|
||||
```markdown
|
||||
See @[Architecture Decision Record](https://notion.so/ADR-abcdef0123456789abcdef0123456789) for context.
|
||||
```
|
||||
|
||||
Both Notion URLs and raw 32-hex IDs work. Invalid targets fall back to plain text `@Title`.
|
||||
|
||||
### Pipe from Another Tool
|
||||
|
||||
```bash
|
||||
@@ -334,8 +398,9 @@ python notion_writer.py -d DB_URL -t "Title" --upsert-by "Name" -f content.md
|
||||
|
||||
---
|
||||
|
||||
*Version 1.1.0 | Claude Code | 2026-04-27*
|
||||
*Version 1.2.0 | Claude Code | 2026-04-27*
|
||||
|
||||
Changelog:
|
||||
- 1.2.0 — Extended block coverage: GitHub-alert callouts, HTML5 `<details>` toggles, Pandoc `::: columns` fenced div, inline `@[Title](id-or-url)` page mentions. Parser made reentrant to support full recursion inside container blocks.
|
||||
- 1.1.0 — Migrated to Notion API 2025-09-03 (multi-source databases). Added `--properties` JSON flag, `--upsert-by` for idempotency, anchor-link parser fix, friendlier API error messages.
|
||||
- 1.0.0 — Initial release with markdown→Notion block conversion.
|
||||
|
||||
@@ -10,7 +10,7 @@ import re
|
||||
import json
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
from typing import Optional, List, Dict, Any
|
||||
from typing import Optional, List, Dict, Any, Union
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from notion_client import Client
|
||||
@@ -34,9 +34,9 @@ def extract_notion_id(url_or_id: str) -> Optional[str]:
|
||||
# Notion URL patterns
|
||||
patterns = [
|
||||
r'notion\.so/(?:[^/]+/)?([a-f0-9]{32})', # notion.so/workspace/page-id
|
||||
r'notion\.so/(?:[^/]+/)?[^-]+-([a-f0-9]{32})', # notion.so/workspace/Page-Title-id
|
||||
r'notion\.so/(?:[^/]+/)?[^/]+?-([a-f0-9]{32})', # notion.so/workspace/Page-Title-id
|
||||
r'notion\.site/(?:[^/]+/)?([a-f0-9]{32})', # public notion.site
|
||||
r'notion\.site/(?:[^/]+/)?[^-]+-([a-f0-9]{32})',
|
||||
r'notion\.site/(?:[^/]+/)?[^/]+?-([a-f0-9]{32})',
|
||||
r'([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})', # UUID format
|
||||
]
|
||||
|
||||
@@ -72,32 +72,153 @@ def _split_table_row(line: str) -> List[str]:
|
||||
return [cell.strip() for cell in stripped.split('|')]
|
||||
|
||||
|
||||
def markdown_to_notion_blocks(markdown_text: str) -> List[Dict[str, Any]]:
|
||||
"""Convert markdown text to Notion block objects."""
|
||||
blocks = []
|
||||
lines = markdown_text.split('\n')
|
||||
i = 0
|
||||
def markdown_to_notion_blocks(content: Union[str, List[str]]) -> List[Dict[str, Any]]:
|
||||
"""Convert markdown text (or pre-split lines) to Notion block objects.
|
||||
|
||||
Reentrant: container detectors recursively call _parse_lines on inner content.
|
||||
"""
|
||||
if isinstance(content, str):
|
||||
lines = content.split('\n')
|
||||
else:
|
||||
lines = list(content)
|
||||
return _parse_lines(lines)
|
||||
|
||||
|
||||
def _parse_lines(lines: List[str]) -> List[Dict[str, Any]]:
|
||||
"""Walk lines and emit Notion blocks. Called recursively by container detectors."""
|
||||
blocks = []
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
line = lines[i]
|
||||
|
||||
# Skip empty lines
|
||||
if not line.strip():
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Table (header row + separator + body rows)
|
||||
# Toggle (HTML5 <details>) — multi-line form, depth-tracked for nesting
|
||||
if line.strip() == '<details>':
|
||||
i += 1
|
||||
# Optional <summary>...</summary> on next non-blank line
|
||||
summary_text = ''
|
||||
while i < len(lines) and not lines[i].strip():
|
||||
i += 1
|
||||
if i < len(lines):
|
||||
summary_match = re.match(r'^\s*<summary>(.*)</summary>\s*$', lines[i])
|
||||
if summary_match:
|
||||
summary_text = summary_match.group(1)
|
||||
i += 1
|
||||
# Collect body lines until matching </details>, depth-tracked
|
||||
inner_lines: List[str] = []
|
||||
depth = 1
|
||||
while i < len(lines) and depth > 0:
|
||||
stripped_inner = lines[i].strip()
|
||||
if stripped_inner == '<details>':
|
||||
depth += 1
|
||||
inner_lines.append(lines[i])
|
||||
elif stripped_inner == '</details>':
|
||||
depth -= 1
|
||||
if depth > 0:
|
||||
inner_lines.append(lines[i])
|
||||
else:
|
||||
inner_lines.append(lines[i])
|
||||
i += 1
|
||||
if depth > 0:
|
||||
# Unclosed <details> at EOF — degrade to paragraphs
|
||||
print("Warning: unclosed <details> at EOF; emitting body as paragraphs",
|
||||
file=sys.stderr)
|
||||
if summary_text:
|
||||
blocks.append(create_paragraph_block(summary_text))
|
||||
for inner in inner_lines:
|
||||
if inner.strip():
|
||||
blocks.append(create_paragraph_block(inner))
|
||||
continue
|
||||
children = _parse_lines(inner_lines)
|
||||
blocks.append(create_toggle_block(summary_text, children))
|
||||
continue
|
||||
|
||||
# Columns (Pandoc fenced div ::: columns)
|
||||
if line.strip() == '::: columns':
|
||||
columns_lines: List[List[str]] = []
|
||||
current_col: Optional[List[str]] = None
|
||||
i += 1
|
||||
depth = 1 # we are inside our own wrapper
|
||||
while i < len(lines) and depth > 0:
|
||||
stripped_col = lines[i].strip()
|
||||
if stripped_col == '::: columns':
|
||||
depth += 1
|
||||
if current_col is not None:
|
||||
current_col.append(lines[i])
|
||||
i += 1
|
||||
continue
|
||||
if stripped_col == '::: column':
|
||||
depth += 1
|
||||
if depth == 2:
|
||||
# Top-level column inside our wrapper
|
||||
if current_col is not None:
|
||||
columns_lines.append(current_col)
|
||||
current_col = []
|
||||
else:
|
||||
# Column inside a nested wrapper — record verbatim
|
||||
if current_col is not None:
|
||||
current_col.append(lines[i])
|
||||
i += 1
|
||||
continue
|
||||
if stripped_col == ':::':
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
# Closing our wrapper
|
||||
if current_col is not None:
|
||||
columns_lines.append(current_col)
|
||||
current_col = None
|
||||
i += 1
|
||||
break
|
||||
if depth == 1:
|
||||
# Closing a top-level column
|
||||
if current_col is not None:
|
||||
columns_lines.append(current_col)
|
||||
current_col = None
|
||||
else:
|
||||
# Closing something nested — record verbatim
|
||||
if current_col is not None:
|
||||
current_col.append(lines[i])
|
||||
i += 1
|
||||
continue
|
||||
# Regular content line — append to current column if open
|
||||
if current_col is not None:
|
||||
current_col.append(lines[i])
|
||||
i += 1
|
||||
if depth > 0:
|
||||
# Unclosed wrapper at EOF — degrade to paragraphs
|
||||
print("Warning: unclosed ::: columns at EOF; emitting as paragraphs",
|
||||
file=sys.stderr)
|
||||
if current_col is not None:
|
||||
columns_lines.append(current_col)
|
||||
for col in columns_lines:
|
||||
for inner in col:
|
||||
if inner.strip():
|
||||
blocks.append(create_paragraph_block(inner))
|
||||
continue
|
||||
# Drop columns that are pure whitespace
|
||||
columns_lines = [c for c in columns_lines if any(li.strip() for li in c)]
|
||||
if len(columns_lines) < 2:
|
||||
# Notion requires >= 2 columns; single column degrades to paragraphs
|
||||
for col in columns_lines:
|
||||
for inner in col:
|
||||
if inner.strip():
|
||||
blocks.append(create_paragraph_block(inner))
|
||||
continue
|
||||
column_blocks = [_parse_lines(col_lines) for col_lines in columns_lines]
|
||||
blocks.append(create_column_list_block(column_blocks))
|
||||
continue
|
||||
|
||||
if _is_table_row(line) and i + 1 < len(lines) and TABLE_SEPARATOR_RE.match(lines[i + 1]):
|
||||
header_cells = _split_table_row(line)
|
||||
i += 2 # skip separator
|
||||
i += 2
|
||||
body_rows: List[List[str]] = []
|
||||
while i < len(lines) and _is_table_row(lines[i]):
|
||||
body_rows.append(_split_table_row(lines[i]))
|
||||
i += 1
|
||||
blocks.append(create_table_block(header_cells, body_rows))
|
||||
continue
|
||||
|
||||
# Headers
|
||||
if line.startswith('######'):
|
||||
blocks.append(create_heading_block(line[6:].strip(), 3))
|
||||
elif line.startswith('#####'):
|
||||
@@ -110,8 +231,6 @@ def markdown_to_notion_blocks(markdown_text: str) -> List[Dict[str, Any]]:
|
||||
blocks.append(create_heading_block(line[2:].strip(), 2))
|
||||
elif line.startswith('#'):
|
||||
blocks.append(create_heading_block(line[1:].strip(), 1))
|
||||
|
||||
# Code blocks
|
||||
elif line.startswith('```'):
|
||||
language = line[3:].strip() or 'plain text'
|
||||
code_lines = []
|
||||
@@ -120,7 +239,6 @@ def markdown_to_notion_blocks(markdown_text: str) -> List[Dict[str, Any]]:
|
||||
code_lines.append(lines[i])
|
||||
i += 1
|
||||
blocks.append(create_code_block('\n'.join(code_lines), language))
|
||||
|
||||
# Checkbox / Todo (must come before generic bullet match)
|
||||
elif line.strip().startswith('- [ ]'):
|
||||
text = line.strip()[5:].strip()
|
||||
@@ -128,37 +246,38 @@ def markdown_to_notion_blocks(markdown_text: str) -> List[Dict[str, Any]]:
|
||||
elif line.strip().startswith('- [x]') or line.strip().startswith('- [X]'):
|
||||
text = line.strip()[5:].strip()
|
||||
blocks.append(create_todo_block(text, True))
|
||||
|
||||
# Bullet list
|
||||
elif line.strip().startswith('- ') or line.strip().startswith('* '):
|
||||
text = line.strip()[2:]
|
||||
blocks.append(create_bulleted_list_block(text))
|
||||
|
||||
# Numbered list
|
||||
elif re.match(r'^\d+\.\s', line.strip()):
|
||||
text = re.sub(r'^\d+\.\s', '', line.strip())
|
||||
blocks.append(create_numbered_list_block(text))
|
||||
|
||||
# Blockquote
|
||||
# Blockquote — GitHub-alert callout takes priority over generic quote
|
||||
elif line.startswith('>'):
|
||||
text = line[1:].strip()
|
||||
blocks.append(create_quote_block(text))
|
||||
|
||||
# Horizontal rule
|
||||
first_body = line[1:].strip()
|
||||
alert_match = ALERT_RE.match(first_body)
|
||||
if alert_match:
|
||||
alert_type = alert_match.group(1)
|
||||
body_lines: List[str] = []
|
||||
i += 1
|
||||
while i < len(lines) and lines[i].lstrip().startswith('>'):
|
||||
body_lines.append(lines[i].lstrip()[1:].lstrip())
|
||||
i += 1
|
||||
body_text = '\n'.join(body_lines)
|
||||
blocks.append(create_callout_block(body_text, alert_type))
|
||||
continue
|
||||
blocks.append(create_quote_block(first_body))
|
||||
elif line.strip() in ['---', '***', '___']:
|
||||
blocks.append(create_divider_block())
|
||||
|
||||
# Regular paragraph
|
||||
else:
|
||||
blocks.append(create_paragraph_block(line))
|
||||
|
||||
i += 1
|
||||
|
||||
return blocks
|
||||
|
||||
|
||||
INLINE_PATTERNS = [
|
||||
('code', re.compile(r'`([^`\n]+)`')),
|
||||
('mention', re.compile(r'@\[([^\]]+)\]\(([^)\s]+)\)')), # must precede 'link' so @ takes priority
|
||||
('link', re.compile(r'\[([^\]]+)\]\(([^)\s]+)\)')),
|
||||
('bold_star', re.compile(r'\*\*([^*\n]+)\*\*')),
|
||||
('bold_under', re.compile(r'__([^_\n]+)__')),
|
||||
@@ -167,6 +286,15 @@ INLINE_PATTERNS = [
|
||||
('italic_under', re.compile(r'(?<![a-zA-Z0-9_])_([^_\n]+)_(?![a-zA-Z0-9_])')),
|
||||
]
|
||||
|
||||
ALERT_TYPES = {
|
||||
'NOTE': ('ℹ️', 'blue_background'),
|
||||
'TIP': ('💡', 'green_background'),
|
||||
'IMPORTANT': ('☝️', 'purple_background'),
|
||||
'WARNING': ('⚠️', 'yellow_background'),
|
||||
'CAUTION': ('🚨', 'red_background'),
|
||||
}
|
||||
ALERT_RE = re.compile(r'^\s*\[!(NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]\s*$')
|
||||
|
||||
|
||||
_ABSOLUTE_URL_RE = re.compile(r'^(?:[a-zA-Z][a-zA-Z0-9+.-]*:)')
|
||||
|
||||
@@ -220,6 +348,22 @@ def parse_rich_text(text: str) -> List[Dict[str, Any]]:
|
||||
|
||||
if kind == 'code':
|
||||
spans.append(_rich_span(m.group(1), code=True))
|
||||
elif kind == 'mention':
|
||||
mention_title, mention_target = m.group(1), m.group(2)
|
||||
page_id = extract_notion_id(mention_target)
|
||||
if page_id:
|
||||
# Notion mention API requires dashed UUID; extract_notion_id returns dashless.
|
||||
spans.append({
|
||||
"type": "mention",
|
||||
"mention": {
|
||||
"type": "page",
|
||||
"page": {"id": format_id_with_dashes(page_id)},
|
||||
},
|
||||
"plain_text": mention_title,
|
||||
})
|
||||
else:
|
||||
# Invalid ID — degrade to plain text "@Title"
|
||||
spans.append(_rich_span(f"@{mention_title}"))
|
||||
elif kind == 'link':
|
||||
link_text, link_url = m.group(1), m.group(2)
|
||||
if _is_absolute_url(link_url):
|
||||
@@ -304,6 +448,46 @@ def create_quote_block(text: str) -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def create_callout_block(text: str, alert_type: str) -> Dict[str, Any]:
|
||||
"""Create a callout block with icon + color from a GitHub alert type."""
|
||||
icon, color = ALERT_TYPES[alert_type]
|
||||
return {
|
||||
"object": "block",
|
||||
"type": "callout",
|
||||
"callout": {
|
||||
"rich_text": parse_rich_text(text),
|
||||
"icon": {"type": "emoji", "emoji": icon},
|
||||
"color": color,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def create_toggle_block(summary_text: str, children_blocks: List[Dict[str, Any]]) -> Dict[str, Any]:
|
||||
"""Create a toggle block with summary rich-text and child blocks."""
|
||||
return {
|
||||
"object": "block",
|
||||
"type": "toggle",
|
||||
"toggle": {
|
||||
"rich_text": parse_rich_text(summary_text),
|
||||
"children": children_blocks,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def create_column_list_block(columns: List[List[Dict[str, Any]]]) -> Dict[str, Any]:
|
||||
"""Create a column_list block; each item in `columns` is the children-list for one column."""
|
||||
return {
|
||||
"object": "block",
|
||||
"type": "column_list",
|
||||
"column_list": {
|
||||
"children": [
|
||||
{"object": "block", "type": "column", "column": {"children": col_blocks}}
|
||||
for col_blocks in columns
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def create_code_block(code: str, language: str) -> Dict[str, Any]:
|
||||
return {
|
||||
"object": "block",
|
||||
|
||||
@@ -165,6 +165,219 @@ def test_blocks_table():
|
||||
"link inline formatting preserved inside table cell")
|
||||
|
||||
|
||||
def test_callout_note():
|
||||
md = "> [!NOTE]\n> This is a note.\n> Spans multiple lines."
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(len(blocks) == 1, "exactly one block emitted")
|
||||
_assert(blocks[0]["type"] == "callout", "block type is callout")
|
||||
callout = blocks[0]["callout"]
|
||||
_assert(callout["icon"] == {"type": "emoji", "emoji": "ℹ️"}, "NOTE icon is ℹ️")
|
||||
_assert(callout["color"] == "blue_background", "NOTE color is blue_background")
|
||||
body_text = "".join(s["text"]["content"] for s in callout["rich_text"])
|
||||
_assert("This is a note." in body_text, "body line 1 preserved")
|
||||
_assert("Spans multiple lines." in body_text, "body line 2 preserved")
|
||||
|
||||
|
||||
def test_callout_tip():
|
||||
blocks = markdown_to_notion_blocks("> [!TIP]\n> Use this trick.")
|
||||
_assert(blocks[0]["type"] == "callout", "TIP block is callout")
|
||||
_assert(blocks[0]["callout"]["icon"]["emoji"] == "💡", "TIP icon is 💡")
|
||||
_assert(blocks[0]["callout"]["color"] == "green_background", "TIP color is green")
|
||||
|
||||
|
||||
def test_callout_important():
|
||||
blocks = markdown_to_notion_blocks("> [!IMPORTANT]\n> Read this.")
|
||||
_assert(blocks[0]["callout"]["icon"]["emoji"] == "☝️", "IMPORTANT icon is ☝️")
|
||||
_assert(blocks[0]["callout"]["color"] == "purple_background", "IMPORTANT color is purple")
|
||||
|
||||
|
||||
def test_callout_warning():
|
||||
blocks = markdown_to_notion_blocks("> [!WARNING]\n> Be careful.")
|
||||
_assert(blocks[0]["callout"]["icon"]["emoji"] == "⚠️", "WARNING icon is ⚠️")
|
||||
_assert(blocks[0]["callout"]["color"] == "yellow_background", "WARNING color is yellow")
|
||||
|
||||
|
||||
def test_callout_caution():
|
||||
blocks = markdown_to_notion_blocks("> [!CAUTION]\n> Do not proceed.")
|
||||
_assert(blocks[0]["callout"]["icon"]["emoji"] == "🚨", "CAUTION icon is 🚨")
|
||||
_assert(blocks[0]["callout"]["color"] == "red_background", "CAUTION color is red")
|
||||
|
||||
|
||||
def test_callout_unknown_falls_through():
|
||||
"""An unrecognized alert type renders as a plain quote with the marker preserved."""
|
||||
blocks = markdown_to_notion_blocks("> [!BOGUS]\n> some content")
|
||||
_assert(blocks[0]["type"] == "quote", "unknown alert renders as quote, not callout")
|
||||
quote_text = blocks[0]["quote"]["rich_text"][0]["text"]["content"]
|
||||
_assert("[!BOGUS]" in quote_text, "[!BOGUS] marker preserved in quote text")
|
||||
|
||||
|
||||
def test_toggle_basic():
|
||||
md = "<details>\n<summary>Click to expand</summary>\n\nInner paragraph.\n</details>"
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(len(blocks) == 1, "one block emitted")
|
||||
_assert(blocks[0]["type"] == "toggle", "block type is toggle")
|
||||
summary_text = blocks[0]["toggle"]["rich_text"][0]["text"]["content"]
|
||||
_assert(summary_text == "Click to expand", "summary text preserved")
|
||||
children = blocks[0]["toggle"]["children"]
|
||||
_assert(len(children) == 1, "one child block")
|
||||
_assert(children[0]["type"] == "paragraph", "child is paragraph")
|
||||
|
||||
|
||||
def test_toggle_nested_blocks():
|
||||
"""Toggle body can hold any block type via _parse_lines recursion."""
|
||||
md = """<details>
|
||||
<summary>Debug log</summary>
|
||||
|
||||
- step one
|
||||
- step two
|
||||
|
||||
```python
|
||||
print("hello")
|
||||
```
|
||||
</details>"""
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(blocks[0]["type"] == "toggle", "block is toggle")
|
||||
children = blocks[0]["toggle"]["children"]
|
||||
types = [c["type"] for c in children]
|
||||
_assert("bulleted_list_item" in types, "list child preserved")
|
||||
_assert("code" in types, "code child preserved")
|
||||
code_block = next(c for c in children if c["type"] == "code")
|
||||
_assert(code_block["code"]["language"] == "python", "code language preserved")
|
||||
|
||||
|
||||
def test_toggle_nested_toggle():
|
||||
"""details_depth tracking allows <details> inside <details>."""
|
||||
md = """<details>
|
||||
<summary>outer</summary>
|
||||
|
||||
<details>
|
||||
<summary>inner</summary>
|
||||
|
||||
inner content
|
||||
</details>
|
||||
</details>"""
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(blocks[0]["type"] == "toggle", "outer block is toggle")
|
||||
outer_children = blocks[0]["toggle"]["children"]
|
||||
_assert(len(outer_children) == 1, "outer has exactly one child")
|
||||
_assert(outer_children[0]["type"] == "toggle", "outer child is also toggle")
|
||||
inner_summary = outer_children[0]["toggle"]["rich_text"][0]["text"]["content"]
|
||||
_assert(inner_summary == "inner", "inner summary preserved")
|
||||
|
||||
|
||||
def test_toggle_unclosed_falls_through():
|
||||
"""Missing </details> at EOF degrades to paragraphs, no crash."""
|
||||
md = "<details>\n<summary>oops</summary>\n\nbody line"
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
types = [b["type"] for b in blocks]
|
||||
_assert("toggle" not in types, "no toggle emitted on unclosed <details>")
|
||||
paragraph_texts = [
|
||||
b["paragraph"]["rich_text"][0]["text"]["content"]
|
||||
for b in blocks if b["type"] == "paragraph"
|
||||
]
|
||||
joined = " ".join(paragraph_texts)
|
||||
_assert("oops" in joined, "summary preserved as paragraph")
|
||||
_assert("body line" in joined, "body preserved as paragraph")
|
||||
|
||||
|
||||
def test_columns_two():
|
||||
md = """::: columns
|
||||
::: column
|
||||
left content
|
||||
:::
|
||||
::: column
|
||||
right content
|
||||
:::
|
||||
:::"""
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(len(blocks) == 1, "one block emitted")
|
||||
_assert(blocks[0]["type"] == "column_list", "block type is column_list")
|
||||
cols = blocks[0]["column_list"]["children"]
|
||||
_assert(len(cols) == 2, "two columns")
|
||||
_assert(cols[0]["type"] == "column", "child 0 is column")
|
||||
_assert(cols[1]["type"] == "column", "child 1 is column")
|
||||
left_para = cols[0]["column"]["children"][0]["paragraph"]["rich_text"][0]["text"]["content"]
|
||||
right_para = cols[1]["column"]["children"][0]["paragraph"]["rich_text"][0]["text"]["content"]
|
||||
_assert(left_para == "left content", "left column content preserved")
|
||||
_assert(right_para == "right content", "right column content preserved")
|
||||
|
||||
|
||||
def test_columns_with_blocks():
|
||||
"""Columns can hold any block type via per-column _parse_lines recursion."""
|
||||
md = """::: columns
|
||||
::: column
|
||||
- bullet a
|
||||
- bullet b
|
||||
:::
|
||||
::: column
|
||||
```python
|
||||
print(1)
|
||||
```
|
||||
:::
|
||||
:::"""
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
_assert(blocks[0]["type"] == "column_list", "outer is column_list")
|
||||
cols = blocks[0]["column_list"]["children"]
|
||||
_assert(len(cols) == 2, "two columns")
|
||||
left_types = [c["type"] for c in cols[0]["column"]["children"]]
|
||||
right_types = [c["type"] for c in cols[1]["column"]["children"]]
|
||||
_assert(left_types.count("bulleted_list_item") == 2, "left column has 2 bullets")
|
||||
_assert("code" in right_types, "right column has code block")
|
||||
|
||||
|
||||
def test_columns_single_degrades():
|
||||
"""Single-column ::: columns block degrades to paragraphs (Notion requires >= 2)."""
|
||||
md = """::: columns
|
||||
::: column
|
||||
only one column here
|
||||
:::
|
||||
:::"""
|
||||
blocks = markdown_to_notion_blocks(md)
|
||||
types = [b["type"] for b in blocks]
|
||||
_assert("column_list" not in types, "no column_list emitted for single column")
|
||||
paragraph_texts = [
|
||||
b["paragraph"]["rich_text"][0]["text"]["content"]
|
||||
for b in blocks if b["type"] == "paragraph"
|
||||
]
|
||||
joined = " ".join(paragraph_texts)
|
||||
_assert("only one column here" in joined, "content preserved as paragraph")
|
||||
|
||||
|
||||
def test_mention_id():
|
||||
"""@[Title](32-hex-id) produces a mention rich-text span with page reference."""
|
||||
raw_id = "abcdef0123456789abcdef0123456789"
|
||||
spans = parse_rich_text(f"see @[Roadmap]({raw_id}) for plans")
|
||||
mention_spans = [s for s in spans if s.get("type") == "mention"]
|
||||
_assert(len(mention_spans) == 1, "one mention span emitted")
|
||||
m = mention_spans[0]
|
||||
_assert(m["mention"]["type"] == "page", "mention type is page")
|
||||
_assert(m["mention"]["page"]["id"].replace("-", "") == raw_id, "page id resolved")
|
||||
_assert(m.get("plain_text") == "Roadmap", "plain_text is the title")
|
||||
|
||||
|
||||
def test_mention_url():
|
||||
"""@[Title](https://notion.so/Page-Title-id) extracts ID from URL."""
|
||||
raw_id = "abcdef0123456789abcdef0123456789"
|
||||
url = f"https://notion.so/My-Page-{raw_id}"
|
||||
spans = parse_rich_text(f"check @[My Page]({url})")
|
||||
mention_spans = [s for s in spans if s.get("type") == "mention"]
|
||||
_assert(len(mention_spans) == 1, "one mention span emitted from URL form")
|
||||
_assert(mention_spans[0]["mention"]["page"]["id"].replace("-", "") == raw_id,
|
||||
"ID extracted from Notion URL")
|
||||
|
||||
|
||||
def test_mention_invalid_falls_back():
|
||||
"""A non-resolvable target degrades to plain text '@Title' with no link."""
|
||||
spans = parse_rich_text("ping @[Bob](not-a-real-id) please")
|
||||
mention_spans = [s for s in spans if s.get("type") == "mention"]
|
||||
_assert(len(mention_spans) == 0, "no mention span for invalid id")
|
||||
plain_text_joined = "".join(
|
||||
s["text"]["content"] for s in spans if s.get("type") == "text"
|
||||
)
|
||||
_assert("@Bob" in plain_text_joined, "title rendered as plain '@Bob'")
|
||||
_assert("not-a-real-id" not in plain_text_joined, "invalid id stripped from output")
|
||||
|
||||
|
||||
def test_rich_text_anchor_link_becomes_bold():
|
||||
"""TOC-style fragment links must not be sent as Notion link annotations."""
|
||||
spans = parse_rich_text("see [Section A](#section-a) below")
|
||||
@@ -214,6 +427,22 @@ def run_all():
|
||||
test_blocks_todo,
|
||||
test_blocks_quote_code_divider,
|
||||
test_blocks_table,
|
||||
test_callout_note,
|
||||
test_callout_tip,
|
||||
test_callout_important,
|
||||
test_callout_warning,
|
||||
test_callout_caution,
|
||||
test_callout_unknown_falls_through,
|
||||
test_toggle_basic,
|
||||
test_toggle_nested_blocks,
|
||||
test_toggle_nested_toggle,
|
||||
test_toggle_unclosed_falls_through,
|
||||
test_columns_two,
|
||||
test_columns_with_blocks,
|
||||
test_columns_single_degrades,
|
||||
test_mention_id,
|
||||
test_mention_url,
|
||||
test_mention_invalid_falls_back,
|
||||
test_rich_text_anchor_link_becomes_bold,
|
||||
test_rich_text_relative_link_becomes_plain,
|
||||
test_rich_text_absolute_link_preserved,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,229 @@
|
||||
# Notion Writer (Skill 32) — Extended Block Coverage Design
|
||||
|
||||
> **Date**: 2026-04-27
|
||||
> **Status**: Approved (brainstorming)
|
||||
> **Scope**: Phase 3c — add callout, toggle, column, and page-mention block support to `notion_writer.py`'s markdown→Notion parser
|
||||
> **Sequence**: First of three Phase 3 sub-projects (3c → 3b → 3a per user-approved order)
|
||||
> **Predecessor**: Phase 2 commit `144a17c` (multi-source data API migration)
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
Extend the markdown→Notion-blocks converter in `custom-skills/32-notion-writer/code/scripts/notion_writer.py` to support four block types currently unhandled: **callouts**, **toggles**, **columns**, and **inline page mentions**. The parser remains pure (no API calls during parse), preserves existing block-type behavior unchanged, and stays under the project's "small, well-tested" line-scanner architecture.
|
||||
|
||||
This work also lays the foundation for Phase 3b (Notion-as-RAG export), which needs a clean blocks→markdown reverse converter; the syntax chosen here must round-trip cleanly.
|
||||
|
||||
---
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Notion blocks not in the four-item list above (image upload, bookmark, embed, equation, breadcrumb, table-of-contents, synced block — defer until a concrete use case demands them).
|
||||
- LLM-driven content distillation (belongs to Phase 3b downstream of `90-reference-curator`).
|
||||
- Search-by-title page-mention resolution (rejected during brainstorming for purity reasons; explicit ID/URL only).
|
||||
- Cross-block-type round-trip tests (deferred to Phase 3b when the reverse converter exists).
|
||||
|
||||
---
|
||||
|
||||
## Architectural decisions (locked during brainstorming)
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
|---|---|---|
|
||||
| Syntax style | **Hybrid**: GitHub alerts (`> [!NOTE]`) for callouts, HTML5 `<details>` for toggles, Pandoc fenced div (`::: columns`) for columns | Each element uses the most-portable syntax available; renders correctly on GitHub for the formats that have native support |
|
||||
| Container nesting | **Full recursion** — anything legal at top level is legal inside a toggle or column, including nested toggles/columns | Dev logs commonly put code blocks and lists inside toggles; the limitation is unacceptable |
|
||||
| Page-mention syntax | **ID-or-URL** — `@[Title](page-id)` and `@[Title](https://notion.so/Page-id)` both accepted | URL form is what users naturally paste; ID form is what 3b's reverse converter emits |
|
||||
| Implementation shape | **Reentrant flat parser** — `markdown_to_notion_blocks` accepts string or list of lines; container detectors recurse on inner content | Smallest diff, preserves existing line-scanner architecture, additive |
|
||||
|
||||
---
|
||||
|
||||
## Public API
|
||||
|
||||
`markdown_to_notion_blocks` becomes reentrant by widening its parameter type:
|
||||
|
||||
```python
|
||||
def markdown_to_notion_blocks(content: Union[str, List[str]]) -> List[Dict[str, Any]]:
|
||||
lines = content.splitlines() if isinstance(content, str) else list(content)
|
||||
return _parse_lines(lines)
|
||||
```
|
||||
|
||||
The current loop body moves into a new private `_parse_lines(lines: List[str])`. All existing callers continue to pass strings; nothing breaks.
|
||||
|
||||
---
|
||||
|
||||
## New block factories
|
||||
|
||||
Added alongside existing `create_*_block` helpers:
|
||||
|
||||
| Factory | Output shape |
|
||||
|---|---|
|
||||
| `create_callout_block(rich_text_spans, alert_type)` | `{"type": "callout", "callout": {"rich_text": ..., "icon": {"type": "emoji", "emoji": "ℹ️"}, "color": "blue_background"}}` |
|
||||
| `create_toggle_block(summary_spans, children_blocks)` | `{"type": "toggle", "toggle": {"rich_text": ..., "children": ...}}` |
|
||||
| `create_column_list_block(column_blocks_lists)` | `{"type": "column_list", "column_list": {"children": [{"type": "column", "column": {"children": [...]}}]}}` |
|
||||
|
||||
Page mentions are inline rich-text — no new block factory; one new pattern in `INLINE_PATTERNS`.
|
||||
|
||||
---
|
||||
|
||||
## Constants
|
||||
|
||||
Added at the top of the file alongside `INLINE_PATTERNS`:
|
||||
|
||||
```python
|
||||
ALERT_TYPES = {
|
||||
'NOTE': ('ℹ️', 'blue_background'),
|
||||
'TIP': ('💡', 'green_background'),
|
||||
'IMPORTANT': ('☝️', 'purple_background'),
|
||||
'WARNING': ('⚠️', 'yellow_background'),
|
||||
'CAUTION': ('🚨', 'red_background'),
|
||||
}
|
||||
ALERT_RE = re.compile(r'^\s*\[!(NOTE|TIP|IMPORTANT|WARNING|CAUTION)\]\s*$')
|
||||
```
|
||||
|
||||
The icons match GitHub's render (shape and hue); colors slot into Notion's named-color palette directly.
|
||||
|
||||
---
|
||||
|
||||
## Block-type details
|
||||
|
||||
### Callout
|
||||
|
||||
- **Detect**: blockquote line where the first content line (after `>`) matches `ALERT_RE`
|
||||
- **Body**: collect subsequent contiguous `>` lines until a non-`>` line; concatenate with `\n` separators into one rich-text run (matches GitHub's render — alerts are single-block, no nested lists/code). Body text passes through `parse_rich_text` so bold/italic/code/links/mentions work inside the callout.
|
||||
- **Unknown alert type** (e.g., `> [!FOO]`): fall through to existing quote handling, `[!FOO]` preserved as text
|
||||
- **Notion shape**: callout block with `rich_text`, `icon.emoji`, `color`
|
||||
|
||||
### Toggle
|
||||
|
||||
- **Detect**: line equals `<details>` (whitespace-tolerant). Single-line forms like `<details><summary>X</summary>body</details>` are **not supported** in v1 — require the multi-line layout (open tag on its own line). Most markdown editors that emit `<details>` already use multi-line.
|
||||
- **Summary**: next non-blank line if it matches `<summary>(.*)</summary>`; if absent, summary is empty rich-text. Summary text passes through `parse_rich_text` so bold/italic/code/links/mentions work inside it.
|
||||
- **Body**: lines between `</summary>` (or first non-summary line) and the matching `</details>`, depth-tracked via `details_depth` int so `<details>` inside `<details>` works correctly
|
||||
- **Recurse**: body lines fed to `_parse_lines`, result attached as `children`
|
||||
- **Unclosed** at EOF: emit body lines as plain paragraphs, one-line stderr warning
|
||||
|
||||
### Columns
|
||||
|
||||
- **Detect**: line equals `::: columns`
|
||||
- **Inner separators**: `::: column` begins each child column
|
||||
- **Close**: bare `:::` closes either a column or the wrapper, depending on `colon_depth`
|
||||
- **Recurse**: each column's lines fed to `_parse_lines`, results attached as `children` of `column` blocks; `column` blocks attached as `children` of `column_list`
|
||||
- **Validation**: ≥2 columns required by Notion; single-column `::: columns` block degrades to plain paragraphs (skip wrapper)
|
||||
- **Unclosed** at EOF: emit lines as plain paragraphs, one-line stderr warning
|
||||
|
||||
### Page mentions (inline rich-text)
|
||||
|
||||
- **Pattern**: new entry in `INLINE_PATTERNS`: `('mention', re.compile(r'@\[([^\]]+)\]\(([^)\s]+)\)'))`
|
||||
- **Resolution**: `extract_notion_id(target)` reuses existing logic — accepts raw 32-char hex, dashed UUID, or full Notion URL
|
||||
- **Valid ID**: rich-text span with shape:
|
||||
```python
|
||||
{
|
||||
"type": "mention",
|
||||
"mention": {"type": "page", "page": {"id": resolved_id}},
|
||||
"plain_text": title_text,
|
||||
}
|
||||
```
|
||||
- **Invalid ID** (extract returns None): plain text `@Title` with no link annotation, no warning (graceful — many false positives are possible with `@`)
|
||||
|
||||
---
|
||||
|
||||
## Recursion mechanics
|
||||
|
||||
The line-scanning loop in `_parse_lines` gets three new detectors, ordered before existing ones:
|
||||
|
||||
```
|
||||
for line in lines:
|
||||
if in_code_fence: ... # existing — wins over all
|
||||
if in_table: ... # existing
|
||||
if line == '<details>': ... # NEW — capture to </details>, recurse
|
||||
if line == '::: columns': ... # NEW — capture columns, recurse per column
|
||||
if line.startswith('>') and ALERT_RE.match(rest): ... # NEW — collect callout body
|
||||
# fall through to existing detectors (heading/list/quote/code-fence/table/etc.)
|
||||
```
|
||||
|
||||
Recursion algorithm for the two recursive container types:
|
||||
|
||||
1. Detect open delimiter at line *i*
|
||||
2. Scan forward to matching close, tracking depth for nested same-kind delimiters (`<details>`/`</details>` for toggles, `:::`/`::: columns` for columns)
|
||||
3. Slice inner lines into a sub-list (per-column sub-lists for columns)
|
||||
4. Recursively call `_parse_lines(sub_lines)`
|
||||
5. Wrap result in the appropriate Notion container block
|
||||
6. Advance the outer loop index past the close
|
||||
|
||||
State held by the scanner: two ints (`details_depth`, `colon_depth`), since the two container kinds don't share delimiters with each other or with existing scanner state.
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
Permissive philosophy — never fail the push, degrade to readable text on malformed input. Same posture as the Phase 2 anchor-link fix.
|
||||
|
||||
| Failure | Behavior |
|
||||
|---|---|
|
||||
| Unclosed `<details>` at EOF | Emit body lines as paragraphs; one-line stderr warning |
|
||||
| Unclosed `::: columns` at EOF | Same |
|
||||
| Single-column `::: columns` block | Degrade to paragraphs (skip wrapper, no warning — caller's choice) |
|
||||
| Unknown alert type `> [!FOO]` | Fall through to existing quote rendering, preserving `[!FOO]` text |
|
||||
| Invalid mention ID | Plain text `@Title`, no warning (avoid false-positive spam from any `@[x](y)` pattern) |
|
||||
| Missing `<summary>` in `<details>` | Empty summary, body parses normally |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
Twelve new tests added to `test_parser.py`, bringing total from 16 → 28. Round-trip (markdown → blocks → markdown) tests deferred to Phase 3b.
|
||||
|
||||
| Test | What it verifies |
|
||||
|---|---|
|
||||
| `test_callout_note` | `> [!NOTE]` emits `callout` with ℹ️ icon and `blue_background` |
|
||||
| `test_callout_tip` | `> [!TIP]` → 💡, green |
|
||||
| `test_callout_important` | `> [!IMPORTANT]` → ☝️, purple |
|
||||
| `test_callout_warning` | `> [!WARNING]` → ⚠️, yellow |
|
||||
| `test_callout_caution` | `> [!CAUTION]` → 🚨, red |
|
||||
| `test_callout_unknown_falls_through` | `> [!BOGUS]` becomes a quote, `[!BOGUS]` preserved as text |
|
||||
| `test_toggle_basic` | `<details><summary>X</summary>body</details>` → toggle with summary X and paragraph children |
|
||||
| `test_toggle_nested_blocks` | Toggle containing list + code block (validates `_parse_lines` recursion) |
|
||||
| `test_toggle_nested_toggle` | Toggle inside toggle (validates `details_depth` tracking) |
|
||||
| `test_toggle_unclosed_falls_through` | Missing `</details>` produces text paragraphs, no crash |
|
||||
| `test_columns_two` | Basic two-column layout with paragraphs |
|
||||
| `test_columns_with_blocks` | Columns containing lists/code (validates per-column recursion) |
|
||||
| `test_columns_single_degrades` | Single-column `::: columns` block emits paragraphs, no `column_list` |
|
||||
| `test_mention_id` | `@[Title](32-hex-id)` produces `mention` rich-text span |
|
||||
| `test_mention_url` | `@[Title](https://notion.so/Title-id)` resolves to ID |
|
||||
| `test_mention_invalid_falls_back` | Bad ID → plain text `@Title`, no link/mention annotation |
|
||||
|
||||
(Test count is 16 in this list because the 5 callouts are listed individually; some may collapse into parametrized tests during implementation. Total stays ≥12 new.)
|
||||
|
||||
---
|
||||
|
||||
## File changes
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `custom-skills/32-notion-writer/code/scripts/notion_writer.py` | Refactor `markdown_to_notion_blocks` to be reentrant; add `_parse_lines`; add 3 block factories + 1 inline pattern; add `ALERT_TYPES` and `ALERT_RE` constants |
|
||||
| `custom-skills/32-notion-writer/code/scripts/test_parser.py` | +12 tests; total 28 |
|
||||
| `custom-skills/32-notion-writer/code/CLAUDE.md` | Document new syntax in the Markdown Support section; bump version footer to 1.2.0 with changelog entry |
|
||||
|
||||
No new files. No new dependencies. The parser remains pure.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope (parking lot)
|
||||
|
||||
- **3b — Notion-as-RAG export**: requires a blocks → markdown reverse converter that emits the same syntax this design defines. The hybrid syntax was chosen partly to round-trip cleanly. Implementation deferred.
|
||||
- **3a — Metadata-aware migration**: cross-database moves with auto-mapped properties, dry-run diff, type-aware transforms. Builds on `31-notion-organizer/scripts/schema_migrator.py`. Deferred.
|
||||
- **Block-type expansion** beyond the four named: image (upload), bookmark, embed, equation, synced block, breadcrumb, table-of-contents — added when concrete use cases demand them.
|
||||
|
||||
---
|
||||
|
||||
## Implementation transition
|
||||
|
||||
After this spec is approved by the user, transition to `superpowers:writing-plans` to produce the step-by-step implementation plan. The plan will likely break into:
|
||||
|
||||
1. Refactor `markdown_to_notion_blocks` to reentrant form (no behavior change; verify all 16 existing tests still pass)
|
||||
2. Add callout detector + factory + 6 callout tests
|
||||
3. Add toggle detector + factory + 4 toggle tests (depth tracking covered here)
|
||||
4. Add columns detector + factory + 3 columns tests (depth tracking covered here)
|
||||
5. Add page-mention inline pattern + 3 mention tests
|
||||
6. Update `notion_writer.py` CLAUDE.md docs + version bump
|
||||
7. Run full test suite (28 expected); commit
|
||||
|
||||
Each step is independently testable and revertable.
|
||||
Reference in New Issue
Block a user