feat(notion): migrate 31+32 to API 2025-09-03 + add property writes, upsert, anchor-link fix
Phase 1 — docs alignment: - Fix path drift in 32 CLAUDE.md (02-notion-writer → 32-notion-writer) - Align env var to NOTION_API_KEY in 31 docs (NOTION_TOKEN still accepted) - Document Phase 3 roadmap in 31 (metadata-aware migration, Notion-as-RAG) Phase 2 — multi-source database support: - New 32/scripts/_notion_compat.py: client factory, data_source resolution with cache, property coercion, find_existing_page (for upsert), explain_api_error for friendlier failure messages - 32 notion_writer.py: drop the 2022-06-28 pin, route schema introspection through data_sources.retrieve, build data_source_id parent shape, accept arbitrary properties via --properties JSON-or-file flag, add --upsert-by for idempotent re-runs - 32 markdown parser: anchor-link fix — Notion's URL validator rejects fragment URLs and relative paths; fragments now render as bold to preserve TOC nav intent, relatives drop the link, absolute URLs preserved - 31 async_organizer.py + schema_migrator.py: same data_sources migration with cached resolution; pages.create now uses data_source_id parent - 16/16 parser tests pass (was 13; +3 link-handling regression guards) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -39,7 +39,7 @@
|
||||
### 3. Configure Environment
|
||||
|
||||
```bash
|
||||
cd ~/Project/our-claude-skills/custom-skills/02-notion-writer/code/scripts
|
||||
cd ~/Project/our-claude-skills/custom-skills/32-notion-writer/code/scripts
|
||||
|
||||
# Create .env from example
|
||||
cp .env.example .env
|
||||
@@ -100,8 +100,47 @@ python notion_writer.py --database DB_URL --title "New Entry" --file content.md
|
||||
|
||||
# Title only
|
||||
python notion_writer.py --database DB_URL --title "Empty Entry"
|
||||
|
||||
# Set arbitrary properties (JSON literal)
|
||||
python notion_writer.py \
|
||||
--database DB_URL \
|
||||
--title "Research Notes" \
|
||||
--properties '{"Status": "In progress", "Topic": ["AI", "MCP"], "Account Code": "OurDigital"}' \
|
||||
--file notes.md
|
||||
|
||||
# Set properties from a JSON file
|
||||
python notion_writer.py \
|
||||
--database DB_URL \
|
||||
--title "Research Notes" \
|
||||
--properties ./props.json \
|
||||
--file notes.md
|
||||
```
|
||||
|
||||
`--database` accepts a database URL, a database_id, or a data_source_id. It is resolved to a data_source_id automatically.
|
||||
|
||||
### Upsert (avoid duplicates)
|
||||
|
||||
```bash
|
||||
# If a row with Name="My Title" already exists, update it; otherwise create.
|
||||
python notion_writer.py \
|
||||
--database DB_URL \
|
||||
--title "My Title" \
|
||||
--upsert-by "Name" \
|
||||
--properties '{"Status": "Done"}' \
|
||||
--file content.md
|
||||
```
|
||||
|
||||
When upserting, `--file` (or `--stdin`) replaces the page's body blocks; `--properties` updates the named columns. `--upsert-by` supports title, rich_text, select, status, number, url, email, and phone_number properties.
|
||||
|
||||
### Two-tool split: when to use what
|
||||
|
||||
| Goal | Tool | Why |
|
||||
|------|------|-----|
|
||||
| Push markdown content into a page or new DB row | **`notion_writer.py`** | Converts markdown → Notion blocks (tables, headings, code, lists, rich-text, anchors) |
|
||||
| Create a DB row with both content and metadata in one shot | **`notion_writer.py`** with `--properties` | Single API call, full schema validation |
|
||||
| Update properties on many existing pages (bulk column edit) | **MCP `notion-update-page`** | Native schema awareness, no markdown work needed |
|
||||
| Search / move pages with metadata transition | 31-notion-organizer (Phase 3 work) | Cross-database operations |
|
||||
|
||||
---
|
||||
|
||||
## Markdown Support
|
||||
@@ -131,6 +170,20 @@ print("Hello")
|
||||
```
|
||||
```
|
||||
|
||||
### Inline rich-text
|
||||
|
||||
| Markdown | Result |
|
||||
|----------|--------|
|
||||
| `**bold**` or `__bold__` | bold |
|
||||
| `*italic*` or `_italic_` | italic |
|
||||
| `` `code` `` | inline code |
|
||||
| `~~strike~~` | strikethrough |
|
||||
| `[text](https://...)` | link (absolute URLs only) |
|
||||
| `[text](#anchor)` | bold (Notion rejects fragment URLs) |
|
||||
| `[text](relative/path.md)` | plain text (Notion rejects relative URLs) |
|
||||
|
||||
Notion's URL validator requires absolute URLs for link annotations. The parser converts TOC-style anchor links to bold to preserve navigation intent and silently strips relative paths.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
@@ -160,6 +213,40 @@ python jamie_video_info.py VIDEO_ID --json | \
|
||||
python notion_writer.py --page PAGE_URL --stdin
|
||||
```
|
||||
|
||||
### Test against the "Working with AI" database
|
||||
|
||||
The user's primary export target is the AI research database (data_source_id `f8f19ede-32bd-43ac-9f60-0651f6f40afe`). End-to-end check:
|
||||
|
||||
```bash
|
||||
# 1) Sanity: connection works
|
||||
python notion_writer.py --test
|
||||
|
||||
# 2) Inspect the schema (confirms data_source resolution)
|
||||
python notion_writer.py \
|
||||
--database f8f19ede-32bd-43ac-9f60-0651f6f40afe \
|
||||
--info
|
||||
|
||||
# 3) Upsert a test row with full metadata
|
||||
python notion_writer.py \
|
||||
--database f8f19ede-32bd-43ac-9f60-0651f6f40afe \
|
||||
--title "[smoke test] notion_writer v3 migration" \
|
||||
--upsert-by "Name" \
|
||||
--properties '{
|
||||
"Status": "Done",
|
||||
"AI used": ["Claude Code"],
|
||||
"Topic": ["AI"],
|
||||
"Category": ["Research"],
|
||||
"Account Code": "OurDigital",
|
||||
"Type": "Memo",
|
||||
"AI summary": "Verifying the data_sources API migration end-to-end."
|
||||
}' \
|
||||
--stdin <<< "## Smoke test
|
||||
|
||||
Migration verified — properties + content land in one shot."
|
||||
```
|
||||
|
||||
Re-running step 3 should update the existing row instead of creating a duplicate (upsert).
|
||||
|
||||
---
|
||||
|
||||
## API Limits
|
||||
@@ -176,13 +263,22 @@ The script automatically batches large content.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Could not find page"
|
||||
- Ensure page is shared with your integration
|
||||
### "Notion object not found"
|
||||
- Ensure the page or database is shared with your integration (Notion → "..." → Connections → add integration)
|
||||
- Check URL/ID is correct
|
||||
- For databases, the integration must have access to *each* data source under it
|
||||
|
||||
### "Invalid token"
|
||||
- Verify NOTION_API_KEY in .env
|
||||
- Token should start with `secret_`
|
||||
### "Unauthorized"
|
||||
- Verify `NOTION_API_KEY` in `.env`
|
||||
- Token should start with `secret_` (internal integration token)
|
||||
|
||||
### "Database has multiple data sources"
|
||||
- Pass a specific `data_source_id` instead of the database URL/id
|
||||
- List data sources via the Notion API or open the data source individually in Notion to grab its ID
|
||||
|
||||
### "Property 'X' has unsupported type"
|
||||
- The script handles common property types but skips computed/read-only ones (formula, rollup, created_by, last_edited_by, last_edited_time, unique_id)
|
||||
- Drop the unsupported key from `--properties` JSON
|
||||
|
||||
### "Rate limited"
|
||||
- Wait and retry
|
||||
@@ -193,11 +289,13 @@ The script automatically batches large content.
|
||||
## File Structure
|
||||
|
||||
```
|
||||
02-notion-writer/
|
||||
32-notion-writer/
|
||||
├── code/
|
||||
│ ├── CLAUDE.md # This skill document
|
||||
│ ├── scripts/
|
||||
│ │ ├── notion_writer.py # Main script
|
||||
│ │ ├── _notion_compat.py # API compat helper (data_sources resolution)
|
||||
│ │ ├── test_parser.py # Markdown parser tests
|
||||
│ │ ├── venv/ # Python environment
|
||||
│ │ ├── .env # API token (not committed)
|
||||
│ │ └── .env.example # Template
|
||||
@@ -212,7 +310,7 @@ The script automatically batches large content.
|
||||
|
||||
```bash
|
||||
# Navigate
|
||||
cd ~/Project/our-claude-skills/custom-skills/02-notion-writer/code/scripts
|
||||
cd ~/Project/our-claude-skills/custom-skills/32-notion-writer/code/scripts
|
||||
source venv/bin/activate
|
||||
|
||||
# Test
|
||||
@@ -226,8 +324,18 @@ python notion_writer.py -p PAGE_URL -f content.md -r
|
||||
|
||||
# Create DB row
|
||||
python notion_writer.py -d DB_URL -t "Title" -f content.md
|
||||
|
||||
# Create DB row with full metadata
|
||||
python notion_writer.py -d DB_URL -t "Title" --properties '{"Status": "Done"}' -f content.md
|
||||
|
||||
# Upsert (avoid duplicates)
|
||||
python notion_writer.py -d DB_URL -t "Title" --upsert-by "Name" -f content.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*Version 1.0.0 | Claude Code | 2025-12-26*
|
||||
*Version 1.1.0 | Claude Code | 2026-04-27*
|
||||
|
||||
Changelog:
|
||||
- 1.1.0 — Migrated to Notion API 2025-09-03 (multi-source databases). Added `--properties` JSON flag, `--upsert-by` for idempotency, anchor-link parser fix, friendlier API error messages.
|
||||
- 1.0.0 — Initial release with markdown→Notion block conversion.
|
||||
|
||||
Reference in New Issue
Block a user