Compare commits
6 Commits
3fbce9dafb
...
1706a820fe
| Author | SHA1 | Date | |
|---|---|---|---|
| 1706a820fe | |||
| 3a8edebfef | |||
| 4f48ba3c59 | |||
| ba88247496 | |||
| a55e77d1b0 | |||
| 35f155fa90 |
@@ -39,7 +39,7 @@ This is a Claude Skills collection repository containing:
|
||||
| 14 | seo-core-web-vitals | LCP, CLS, FID, INP metrics | "Core Web Vitals", "page speed" |
|
||||
| 15 | seo-search-console | GSC data analysis | "Search Console", "rankings" |
|
||||
| 16 | seo-schema-validator | Structured data validation | "validate schema", "JSON-LD" |
|
||||
| 17 | seo-schema-generator | Schema markup creation | "generate schema", "create JSON-LD" |
|
||||
| 17 | seo-schema-generator | JSON-LD generation — Mode 1 from existing site, Mode 2 from collected sources → claims register → drafts → validate (16) | "generate schema", "create JSON-LD", "source-to-schema", "schema from site" |
|
||||
| 18 | seo-local-audit | NAP, GBP, citations | "local SEO", "Google Business Profile" |
|
||||
| 19 | seo-keyword-strategy | Keyword expansion, intent, clustering, gaps | "keyword research", "keyword strategy" |
|
||||
| 20 | seo-serp-analysis | Google/Naver SERP features, competitor positions | "SERP analysis", "SERP features" |
|
||||
|
||||
100
custom-skills/16-seo-schema-validator/SKILL.md
Normal file
100
custom-skills/16-seo-schema-validator/SKILL.md
Normal file
@@ -0,0 +1,100 @@
|
||||
---
|
||||
name: 16-seo-schema-validator
|
||||
description: |
|
||||
Validates an AUTHORED JSON-LD schema dataset (pre-deployment QA) and audits
|
||||
live structured data (post-deployment). Runs a 5-layer offline validation
|
||||
pipeline (coverage, syntax, vocabulary, Google rich-result requirements,
|
||||
business-logic/consistency) and emits a severity-ranked defect log, a gate
|
||||
decision, and a Markdown report. Fills the "16-seo-schema-validator" slot
|
||||
referenced by seo-comprehensive-audit.
|
||||
Triggers: schema validation, JSON-LD QA, structured data check, schema 검수,
|
||||
스키마 유효성 검증, 구조화 데이터 검토, rich result eligibility, schema 오류 점검.
|
||||
version: "1.0"
|
||||
author: OurDigital / D.intelligence
|
||||
environment: Code
|
||||
---
|
||||
|
||||
# SEO Schema Validator (16)
|
||||
|
||||
Quality-assure structured data at scale. Built for the kind of failure where a
|
||||
client review of hundreds of authored entries surfaces "too many errors" — by
|
||||
moving the cheap, machine-checkable errors OUT of the human review and INTO an
|
||||
automated gate that runs first.
|
||||
|
||||
## Two modes
|
||||
|
||||
| Mode | When | Input | Adds |
|
||||
|------|------|-------|------|
|
||||
| **A — Dataset QA (default)** | Before deployment, while authoring/reviewing | An authored dataset: `.xlsx` / `.csv` (one row per entry, a JSON-LD column), `.jsonl`, `.json`, or a directory of `.json/.jsonld` | Layer 0 coverage vs the canonical URL list |
|
||||
| **B — Live audit** | After deployment, or feeding `seo-comprehensive-audit` | Live URLs (extract embedded JSON-LD first, then validate) | Layer 5 rendering-reality (schema present in rendered HTML, matches content) |
|
||||
|
||||
This skill's primary job is **Mode A**: catch errors before the client sees them.
|
||||
|
||||
## The 5 validation layers
|
||||
|
||||
| # | Layer | Catches | Default severity |
|
||||
|---|-------|---------|------------------|
|
||||
| L0 | Coverage | URLs with no entry; entries whose URL isn't in the inventory | P1 / P2 |
|
||||
| L1 | Syntax | invalid JSON, missing/wrong `@context`, no `@type`, encoding corruption | P0 / P1 |
|
||||
| L2 | Vocabulary | unknown type, property not valid for type, bad value formats (URL/date/lang/currency/number) | P1 / P2 |
|
||||
| L3 | Rich-result | Google **required** missing (blocks rich result); recommended absent | P0 / P2 |
|
||||
| L4 | Consistency | NAP mismatch across a property, `@id` dupes/dangling refs, swapped geo, placeholder text, duplicate descriptions | P0 / P1 |
|
||||
|
||||
Full rationale and the type→requirement matrix: `references/validation-methodology.md`.
|
||||
Severity + category codes: `references/defect-taxonomy.md`.
|
||||
Hotel page-type → schema-type map: `references/hotel-type-map.md`.
|
||||
Client-facing report + P1 decision log: `templates/client-qa-report-template.md`, `templates/decision-log.md`.
|
||||
|
||||
## Stage gates (aligned to the project lifecycle 설계→개발→테스트→안정화→런칭 후)
|
||||
|
||||
- **G1 설계** — Lock the schema spec and the page-type→type map (`hotel-type-map.md`). Approve the entry template. *DoD:* every page template has an assigned schema type and a required-property list.
|
||||
- **G2 개발** — Authors produce entries. Run the validator with `--strict`. *DoD (gate):* **zero P0**, JSON parses for 100% of entries. Entries failing this NEVER reach client review.
|
||||
- **G3 테스트** — Re-run; triage P1 in `defect_log.csv` (assign owner/status). Client reviews ONLY the clean, P0-free entries, against a defect report — not raw JSON. *DoD:* P1 triaged, decisions logged in `templates/decision-log.md`.
|
||||
- **G4 안정화** — Fix → re-run → confirm no regressions. Spot-check a sample in Google Rich Results Test (online, outside this runtime). *DoD:* P0=0, P1 accepted/closed, online validator green on sample.
|
||||
- **G5 런칭 후** — Mode B live audit + GSC "Rich results" report monitoring. *DoD:* deployed schema matches authored dataset; no new GSC structured-data errors.
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
# Mode A — validate an authored dataset (the common case)
|
||||
python scripts/validate_schema.py path/to/schema_dataset.xlsx \
|
||||
--url-list path/to/URLlist.xlsx \
|
||||
--out schema_qa_out
|
||||
|
||||
# Highest signal for the pre-review gate (unexpected props -> P1, drop optional recommended)
|
||||
python scripts/validate_schema.py dataset.csv --strict --no-recommended --out qa_strict
|
||||
|
||||
# Try it on the bundled flawed fixture first
|
||||
python scripts/make_sample.py
|
||||
python scripts/validate_schema.py fixtures/sample_schema.csv --out demo_out
|
||||
```
|
||||
|
||||
**Input expectations (Mode A tabular):** the loader auto-detects a JSON-LD column
|
||||
(`jsonld`, `schema`, `structured_data`, `스키마`, …) plus optional `url`/`메뉴 URL`,
|
||||
`lang`/`언어코드`, `device`/`PC/MOBILE`, `page_type` columns. Multi-sheet `.xlsx`
|
||||
is supported (each sheet with a JSON-LD column is read). No JSON-LD column → clear error.
|
||||
|
||||
## Reading the output
|
||||
|
||||
- `report.md` — counts, **gate decision (PASS = zero P0)**, defects-by-category, top P0 entries, next step.
|
||||
- `defect_log.csv` — one row per finding with `status/owner/note` columns ready for triage. This is the client-facing artifact (open issues, not raw schema).
|
||||
- `results.json` — full machine-readable results for dashboards / CI.
|
||||
|
||||
**The rule:** an entry advances to client review only when it has **zero P0**. P1 =
|
||||
triage backlog (fix before launch). P2 = optimization backlog (recommended props, style).
|
||||
|
||||
## Limits & honesty
|
||||
|
||||
- Offline by design — the runtime can't reach schema.org or Google. The bundled
|
||||
rule set (`scripts/schema_rules.json`) is a curated hotel-focused subset; unknown
|
||||
types/properties degrade to warnings (never hard errors) to avoid false positives.
|
||||
- Authoritative rich-result eligibility still requires Google's online testers on a
|
||||
sample at G4. This skill makes that sample small and clean, not redundant.
|
||||
- Adding a new schema type or tightening a rule = edit `schema_rules.json` only.
|
||||
|
||||
## Integration
|
||||
|
||||
`seo-comprehensive-audit` calls this skill as pipeline stage 4 ("Schema Validation").
|
||||
For that orchestrator, run **Mode B** on a sample of live URLs and return the score
|
||||
(100 − weighted defect penalty) and the issue list. For day-to-day client work, run
|
||||
**Mode A** on the authored dataset.
|
||||
@@ -1,148 +1,57 @@
|
||||
# CLAUDE.md
|
||||
# CLAUDE.md — seo-schema-validator (Claude Code)
|
||||
|
||||
## Overview
|
||||
## Canonical entry point
|
||||
|
||||
Structured data validator: extract, parse, and validate JSON-LD, Microdata, and RDFa markup against schema.org vocabulary.
|
||||
This skill was upgraded to a **5-layer dataset-QA pipeline**. The authoritative
|
||||
directive and run instructions live in the skill root:
|
||||
|
||||
## Quick Start
|
||||
- **`../SKILL.md`** — modes, the 5 layers, stage gates, how to run.
|
||||
- **`../scripts/validate_schema.py`** — the validator (run this, not the legacy script below).
|
||||
- **`../scripts/schema_rules.json`** — the offline rule set (edit this to add a type/rule).
|
||||
- **`../references/`** — `validation-methodology.md`, `defect-taxonomy.md`, `hotel-type-map.md`.
|
||||
- **`../templates/`** — `client-qa-report-template.md`, `decision-log.md`.
|
||||
|
||||
```bash
|
||||
pip install -r scripts/requirements.txt
|
||||
python scripts/schema_validator.py --url https://example.com
|
||||
# Primary use — QA an AUTHORED dataset before the client sees it (Mode A)
|
||||
python ../scripts/validate_schema.py DATASET.xlsx --url-list URLLIST.xlsx --out schema_qa_out
|
||||
|
||||
# Highest-signal pre-review gate
|
||||
python ../scripts/validate_schema.py DATASET.csv --strict --no-recommended --out qa_strict
|
||||
|
||||
# Try the bundled flawed fixture first
|
||||
python ../scripts/make_sample.py
|
||||
python ../scripts/validate_schema.py ../fixtures/sample_schema.csv --out demo_out
|
||||
|
||||
# Post-deploy live audit (Mode B) — feeds seo-comprehensive-audit stage 4
|
||||
python ../scripts/validate_schema.py --live https://example.com --out live_out
|
||||
```
|
||||
|
||||
## Scripts
|
||||
Gate rule: **PASS = zero P0.** The process exits 1 when the gate fails, so it stops
|
||||
`&&` chains and CI. Only P0-free entries advance to client review.
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `schema_validator.py` | Extract and validate structured data |
|
||||
| `base_client.py` | Shared utilities |
|
||||
## Legacy single-URL tool (kept for quick one-offs)
|
||||
|
||||
## Usage
|
||||
`scripts/schema_validator.py --url <URL>` extracts and validates structured data from
|
||||
one live page (JSON-LD / Microdata / RDFa via extruct). It predates the pipeline and is
|
||||
**not** the gate. For any dataset or client-facing QA, use `validate_schema.py` above.
|
||||
|
||||
```bash
|
||||
# Validate page schema
|
||||
python scripts/schema_validator.py --url https://example.com
|
||||
|
||||
# JSON output
|
||||
pip install -r scripts/requirements.txt # extruct, jsonschema, rdflib, lxml, requests
|
||||
python scripts/schema_validator.py --url https://example.com --json
|
||||
|
||||
# Validate local file
|
||||
python scripts/schema_validator.py --file schema.json
|
||||
|
||||
# Check Rich Results eligibility
|
||||
python scripts/schema_validator.py --url https://example.com --rich-results
|
||||
```
|
||||
|
||||
## Supported Formats
|
||||
## Notion output (OurDigital SEO Audit Log)
|
||||
|
||||
| Format | Detection |
|
||||
|--------|-----------|
|
||||
| JSON-LD | `<script type="application/ld+json">` |
|
||||
| Microdata | `itemscope`, `itemtype`, `itemprop` |
|
||||
| RDFa | `vocab`, `typeof`, `property` |
|
||||
|
||||
## Validation Levels
|
||||
|
||||
### 1. Syntax Validation
|
||||
- Valid JSON structure
|
||||
- Proper nesting
|
||||
- No syntax errors
|
||||
|
||||
### 2. Schema.org Vocabulary
|
||||
- Valid @type values
|
||||
- Known properties
|
||||
- Correct property types
|
||||
|
||||
### 3. Google Rich Results
|
||||
- Required properties present
|
||||
- Recommended properties
|
||||
- Feature-specific requirements
|
||||
|
||||
## Schema Types Validated
|
||||
|
||||
| Type | Required Properties | Rich Result |
|
||||
|------|---------------------|-------------|
|
||||
| Article | headline, author, datePublished | Yes |
|
||||
| Product | name, offers | Yes |
|
||||
| LocalBusiness | name, address | Yes |
|
||||
| FAQPage | mainEntity | Yes |
|
||||
| Organization | name, url | Yes |
|
||||
| BreadcrumbList | itemListElement | Yes |
|
||||
| WebSite | name, url | Sitelinks |
|
||||
|
||||
## Output
|
||||
|
||||
```json
|
||||
{
|
||||
"url": "https://example.com",
|
||||
"schemas_found": 3,
|
||||
"schemas": [
|
||||
{
|
||||
"@type": "Organization",
|
||||
"valid": true,
|
||||
"rich_results_eligible": true,
|
||||
"issues": [],
|
||||
"warnings": []
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"valid": 3,
|
||||
"invalid": 0,
|
||||
"rich_results_eligible": 2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Issue Severity
|
||||
|
||||
| Level | Description |
|
||||
|-------|-------------|
|
||||
| Error | Invalid schema, blocks rich results |
|
||||
| Warning | Missing recommended property |
|
||||
| Info | Optimization suggestion |
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
extruct>=0.16.0
|
||||
jsonschema>=4.21.0
|
||||
rdflib>=7.0.0
|
||||
lxml>=5.1.0
|
||||
requests>=2.31.0
|
||||
```
|
||||
|
||||
## Notion Output (Required)
|
||||
|
||||
**IMPORTANT**: All audit reports MUST be saved to the OurDigital SEO Audit Log database.
|
||||
|
||||
### Database Configuration
|
||||
When a run is part of an OurDigital/D.intelligence audit, log a summary to the SEO Audit
|
||||
Log database. Per the user-level Notion rule, push **page content** with the
|
||||
`notion-writer` skill; use Notion MCP only for **properties** (Status, Category, etc.).
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Database ID | `2c8581e5-8a1e-8035-880b-e38cefc2f3ef` |
|
||||
| URL | https://www.notion.so/dintelligence/2c8581e58a1e8035880be38cefc2f3ef |
|
||||
|
||||
### Required Properties
|
||||
|
||||
| Property | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| Issue | Title | Report title (Korean + date) |
|
||||
| Site | URL | Audited website URL |
|
||||
| Category | Select | Technical SEO, On-page SEO, Performance, Schema/Structured Data, Sitemap, Robots.txt, Content, Local SEO |
|
||||
| Priority | Select | Critical, High, Medium, Low |
|
||||
| Found Date | Date | Audit date (YYYY-MM-DD) |
|
||||
| Audit ID | Rich Text | Format: [TYPE]-YYYYMMDD-NNN |
|
||||
|
||||
### Language Guidelines
|
||||
|
||||
- Report content in Korean (한국어)
|
||||
- Keep technical English terms as-is (e.g., SEO Audit, Core Web Vitals, Schema Markup)
|
||||
- URLs and code remain unchanged
|
||||
|
||||
### Example MCP Call
|
||||
|
||||
```bash
|
||||
mcp-cli call notion/API-post-page '{"parent": {"database_id": "2c8581e5-8a1e-8035-880b-e38cefc2f3ef"}, "properties": {...}}'
|
||||
```
|
||||
| Category | `Schema/Structured Data` |
|
||||
| Priority | map gate: FAIL→Critical/High, PASS-with-P1→Medium, PASS-clean→Low |
|
||||
| Audit ID | `SCHEMA-YYYYMMDD-NNN` |
|
||||
|
||||
Report content in Korean; keep technical terms (Schema, JSON-LD, rich result) and
|
||||
URLs/code unchanged.
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
url,언어코드,PC/MOBILE,page_type,스키마
|
||||
https://www.josunhotel.com/en/brand/grand,en,PC,brand-hub,"{""@context"": ""https://schema.org"", ""@type"": ""Organization"", ""@id"": ""https://www.josunhotel.com/#org"", ""name"": ""Josun Hotels & Resorts"", ""url"": ""https://www.josunhotel.com/"", ""logo"": ""https://www.josunhotel.com/logo.png"", ""sameAs"": [""https://www.instagram.com/josunhotelsandresorts/""]}"
|
||||
https://www.josunhotel.com/ko/grand,ko,PC,hotel,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", name: ""그랜드조선"",}"
|
||||
https://www.josunhotel.com/ko/grand/rooms,ko,MOBILE,rooms,"{""@context"": ""https://schema.org"", ""name"": ""디럭스룸"", ""url"": ""https://www.josunhotel.com/ko/grand/rooms""}"
|
||||
https://www.josunhotel.com/ko/palace,ko,PC,hotel,"{""@context"": ""https://example.org"", ""@type"": ""Hotel"", ""name"": ""조선팰리스"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""테헤란로 231"", ""addressLocality"": ""서울"", ""addressCountry"": ""KR""}}"
|
||||
https://www.josunhotel.com/ko/lescape,ko,PC,hotel,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", ""name"": ""레스케이프 호텔"", ""telephone"": ""+82-2-317-4000"", ""description"": ""조선호텔앤리조트가 운영하는 럭셔리 호텔로, 도심 속에서 품격 있는 휴식을 제공합니다. 최상의 서비스와 시설을 경험하실 수 있습니다.""}"
|
||||
https://www.josunhotel.com/ko/grand/dining,ko,PC,restaurant,"{""@context"": ""https://schema.org"", ""@type"": ""Restaurant"", ""name"": ""예시 레스토랑"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""수정필요"", ""addressCountry"": ""KR""}, ""servesCuisine"": ""Korean""}"
|
||||
https://www.josunhotel.com/ko/westin,ko,PC,hotel,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", ""name"": ""웨스틴 조선 서울"", ""telephone"": ""+82-2-771-0500"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""소공로 106"", ""addressLocality"": ""서울"", ""addressCountry"": ""KR""}, ""description"": ""조선호텔앤리조트가 운영하는 럭셔리 호텔로, 도심 속에서 품격 있는 휴식을 제공합니다. 최상의 서비스와 시설을 경험하실 수 있습니다.""}"
|
||||
https://www.josunhotel.com/en/westin,en,PC,hotel,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", ""name"": ""웨스틴 조선 서울"", ""telephone"": ""+82-2-771-9999"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""소공로 106"", ""addressLocality"": ""Seoul"", ""addressCountry"": ""KR""}, ""description"": ""조선호텔앤리조트가 운영하는 럭셔리 호텔로, 도심 속에서 품격 있는 휴식을 제공합니다. 최상의 서비스와 시설을 경험하실 수 있습니다.""}"
|
||||
https://www.josunhotel.com/ko,ko,PC,home,"{""@context"": ""https://schema.org"", ""@type"": ""WebSite"", ""name"": ""조선호텔앤리조트"", ""url"": ""https://www.josunhotel.com/"", ""publisher"": {""@id"": ""https://www.josunhotel.com/#missing-org""}}"
|
||||
https://www.josunhotel.com/ko/grand/location,ko,PC,location,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", ""name"": ""그랜드 조선 부산"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""동백로 60"", ""addressLocality"": ""부산"", ""addressCountry"": ""KR""}, ""geo"": {""@type"": ""GeoCoordinates"", ""latitude"": 129.1603, ""longitude"": 35.1586}}"
|
||||
https://www.josunhotel.com/ko/offers/spring,ko,PC,offer,"{""@context"": ""https://schema.org"", ""@type"": ""Offer"", ""price"": ""350000"", ""priceCurrency"": ""KRW"", ""validFrom"": ""2026년 3월 1일"", ""url"": ""https://www.josunhotel.com/ko/offers/spring""}"
|
||||
https://www.josunhotel.com/ko/offers/dining,ko,PC,offer,"{""@context"": ""https://schema.org"", ""@type"": ""Offer"", ""price"": ""120000"", ""priceCurrency"": ""₩"", ""availability"": ""https://schema.org/InStock""}"
|
||||
https://www.josunhotel.com/ko/spa,ko,PC,facility,"{""@context"": ""https://schema.org"", ""@type"": ""SpaResort"", ""name"": ""조선 스파""}"
|
||||
https://www.josunhotel.com/ko/grand/intro,ko,PC,hotel,"{""@context"": ""https://schema.org"", ""@type"": ""Hotel"", ""name"": ""그랜드 조선 제주"", ""address"": {""@type"": ""PostalAddress"", ""streetAddress"": ""중문관광로 75"", ""addressLocality"": ""제주"", ""addressCountry"": ""KR""}, ""description"": ""조선호텔앤리조트가 운영하는 럭셔리 호텔로, 도심 속에서 품격 있는 휴식을 제공합니다. 최상의 서비스와 시설을 경험하실 수 있습니다.""}"
|
||||
https://www.josunhotel.com/ko/faq?stale=1,ko,MOBILE,faq,"{""@context"": ""https://schema.org"", ""@type"": ""FAQPage"", ""mainEntity"": [{""@type"": ""Question"", ""name"": ""체크인 시간은 언제인가요?"", ""acceptedAnswer"": {""@type"": ""Answer"", ""text"": ""오후 3시부터 체크인 가능합니다.""}}]}"
|
||||
|
@@ -0,0 +1,74 @@
|
||||
# Defect Taxonomy
|
||||
|
||||
Every code `validate_schema.py` can emit, its default severity, and what to do.
|
||||
The validator writes these to `defect_log.csv` (columns: `entry_id, url, node_type,
|
||||
layer, code, severity, message, status, owner, note`) and `results.json`.
|
||||
|
||||
## Severity model
|
||||
|
||||
| Severity | Definition | Owner action |
|
||||
|---|---|---|
|
||||
| **P0** | Blocker — breaks parsing, blocks the rich result, or ships wrong/placeholder data. **Fails the gate.** | Must fix before the entry reaches client review. |
|
||||
| **P1** | Real defect, doesn't block the rich result. | Fix before launch; track in the triage log. |
|
||||
| **P2** | Optimization — recommended properties, formatting, orphan URLs. | Backlog; fix opportunistically. |
|
||||
|
||||
`--strict` promotes vocabulary/format warnings (and unknown types) from P2 to P1 and
|
||||
turns on `UNEXPECTED_PROPERTY`. `--no-recommended` drops `MISSING_RECOMMENDED` entirely.
|
||||
**Neither changes the gate — the gate is always "zero P0."**
|
||||
|
||||
## Code reference
|
||||
|
||||
### Layer 0 — Coverage
|
||||
| Code | Sev | Trigger | Fix |
|
||||
|---|---|---|---|
|
||||
| `COVERAGE_MISSING` | P1 | Inventory URL has no authored entry. | Author the entry, or remove the URL from the inventory. |
|
||||
| `COVERAGE_ORPHAN` | P2 | Entry URL isn't in the inventory. | Fix the URL typo, or update the canonical list. |
|
||||
|
||||
### Layer 1 — Syntax
|
||||
| Code | Sev | Trigger | Fix |
|
||||
|---|---|---|---|
|
||||
| `INVALID_JSON` | P0 | JSON does not parse. | Fix the JSON (trailing comma, unquoted key, smart quotes). |
|
||||
| `NO_SCHEMA_IN_HTML` | P0 | Live page has no `ld+json` block (Mode B). | Confirm the tag deployed and renders. |
|
||||
| `MISSING_CONTEXT` | P1 | No top-level `@context`. | Add `"@context": "https://schema.org"`. |
|
||||
| `WRONG_CONTEXT` | P1 | `@context` isn't schema.org. | Correct the context URL. |
|
||||
| `NO_TYPE` | P1 | No `@type` anywhere in the entry. | Add the intended `@type`. |
|
||||
| `ENCODING_CORRUPTION` | P1 | Replacement char `<60>` present. | Re-export as UTF-8; check the source pipeline. |
|
||||
| `FETCH_ERROR` | P1 | Live URL could not be fetched (Mode B). | Check the URL/network; retry. |
|
||||
|
||||
### Layer 2 — Vocabulary & value formats
|
||||
| Code | Sev (strict) | Trigger | Fix |
|
||||
|---|---|---|---|
|
||||
| `UNKNOWN_TYPE` | P2 (P1) | `@type` not in the curated rule set. | If intended, add it to `schema_rules.json`; else correct the type. |
|
||||
| `UNEXPECTED_PROPERTY` | — (P1) | Property unknown for a known type (**strict only**). | Remove the typo'd property, or add it to the type's `allowed`. |
|
||||
| `BAD_URL` | P2 (P1) | A URL property isn't an `http(s)` URL. | Use an absolute URL. |
|
||||
| `BAD_DATE` | P2 (P1) | A date property isn't ISO-8601. | Use `YYYY-MM-DD` (or full datetime). |
|
||||
| `BAD_LANG` | P2 (P1) | `inLanguage`/`availableLanguage` isn't a BCP-47 code. | Use `ko`, `en`, `ja`, `zh`, … |
|
||||
| `BAD_CURRENCY` | P2 (P1) | `priceCurrency` isn't a 3-letter ISO-4217 code. | Use `KRW`/`USD`, not `₩`/`$`. |
|
||||
| `BAD_NUMBER` | P2 (P1) | A numeric property isn't numeric. | Remove units/commas; keep digits. |
|
||||
|
||||
### Layer 3 — Rich-result requirements
|
||||
| Code | Sev | Trigger | Fix |
|
||||
|---|---|---|---|
|
||||
| `MISSING_REQUIRED` | P0 | A Google-required property is absent. | Add the property — the rich result is blocked without it. |
|
||||
| `MISSING_RECOMMENDED` | P2 | Recommended properties absent (one line per node, lists all). | Add what applies to improve eligibility/appearance. |
|
||||
|
||||
### Layer 4 — Consistency
|
||||
| Code | Sev | Trigger | Fix |
|
||||
|---|---|---|---|
|
||||
| `PLACEHOLDER_TEXT` | P0 | Boilerplate token in a string (`예시`, `수정필요`, `lorem`, `{{`, …). | Replace with real content. |
|
||||
| `NAP_PHONE_MISMATCH` | P0 | Same business, different `telephone` across entries. | Reconcile to the canonical phone. |
|
||||
| `NAP_ADDRESS_MISMATCH` | P0 | Same business, different `streetAddress`. | Reconcile to the canonical address. |
|
||||
| `DUPLICATE_ID` | P1 | One `@id` defined ≥2× with differing content. | Make definitions identical, or split the `@id`. |
|
||||
| `DANGLING_ID` | P1 | `{"@id": …}` reference to a node never defined. | Define the node, or fix the reference. |
|
||||
| `GEO_SWAPPED` | P1 | latitude/longitude transposed (swapping fixes it). | Swap the values. |
|
||||
| `GEO_OUT_OF_RANGE` | P1 | Coordinates impossible (lat∉[-90,90] or lon∉[-180,180]). | Correct the coordinates. |
|
||||
| `DUPLICATE_DESCRIPTION` | P1 | Same description reused across ≥3 entries. | Write distinct descriptions per page. |
|
||||
|
||||
## Triage workflow
|
||||
|
||||
1. Sort `defect_log.csv` by severity (already sorted P0→P1→P2 on write).
|
||||
2. **P0:** assign an owner, fix, re-run. No P0 may survive into client review.
|
||||
3. **P1:** set `owner` + `status`, decide fix-now vs accept; log accepted ones in
|
||||
`templates/decision-log.md`.
|
||||
4. **P2:** schedule into the optimization backlog.
|
||||
5. Re-run after fixes and confirm no regressions before advancing the stage gate.
|
||||
@@ -0,0 +1,68 @@
|
||||
# Hotel Page-Type → Schema-Type Map
|
||||
|
||||
The G1 (설계) deliverable: every page template gets an assigned schema `@type` and a
|
||||
required-property list *before* anyone authors entries. Locking this map first is what
|
||||
prevents the most expensive error — authoring hundreds of entries against the wrong type.
|
||||
|
||||
This is the reusable, hotel-domain map. The worked example is JHR (Josun Hotels &
|
||||
Resorts, `josunhotel.com`) — a multi-brand, multi-language, multi-property group — but
|
||||
the mapping applies to any lodging-group site (it replaces the earlier client-specific
|
||||
draft). Adapt the brand/property layer to the client; the page-type → type rules are stable.
|
||||
|
||||
## Site shape this map assumes
|
||||
|
||||
```
|
||||
대표 허브 (group) → Organization + WebSite (one canonical node set, @id-anchored)
|
||||
브랜드 허브 (brand) → Brand / Organization + Hotel families
|
||||
개별 호텔 (property) → Hotel / LodgingBusiness / Resort
|
||||
├─ 객실 (rooms) → HotelRoom / Suite (nested or itemList)
|
||||
├─ 다이닝 (dining) → Restaurant / BarOrPub
|
||||
├─ 시설·웨딩·연회 → LocalBusiness / MeetingRoom (nested)
|
||||
├─ 프로모션 (offers) → Offer / AggregateOffer
|
||||
└─ FAQ / 안내 → FAQPage
|
||||
```
|
||||
|
||||
Each rendered URL also multiplies by **language × device** (ko/en/ja/zh × PC/MOBILE),
|
||||
which is why the entry count reaches the thousands. The schema `@type` does **not**
|
||||
change across language/device — only the localized string values do. (Use that fact:
|
||||
NAP, geo, and `@id` must stay identical across the language variants of one property;
|
||||
Layer 4 will catch it when they drift.)
|
||||
|
||||
## The map
|
||||
|
||||
| Page template (Korean / English) | Primary `@type` | Required (P0) | Add these (recommended) |
|
||||
|---|---|---|---|
|
||||
| 대표 홈 / group home | `WebSite` (+ `Organization`) | name, url / name, url | publisher, potentialAction(SearchAction), inLanguage / logo, sameAs |
|
||||
| 브랜드 허브 / brand hub | `Organization` (+ `Brand`) | name, url | logo, sameAs, brand |
|
||||
| 호텔 메인 / property home | `Hotel` (or `LodgingBusiness`, `Resort`) | name, address | telephone, image, priceRange, geo, url, starRating, aggregateRating |
|
||||
| 객실 목록·상세 / rooms | `Hotel` w/ `containsPlace`→`HotelRoom`/`Suite`, or `ItemList` | name, address (host) | image, occupancy, bed, amenityFeature |
|
||||
| 다이닝 / restaurant | `Restaurant` (or `BarOrPub`) | name, address | servesCuisine, priceRange, telephone, menu, openingHoursSpecification, acceptsReservations |
|
||||
| 부대시설·웨딩·연회 / facilities | `LocalBusiness` w/ `MeetingRoom` nested | name, address | telephone, openingHoursSpecification, image, url |
|
||||
| 프로모션·패키지 / offers | `Offer` (or `AggregateOffer`) | price, priceCurrency / lowPrice, priceCurrency | availability, url, validFrom, priceValidUntil |
|
||||
| 멤버십 / membership | `MemberProgram` | name | hasTiers, hostingOrganization, url |
|
||||
| 위치·오시는 길 / location | `Hotel` w/ `geo`→`GeoCoordinates` | name, address | geo (lat/long), hasMap |
|
||||
| FAQ / 자주 묻는 질문 | `FAQPage` → `Question`/`Answer` | mainEntity / name, acceptedAnswer / text | — |
|
||||
| 공지·매거진·기사 / article | `Article` / `NewsArticle` / `BlogPosting` | headline | author, datePublished, image, dateModified, publisher |
|
||||
| 모든 하위 페이지 / breadcrumbs | `BreadcrumbList` → `ListItem` | itemListElement / position | item, name |
|
||||
|
||||
## Conventions that keep Layer 4 green
|
||||
|
||||
- **Anchor shared nodes by `@id`.** Define `Organization` and `WebSite` once
|
||||
(`https://…/#organization`, `https://…/#website`) and reference them everywhere with
|
||||
`{"@id": "…"}`. Avoids `DANGLING_ID` (define before you reference) and `DUPLICATE_ID`
|
||||
(don't redefine with different content).
|
||||
- **One canonical NAP per property.** The same `telephone` and `streetAddress` must
|
||||
appear in every language/device variant of a property, or `NAP_*_MISMATCH` (P0) fires.
|
||||
- **Distinct descriptions.** A reused boilerplate description across ≥3 pages →
|
||||
`DUPLICATE_DESCRIPTION` (P1). Write per-page copy.
|
||||
- **geo as `{latitude, longitude}`** in decimal degrees; Korea is lat ≈ 33–39, lon ≈
|
||||
124–132. Transposing them trips `GEO_SWAPPED`.
|
||||
- **No placeholders.** `예시 / 수정필요 / 임시 / lorem / {{…}}` anywhere → `PLACEHOLDER_TEXT`
|
||||
(P0). The gate exists precisely to stop these from reaching the client.
|
||||
|
||||
## Using the map in the lifecycle
|
||||
|
||||
- **G1 설계:** fill this table for the client's actual templates; that *is* the schema
|
||||
spec. DoD: every template has a type + required list.
|
||||
- **G2 개발:** authors produce entries against it; `--strict` run; zero P0 to advance.
|
||||
- **G3+:** the map is the reference reviewers and the validator agree on.
|
||||
@@ -0,0 +1,123 @@
|
||||
# Validation Methodology
|
||||
|
||||
The reasoning behind the 5 layers, and the type → requirement matrix the validator
|
||||
enforces. The matrix is the human-readable mirror of `scripts/schema_rules.json` —
|
||||
if you change one, change the other.
|
||||
|
||||
## Why a machine gate before human review
|
||||
|
||||
At a few dozen entries, a person can eyeball JSON-LD. At hundreds (a multi-language,
|
||||
multi-device, multi-property hotel site easily reaches 2,000+ URLs), eyeballing
|
||||
fails in a predictable way: the reviewer drowns in *mechanical* errors (a missing
|
||||
required field, a bad date format, a typo'd URL) and never reaches the *judgement*
|
||||
errors that actually need a human (is this the right schema type for this page? is
|
||||
this description accurate?).
|
||||
|
||||
The fix is not "review harder." It is to split the work by who is best at it:
|
||||
|
||||
| Error class | Best checker | This skill |
|
||||
|---|---|---|
|
||||
| Mechanical (parse, required-present, value format, duplicate, consistency) | A script, every time | Layers 0–4, automated |
|
||||
| Judgement (type choice, copy accuracy, intent) | A human, once | Client reviews only P0-free entries |
|
||||
|
||||
So the gate runs first. **An entry reaches client review only when it has zero P0.**
|
||||
The client then reviews a clean set against a defect report — never raw JSON in a meeting.
|
||||
|
||||
## The layers, in order
|
||||
|
||||
Each layer assumes the previous one passed for that entry. A fatal L1 failure
|
||||
(unparseable JSON, no `@type`) stops deeper layers for that entry — there is nothing
|
||||
to inspect.
|
||||
|
||||
### L0 — Coverage (needs `--url-list`)
|
||||
Compares the canonical URL inventory against the URLs that actually have an entry.
|
||||
- `COVERAGE_MISSING` (P1): inventory URL with no authored entry — a gap to fill.
|
||||
- `COVERAGE_ORPHAN` (P2): entry whose URL isn't in the inventory — a typo, a stale
|
||||
path, or a list that's out of date. (Expect many orphans if your inventory is a
|
||||
subset; expect ~zero when it's the real canonical list.)
|
||||
|
||||
### L1 — Syntax
|
||||
The cheapest, hardest blockers. If these fail, nothing downstream is trustworthy.
|
||||
- `INVALID_JSON` (P0), `NO_SCHEMA_IN_HTML` (P0, live mode).
|
||||
- `MISSING_CONTEXT` / `WRONG_CONTEXT` / `NO_TYPE` / `ENCODING_CORRUPTION` (P1).
|
||||
|
||||
### L2 — Vocabulary & value formats
|
||||
Is the type known, and are values well-formed?
|
||||
- `UNKNOWN_TYPE` (P2; P1 in `--strict`): type isn't in the curated rule set. A
|
||||
*warning*, not an error — add it to `schema_rules.json` if it's intended.
|
||||
- `BAD_URL` / `BAD_DATE` / `BAD_LANG` / `BAD_CURRENCY` / `BAD_NUMBER` (P2; P1 strict).
|
||||
- `UNEXPECTED_PROPERTY` (P1, `--strict` only): a property not known for a known type.
|
||||
**Off by default** — flagging every unexpected property offline produces exactly the
|
||||
false-positive flood that makes reviewers distrust the tool.
|
||||
|
||||
### L3 — Rich-result requirements
|
||||
The contract Google enforces for eligibility.
|
||||
- `MISSING_REQUIRED` (P0): a required property is absent → the rich result is blocked.
|
||||
- `MISSING_RECOMMENDED` (P2): recommended properties absent. **Aggregated to one line
|
||||
per node** (never one defect per property) — this is the single most important
|
||||
noise-control decision in the tool.
|
||||
|
||||
### L4 — Consistency (cross-node / cross-entry)
|
||||
The errors a per-entry check can't see.
|
||||
- `PLACEHOLDER_TEXT` (P0): boilerplate that escaped authoring (`예시`, `수정필요`,
|
||||
`lorem`, `{{`, …). Almost always a real, embarrassing leak.
|
||||
- `NAP_PHONE_MISMATCH` / `NAP_ADDRESS_MISMATCH` (P0): the same business shows
|
||||
different Name/Address/Phone across entries — a local-SEO and trust problem.
|
||||
- `DUPLICATE_ID` (P1): one `@id` defined twice with different content.
|
||||
- `DANGLING_ID` (P1): a `{"@id": …}` reference points at a node never defined.
|
||||
- `GEO_SWAPPED` / `GEO_OUT_OF_RANGE` (P1): latitude/longitude transposed or impossible.
|
||||
- `DUPLICATE_DESCRIPTION` (P1): the same description reused across ≥3 entries.
|
||||
|
||||
## Severity → gate
|
||||
|
||||
| Severity | Meaning | Gate effect |
|
||||
|---|---|---|
|
||||
| **P0** | Blocker. Breaks parsing, blocks the rich result, or publishes wrong data. | **Fails the gate.** Process exits 1. Entry must not reach client review. |
|
||||
| **P1** | Fix before launch. Real defect, doesn't block the rich result. | Triage backlog. |
|
||||
| **P2** | Optimization. Recommended props, style, orphan URLs. | Optimization backlog. |
|
||||
|
||||
Full code list: `defect-taxonomy.md`.
|
||||
|
||||
## Type → requirement matrix (mirror of `schema_rules.json`)
|
||||
|
||||
`required` missing → **P0**. `recommended` missing → **P2** (aggregated). Anything in
|
||||
`allowed` is accepted silently. Properties outside all three are flagged only in `--strict`.
|
||||
|
||||
| Type | Required (P0 if missing) | Recommended (P2 if missing) |
|
||||
|---|---|---|
|
||||
| Organization | name, url | logo, sameAs, contactPoint, address |
|
||||
| WebSite | name, url | publisher, potentialAction, inLanguage |
|
||||
| WebPage | name | url, isPartOf, primaryImageOfPage, breadcrumb, datePublished, dateModified |
|
||||
| Hotel / LodgingBusiness / Resort | name, address | telephone, image, priceRange, geo, url, starRating, aggregateRating |
|
||||
| LocalBusiness | name, address | telephone, openingHoursSpecification, geo, image, url, priceRange, aggregateRating |
|
||||
| Restaurant / FoodEstablishment | name, address | servesCuisine, priceRange, telephone, menu, openingHoursSpecification |
|
||||
| FAQPage | mainEntity | — |
|
||||
| Question | name, acceptedAnswer | — |
|
||||
| Answer | text | — |
|
||||
| BreadcrumbList / ItemList | itemListElement | — |
|
||||
| ListItem | position | item, name |
|
||||
| Product | name | image, offers, brand, aggregateRating, review, description, sku |
|
||||
| Offer | price, priceCurrency | availability, url, validFrom, priceValidUntil |
|
||||
| Article / NewsArticle / BlogPosting | headline | author, datePublished, image, dateModified, publisher |
|
||||
| Event | name, startDate, location | endDate, offers, performer, image, eventStatus, eventAttendanceMode, organizer |
|
||||
| Review | reviewRating, author | datePublished, reviewBody, itemReviewed |
|
||||
| AggregateRating | ratingValue | reviewCount, ratingCount, bestRating |
|
||||
| MemberProgram | name | hasTiers, hostingOrganization, url |
|
||||
|
||||
**Container types** (validated for value formats, but *not* for required/recommended,
|
||||
because they only ever appear nested): PostalAddress, GeoCoordinates, ImageObject,
|
||||
ContactPoint, OpeningHoursSpecification, Rating, Brand, EntryPoint, Place, OfferCatalog,
|
||||
ReserveAction, MeetingRoom, Room/HotelRoom/Suite, MemberProgramTier, Menu/MenuItem, … (full
|
||||
list in `schema_rules.json` → `container_types`).
|
||||
|
||||
## Extending the rules
|
||||
|
||||
Add a type, tighten a requirement, or recognize a new container by editing
|
||||
`scripts/schema_rules.json` **only** — no Python change needed:
|
||||
- New rich-result type → add to `known_types` with `required` / `recommended` / `allowed`.
|
||||
- New nested type to stop "unknown type" warnings → add to `container_types`.
|
||||
- New value-format property → add to the relevant `value_formats` group.
|
||||
- New placeholder token to catch → add to `placeholder_tokens`.
|
||||
|
||||
After any edit, re-run `make_sample.py` + `validate_schema.py` against the fixture to
|
||||
confirm you didn't regress.
|
||||
160
custom-skills/16-seo-schema-validator/scripts/make_sample.py
Normal file
160
custom-skills/16-seo-schema-validator/scripts/make_sample.py
Normal file
@@ -0,0 +1,160 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
make_sample.py — generate fixtures/sample_schema.csv.
|
||||
|
||||
A small, deliberately FLAWED hotel dataset (Josun-style, fictional values) that
|
||||
seeds at least one defect per validation layer. Use it to learn the tool and to
|
||||
regression-test changes to validate_schema.py or schema_rules.json:
|
||||
|
||||
python make_sample.py
|
||||
python validate_schema.py ../fixtures/sample_schema.csv --out /tmp/demo_out
|
||||
|
||||
Each row's comment names the defect(s) it is designed to trigger.
|
||||
"""
|
||||
|
||||
import csv
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
OUT = Path(__file__).resolve().parent.parent / "fixtures" / "sample_schema.csv"
|
||||
|
||||
CTX = "https://schema.org"
|
||||
SHARED_DESC = ("조선호텔앤리조트가 운영하는 럭셔리 호텔로, 도심 속에서 품격 있는 휴식을 "
|
||||
"제공합니다. 최상의 서비스와 시설을 경험하실 수 있습니다.") # >30 chars, reused 3x
|
||||
|
||||
|
||||
def jd(obj):
|
||||
return json.dumps(obj, ensure_ascii=False)
|
||||
|
||||
|
||||
# Each tuple: (url, lang, device, page_type, jsonld_string)
|
||||
ROWS = []
|
||||
|
||||
# 1) CLEAN Organization — only a recommended gap (P2 MISSING_RECOMMENDED, aggregated)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/en/brand/grand", "en", "PC", "brand-hub",
|
||||
jd({"@context": CTX, "@type": "Organization", "@id": "https://www.josunhotel.com/#org",
|
||||
"name": "Josun Hotels & Resorts", "url": "https://www.josunhotel.com/",
|
||||
"logo": "https://www.josunhotel.com/logo.png",
|
||||
"sameAs": ["https://www.instagram.com/josunhotelsandresorts/"]}),
|
||||
))
|
||||
|
||||
# 2) INVALID JSON (P0 INVALID_JSON) — trailing comma, unquoted key
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/grand", "ko", "PC", "hotel",
|
||||
'{"@context": "https://schema.org", "@type": "Hotel", name: "그랜드조선",}',
|
||||
))
|
||||
|
||||
# 3) MISSING @type (P1 NO_TYPE)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/grand/rooms", "ko", "MOBILE", "rooms",
|
||||
jd({"@context": CTX, "name": "디럭스룸", "url": "https://www.josunhotel.com/ko/grand/rooms"}),
|
||||
))
|
||||
|
||||
# 4) WRONG @context (P1 WRONG_CONTEXT)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/palace", "ko", "PC", "hotel",
|
||||
jd({"@context": "https://example.org", "@type": "Hotel", "name": "조선팰리스",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "테헤란로 231",
|
||||
"addressLocality": "서울", "addressCountry": "KR"}}),
|
||||
))
|
||||
|
||||
# 5) Hotel MISSING REQUIRED address (P0 MISSING_REQUIRED)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/lescape", "ko", "PC", "hotel",
|
||||
jd({"@context": CTX, "@type": "Hotel", "name": "레스케이프 호텔",
|
||||
"telephone": "+82-2-317-4000", "description": SHARED_DESC}),
|
||||
))
|
||||
|
||||
# 6) PLACEHOLDER text (P0 PLACEHOLDER_TEXT)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/grand/dining", "ko", "PC", "restaurant",
|
||||
jd({"@context": CTX, "@type": "Restaurant", "name": "예시 레스토랑",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "수정필요",
|
||||
"addressCountry": "KR"}, "servesCuisine": "Korean"}),
|
||||
))
|
||||
|
||||
# 7a + 7b) NAP PHONE MISMATCH (P0 NAP_PHONE_MISMATCH) — same business, two phones
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/westin", "ko", "PC", "hotel",
|
||||
jd({"@context": CTX, "@type": "Hotel", "name": "웨스틴 조선 서울",
|
||||
"telephone": "+82-2-771-0500",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "소공로 106",
|
||||
"addressLocality": "서울", "addressCountry": "KR"},
|
||||
"description": SHARED_DESC}),
|
||||
))
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/en/westin", "en", "PC", "hotel",
|
||||
jd({"@context": CTX, "@type": "Hotel", "name": "웨스틴 조선 서울",
|
||||
"telephone": "+82-2-771-9999",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "소공로 106",
|
||||
"addressLocality": "Seoul", "addressCountry": "KR"},
|
||||
"description": SHARED_DESC}),
|
||||
))
|
||||
|
||||
# 8) DANGLING @id reference (P1 DANGLING_ID) — publisher points at undefined node
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko", "ko", "PC", "home",
|
||||
jd({"@context": CTX, "@type": "WebSite", "name": "조선호텔앤리조트",
|
||||
"url": "https://www.josunhotel.com/",
|
||||
"publisher": {"@id": "https://www.josunhotel.com/#missing-org"}}),
|
||||
))
|
||||
|
||||
# 9) SWAPPED geo (P1 GEO_SWAPPED) — lat/long transposed for Seoul
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/grand/location", "ko", "PC", "location",
|
||||
jd({"@context": CTX, "@type": "Hotel", "name": "그랜드 조선 부산",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "동백로 60",
|
||||
"addressLocality": "부산", "addressCountry": "KR"},
|
||||
"geo": {"@type": "GeoCoordinates", "latitude": 129.1603, "longitude": 35.1586}}),
|
||||
))
|
||||
|
||||
# 10) BAD date (P2 BAD_DATE) in an Offer-bearing page
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/offers/spring", "ko", "PC", "offer",
|
||||
jd({"@context": CTX, "@type": "Offer", "price": "350000", "priceCurrency": "KRW",
|
||||
"validFrom": "2026년 3월 1일", "url": "https://www.josunhotel.com/ko/offers/spring"}),
|
||||
))
|
||||
|
||||
# 11) BAD currency symbol (P2 BAD_CURRENCY)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/offers/dining", "ko", "PC", "offer",
|
||||
jd({"@context": CTX, "@type": "Offer", "price": "120000", "priceCurrency": "₩",
|
||||
"availability": "https://schema.org/InStock"}),
|
||||
))
|
||||
|
||||
# 12) UNKNOWN type (P2 UNKNOWN_TYPE)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/spa", "ko", "PC", "facility",
|
||||
jd({"@context": CTX, "@type": "SpaResort", "name": "조선 스파"}),
|
||||
))
|
||||
|
||||
# 13) Third reuse of SHARED_DESC → triggers DUPLICATE_DESCRIPTION (P1) across rows 5,7a,7b,13
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/grand/intro", "ko", "PC", "hotel",
|
||||
jd({"@context": CTX, "@type": "Hotel", "name": "그랜드 조선 제주",
|
||||
"address": {"@type": "PostalAddress", "streetAddress": "중문관광로 75",
|
||||
"addressLocality": "제주", "addressCountry": "KR"},
|
||||
"description": SHARED_DESC}),
|
||||
))
|
||||
|
||||
# 14) CLEAN FAQPage — exercises a passing entry (and an inventory-orphan URL for L0 demo)
|
||||
ROWS.append((
|
||||
"https://www.josunhotel.com/ko/faq?stale=1", "ko", "MOBILE", "faq",
|
||||
jd({"@context": CTX, "@type": "FAQPage", "mainEntity": [
|
||||
{"@type": "Question", "name": "체크인 시간은 언제인가요?",
|
||||
"acceptedAnswer": {"@type": "Answer", "text": "오후 3시부터 체크인 가능합니다."}}]}),
|
||||
))
|
||||
|
||||
|
||||
def main():
|
||||
OUT.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(OUT, "w", newline="", encoding="utf-8-sig") as f:
|
||||
w = csv.writer(f)
|
||||
w.writerow(["url", "언어코드", "PC/MOBILE", "page_type", "스키마"]) # Korean aliases on purpose
|
||||
w.writerows(ROWS)
|
||||
print(f"Wrote {len(ROWS)} entries → {OUT}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
# validate_schema.py runs on the Python standard library alone for
|
||||
# CSV / JSON / JSONL / directory inputs (the offline default).
|
||||
#
|
||||
# Optional extras, installed only when you need them:
|
||||
openpyxl>=3.1 # required to read .xlsx datasets and .xlsx URL inventories
|
||||
requests>=2.31 # required only for --live (Mode B) URL fetching
|
||||
200
custom-skills/16-seo-schema-validator/scripts/schema_rules.json
Normal file
200
custom-skills/16-seo-schema-validator/scripts/schema_rules.json
Normal file
@@ -0,0 +1,200 @@
|
||||
{
|
||||
"_meta": {
|
||||
"version": "1.0",
|
||||
"scope": "Curated, hotel-focused subset of schema.org + Google rich-result requirements.",
|
||||
"intent": "Self-contained offline rules (the runtime cannot reach schema.org or Google). Unknown types/properties degrade to warnings, never hard errors, to avoid false positives. To support a new type or tighten a rule, edit THIS file only.",
|
||||
"sources": "schema.org/Hotel, schema.org/LocalBusiness, Google Search Central 'Structured data' rich-result docs (as of 2025)."
|
||||
},
|
||||
|
||||
"valid_contexts": [
|
||||
"https://schema.org",
|
||||
"http://schema.org",
|
||||
"https://schema.org/",
|
||||
"http://schema.org/",
|
||||
"https://www.schema.org",
|
||||
"http://www.schema.org"
|
||||
],
|
||||
|
||||
"global_properties": [
|
||||
"@context", "@type", "@id", "@graph", "@reverse",
|
||||
"name", "alternateName", "legalName", "description", "disambiguatingDescription",
|
||||
"url", "image", "logo", "sameAs", "identifier", "mainEntityOfPage",
|
||||
"additionalType", "subjectOf", "potentialAction", "inLanguage"
|
||||
],
|
||||
|
||||
"known_types": {
|
||||
"Organization": {
|
||||
"required": ["name", "url"],
|
||||
"recommended": ["logo", "sameAs", "contactPoint", "address"],
|
||||
"allowed": ["legalName", "foundingDate", "parentOrganization", "subOrganization", "brand", "telephone", "email", "founder", "numberOfEmployees", "memberOf", "hasMerchantReturnPolicy", "member"]
|
||||
},
|
||||
"Corporation": {
|
||||
"required": ["name", "url"],
|
||||
"recommended": ["logo", "sameAs", "address"],
|
||||
"allowed": ["legalName", "foundingDate", "parentOrganization", "tickerSymbol", "telephone", "email", "brand"]
|
||||
},
|
||||
"WebSite": {
|
||||
"required": ["name", "url"],
|
||||
"recommended": ["publisher", "potentialAction", "inLanguage"],
|
||||
"allowed": ["alternateName", "about", "copyrightHolder", "copyrightYear"]
|
||||
},
|
||||
"WebPage": {
|
||||
"required": ["name"],
|
||||
"recommended": ["url", "isPartOf", "primaryImageOfPage", "breadcrumb", "datePublished", "dateModified"],
|
||||
"allowed": ["about", "mentions", "speakable", "lastReviewed", "reviewedBy", "significantLink"]
|
||||
},
|
||||
"LocalBusiness": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["telephone", "openingHoursSpecification", "geo", "image", "url", "priceRange", "aggregateRating"],
|
||||
"allowed": ["email", "openingHours", "paymentAccepted", "currenciesAccepted", "areaServed", "hasMap", "department", "menu", "review", "containedInPlace", "containsPlace", "amenityFeature"]
|
||||
},
|
||||
"Hotel": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["telephone", "image", "priceRange", "geo", "url", "starRating", "aggregateRating", "checkinTime", "checkoutTime"],
|
||||
"allowed": ["email", "amenityFeature", "petsAllowed", "numberOfRooms", "availableLanguage", "containedInPlace", "containsPlace", "makesOffer", "brand", "currenciesAccepted", "smokingAllowed", "openingHoursSpecification", "audience", "review"]
|
||||
},
|
||||
"LodgingBusiness": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["telephone", "image", "priceRange", "geo", "url", "starRating", "aggregateRating", "checkinTime", "checkoutTime"],
|
||||
"allowed": ["email", "amenityFeature", "petsAllowed", "numberOfRooms", "availableLanguage", "containedInPlace", "containsPlace", "makesOffer", "currenciesAccepted", "smokingAllowed"]
|
||||
},
|
||||
"Resort": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["telephone", "image", "priceRange", "geo", "url", "starRating", "aggregateRating"],
|
||||
"allowed": ["email", "amenityFeature", "numberOfRooms", "containedInPlace", "containsPlace", "checkinTime", "checkoutTime"]
|
||||
},
|
||||
"Restaurant": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["servesCuisine", "priceRange", "telephone", "menu", "openingHoursSpecification", "image", "url", "geo", "acceptsReservations"],
|
||||
"allowed": ["email", "hasMenu", "starRating", "aggregateRating", "review", "containedInPlace", "smokingAllowed"]
|
||||
},
|
||||
"FoodEstablishment": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["servesCuisine", "priceRange", "telephone", "menu", "openingHoursSpecification"],
|
||||
"allowed": ["email", "hasMenu", "acceptsReservations", "containedInPlace"]
|
||||
},
|
||||
"BarOrPub": {
|
||||
"required": ["name", "address"],
|
||||
"recommended": ["telephone", "openingHoursSpecification", "priceRange", "servesCuisine"],
|
||||
"allowed": ["menu", "hasMenu", "image", "url"]
|
||||
},
|
||||
"FAQPage": {
|
||||
"required": ["mainEntity"],
|
||||
"recommended": [],
|
||||
"allowed": ["about", "headline", "datePublished", "dateModified"]
|
||||
},
|
||||
"Question": {
|
||||
"required": ["name", "acceptedAnswer"],
|
||||
"recommended": [],
|
||||
"allowed": ["text", "answerCount", "suggestedAnswer", "upvoteCount", "author"]
|
||||
},
|
||||
"Answer": {
|
||||
"required": ["text"],
|
||||
"recommended": [],
|
||||
"allowed": ["url", "upvoteCount", "author", "dateCreated"]
|
||||
},
|
||||
"BreadcrumbList": {
|
||||
"required": ["itemListElement"],
|
||||
"recommended": [],
|
||||
"allowed": ["numberOfItems", "itemListOrder"]
|
||||
},
|
||||
"ItemList": {
|
||||
"required": ["itemListElement"],
|
||||
"recommended": [],
|
||||
"allowed": ["numberOfItems", "itemListOrder"]
|
||||
},
|
||||
"ListItem": {
|
||||
"required": ["position"],
|
||||
"recommended": ["item", "name"],
|
||||
"allowed": ["url", "image", "nextItem", "previousItem"]
|
||||
},
|
||||
"Product": {
|
||||
"required": ["name"],
|
||||
"recommended": ["image", "offers", "brand", "aggregateRating", "review", "description", "sku"],
|
||||
"allowed": ["gtin", "gtin13", "gtin8", "gtin12", "mpn", "color", "material", "category", "audience", "isVariantOf", "additionalProperty", "hasMerchantReturnPolicy"]
|
||||
},
|
||||
"Offer": {
|
||||
"required": ["price", "priceCurrency"],
|
||||
"recommended": ["availability", "url", "validFrom", "priceValidUntil"],
|
||||
"allowed": ["itemCondition", "seller", "eligibleRegion", "priceSpecification", "shippingDetails", "availabilityStarts"]
|
||||
},
|
||||
"AggregateOffer": {
|
||||
"required": ["lowPrice", "priceCurrency"],
|
||||
"recommended": ["highPrice", "offerCount"],
|
||||
"allowed": ["offers", "availability"]
|
||||
},
|
||||
"Article": {
|
||||
"required": ["headline"],
|
||||
"recommended": ["author", "datePublished", "image", "dateModified", "publisher"],
|
||||
"allowed": ["articleBody", "articleSection", "wordCount", "keywords", "speakable"]
|
||||
},
|
||||
"NewsArticle": {
|
||||
"required": ["headline"],
|
||||
"recommended": ["author", "datePublished", "image", "dateModified", "publisher"],
|
||||
"allowed": ["articleBody", "dateline", "printSection"]
|
||||
},
|
||||
"BlogPosting": {
|
||||
"required": ["headline"],
|
||||
"recommended": ["author", "datePublished", "image", "dateModified", "publisher"],
|
||||
"allowed": ["articleBody", "keywords", "wordCount"]
|
||||
},
|
||||
"Event": {
|
||||
"required": ["name", "startDate", "location"],
|
||||
"recommended": ["endDate", "offers", "performer", "image", "eventStatus", "eventAttendanceMode", "organizer"],
|
||||
"allowed": ["doorTime", "previousStartDate", "typicalAgeRange", "maximumAttendeeCapacity"]
|
||||
},
|
||||
"Review": {
|
||||
"required": ["reviewRating", "author"],
|
||||
"recommended": ["datePublished", "reviewBody", "itemReviewed"],
|
||||
"allowed": ["publisher", "name"]
|
||||
},
|
||||
"AggregateRating": {
|
||||
"required": ["ratingValue"],
|
||||
"recommended": ["reviewCount", "ratingCount", "bestRating"],
|
||||
"allowed": ["worstRating", "itemReviewed"]
|
||||
},
|
||||
"MemberProgram": {
|
||||
"required": ["name"],
|
||||
"recommended": ["hasTiers", "hostingOrganization", "url"],
|
||||
"allowed": ["description", "membershipPointsEarned"]
|
||||
}
|
||||
},
|
||||
|
||||
"container_types": [
|
||||
"PostalAddress", "GeoCoordinates", "GeoShape", "ImageObject", "VideoObject",
|
||||
"ContactPoint", "OpeningHoursSpecification", "Rating", "QuantitativeValue",
|
||||
"MonetaryAmount", "PriceSpecification", "Brand", "EntryPoint", "Place",
|
||||
"OfferCatalog", "ReserveAction", "OrderAction", "SearchAction", "ViewAction",
|
||||
"MeetingRoom", "Room", "HotelRoom", "Suite", "LocationFeatureSpecification",
|
||||
"MemberProgramTier", "MobileApplication", "WebApplication", "SoftwareApplication",
|
||||
"Menu", "MenuItem", "MenuSection", "Country", "AdministrativeArea", "Duration",
|
||||
"PropertyValue", "Person", "Audience", "Language"
|
||||
],
|
||||
|
||||
"value_formats": {
|
||||
"url_props": ["url", "logo", "sameAs", "image", "contentUrl", "thumbnailUrl", "target", "urlTemplate", "installUrl", "menu", "hasMap", "downloadUrl", "embedUrl"],
|
||||
"date_props": ["datePublished", "dateModified", "dateCreated", "startDate", "endDate", "validFrom", "validThrough", "priceValidUntil", "foundingDate", "uploadDate", "availabilityStarts", "availabilityEnds", "lastReviewed", "previousStartDate"],
|
||||
"lang_props": ["inLanguage", "availableLanguage"],
|
||||
"currency_props": ["priceCurrency", "currenciesAccepted"],
|
||||
"number_props": ["price", "lowPrice", "highPrice", "ratingValue", "reviewCount", "ratingCount", "bestRating", "worstRating", "position", "numberOfRooms", "maxValue", "minValue", "offerCount"]
|
||||
},
|
||||
|
||||
"valid_currencies": ["KRW", "USD", "EUR", "JPY", "CNY", "GBP", "HKD", "SGD", "THB", "AUD", "CAD", "CHF", "TWD", "MYR", "PHP", "VND", "IDR", "INR"],
|
||||
|
||||
"valid_language_codes": ["ko", "en", "ja", "zh", "zh-CN", "zh-TW", "zh-Hans", "zh-Hant", "ko-KR", "en-US", "en-GB", "ja-JP", "fr", "de", "es", "ru", "th", "vi", "id", "ms"],
|
||||
|
||||
"placeholder_tokens": [
|
||||
"lorem ipsum", "lorem", "ipsum", "dolor sit", "todo", "tbd", "fixme",
|
||||
"xxx", "yyy", "zzz", "placeholder", "insert here", "insert text",
|
||||
"example.com", "your-domain", "yourdomain", "changeme", "sample text",
|
||||
"{{", "}}", "<insert", "[insert", "n/a", "샘플", "예시", "여기에",
|
||||
"변경필요", "수정필요", "입력필요", "내용입력", "테스트", "임시"
|
||||
],
|
||||
|
||||
"geo": {
|
||||
"lat_min": -90.0, "lat_max": 90.0,
|
||||
"lon_min": -180.0, "lon_max": 180.0,
|
||||
"kr_lat_range": [33.0, 39.0],
|
||||
"kr_lon_range": [124.0, 132.0]
|
||||
}
|
||||
}
|
||||
854
custom-skills/16-seo-schema-validator/scripts/validate_schema.py
Normal file
854
custom-skills/16-seo-schema-validator/scripts/validate_schema.py
Normal file
@@ -0,0 +1,854 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
validate_schema.py — 5-layer offline JSON-LD schema validator.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
When a client reviews hundreds of authored schema entries and says "there are too
|
||||
many errors," the root cause is almost always that nobody ran a machine lint first.
|
||||
Humans end up eyeballing raw JSON in a meeting. This tool moves every cheap,
|
||||
machine-checkable error OUT of human review and INTO an automated gate that runs
|
||||
first — so the client only ever sees clean, P0-free entries plus a defect report.
|
||||
|
||||
It is OFFLINE by design (the runtime cannot reach schema.org or Google). All rules
|
||||
live in schema_rules.json; unknown types/properties degrade to warnings, never hard
|
||||
errors, so the gate does not invent false positives.
|
||||
|
||||
THE 5 LAYERS
|
||||
------------
|
||||
L0 Coverage — URLs with no entry; entries whose URL isn't in the inventory.
|
||||
L1 Syntax — invalid JSON, bad/missing @context, missing @type, encoding corruption.
|
||||
L2 Vocabulary — unknown type, value-format errors (URL/date/lang/currency/number),
|
||||
(strict only) unexpected properties on a known type.
|
||||
L3 Rich-result — Google REQUIRED property missing (blocks rich result); recommended absent.
|
||||
L4 Consistency — NAP mismatch, @id duplicates/dangling refs, swapped geo,
|
||||
placeholder text, duplicate descriptions across entries.
|
||||
|
||||
GATE: PASS iff zero P0. Process exits 1 when the gate fails (so CI/`&&` chains stop).
|
||||
|
||||
Usage:
|
||||
python validate_schema.py DATASET [--url-list URLLIST] [--out DIR]
|
||||
[--strict] [--no-recommended]
|
||||
[--live URL ...] [--rules schema_rules.json]
|
||||
DATASET may be .xlsx / .csv (one row per entry, a JSON-LD column) / .jsonl / .json
|
||||
/ a directory of .json|.jsonld files. With --live, validate live URLs instead.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter, defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
RULES_DEFAULT = Path(__file__).resolve().parent / "schema_rules.json"
|
||||
|
||||
SEVERITY_ORDER = {"P0": 0, "P1": 1, "P2": 2}
|
||||
|
||||
# Header aliases for tabular input. Keys are normalized (lowercased, spaces removed).
|
||||
COLUMN_ALIASES = {
|
||||
"jsonld": ["jsonld", "jsonld", "json-ld", "json_ld", "schema", "schemamarkup",
|
||||
"structureddata", "structured_data", "markup", "스키마", "구조화데이터",
|
||||
"구조화된데이터", "jsonldcode", "스키마코드"],
|
||||
"url": ["url", "메뉴url", "pageurl", "주소", "링크", "loc", "uri", "캐노니컬", "canonical"],
|
||||
"lang": ["lang", "language", "언어", "언어코드", "locale", "lng"],
|
||||
"device": ["device", "pc/mobile", "pcmobile", "pc_mobile", "platform", "디바이스", "기기"],
|
||||
"page_type": ["page_type", "pagetype", "type", "페이지유형", "페이지타입", "menulevel",
|
||||
"menu_level", "메뉴레벨", "template", "템플릿", "유형"],
|
||||
}
|
||||
|
||||
URL_RE = re.compile(r"^https?://[^\s]+$", re.IGNORECASE)
|
||||
# ISO-8601 date or datetime (date, date+time, optional tz). Loose but rejects free text.
|
||||
DATE_RE = re.compile(
|
||||
r"^\d{4}-\d{2}-\d{2}"
|
||||
r"(?:[T ]\d{2}:\d{2}(?::\d{2})?(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?)?$"
|
||||
)
|
||||
LANG_RE = re.compile(r"^[a-zA-Z]{2,3}(?:-[A-Za-z0-9]{2,4})?$")
|
||||
JSONLD_SCRIPT_RE = re.compile(
|
||||
r'<script[^>]+type=["\']application/ld\+json["\'][^>]*>(.*?)</script>',
|
||||
re.IGNORECASE | re.DOTALL,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Defect collection
|
||||
# --------------------------------------------------------------------------- #
|
||||
class DefectLog:
|
||||
"""Accumulates findings. One row per finding, ready for triage."""
|
||||
|
||||
def __init__(self):
|
||||
self.rows = []
|
||||
|
||||
def add(self, severity, layer, code, message, entry_id="", url="", node_type=""):
|
||||
self.rows.append({
|
||||
"entry_id": str(entry_id),
|
||||
"url": url or "",
|
||||
"node_type": node_type or "",
|
||||
"layer": layer,
|
||||
"code": code,
|
||||
"severity": severity,
|
||||
"message": message,
|
||||
"status": "open",
|
||||
"owner": "",
|
||||
"note": "",
|
||||
})
|
||||
|
||||
def counts(self):
|
||||
c = Counter(r["severity"] for r in self.rows)
|
||||
return {"P0": c.get("P0", 0), "P1": c.get("P1", 0), "P2": c.get("P2", 0)}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Input adapters (Mode A: authored dataset / Mode B: live URLs)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _norm_header(h):
|
||||
return re.sub(r"\s+", "", str(h or "").strip().lower())
|
||||
|
||||
|
||||
def _detect_columns(headers):
|
||||
"""Map normalized headers to canonical column roles. Returns {role: index}."""
|
||||
found = {}
|
||||
for idx, h in enumerate(headers):
|
||||
nh = _norm_header(h)
|
||||
for role, aliases in COLUMN_ALIASES.items():
|
||||
if role in found:
|
||||
continue
|
||||
if nh in aliases:
|
||||
found[role] = idx
|
||||
return found
|
||||
|
||||
|
||||
def _row_to_entry(row, cols, entry_id, source_ref):
|
||||
def cell(role):
|
||||
i = cols.get(role)
|
||||
if i is None or i >= len(row):
|
||||
return None
|
||||
v = row[i]
|
||||
return None if v is None else str(v).strip()
|
||||
raw = cell("jsonld")
|
||||
if not raw:
|
||||
return None # blank JSON-LD cell → no entry to validate
|
||||
return {
|
||||
"entry_id": entry_id,
|
||||
"url": cell("url") or "",
|
||||
"lang": cell("lang") or "",
|
||||
"device": cell("device") or "",
|
||||
"page_type": cell("page_type") or "",
|
||||
"raw": raw,
|
||||
"source_ref": source_ref,
|
||||
}
|
||||
|
||||
|
||||
def _load_csv(path):
|
||||
entries = []
|
||||
with open(path, newline="", encoding="utf-8-sig") as f:
|
||||
reader = csv.reader(f)
|
||||
rows = list(reader)
|
||||
if not rows:
|
||||
return entries
|
||||
cols = _detect_columns(rows[0])
|
||||
if "jsonld" not in cols:
|
||||
raise ValueError(
|
||||
f"No JSON-LD column found in {path}. Looked for: "
|
||||
f"{', '.join(COLUMN_ALIASES['jsonld'][:6])} … Headers were: {rows[0]}"
|
||||
)
|
||||
for n, row in enumerate(rows[1:], start=2):
|
||||
e = _row_to_entry(row, cols, f"{Path(path).stem}#r{n}", f"{path}:row{n}")
|
||||
if e:
|
||||
entries.append(e)
|
||||
return entries
|
||||
|
||||
|
||||
def _load_xlsx(path):
|
||||
try:
|
||||
from openpyxl import load_workbook
|
||||
except ImportError:
|
||||
raise SystemExit(
|
||||
"Reading .xlsx needs openpyxl: pip install openpyxl\n"
|
||||
"(or export the sheet to .csv and pass that instead)."
|
||||
)
|
||||
entries = []
|
||||
wb = load_workbook(path, read_only=True, data_only=True)
|
||||
for sheet in wb.worksheets:
|
||||
rows = list(sheet.iter_rows(values_only=True))
|
||||
if not rows:
|
||||
continue
|
||||
cols = _detect_columns(rows[0])
|
||||
if "jsonld" not in cols:
|
||||
continue # a tab without a JSON-LD column (e.g. summary) — skip silently
|
||||
for n, row in enumerate(rows[1:], start=2):
|
||||
e = _row_to_entry(list(row), cols, f"{sheet.title}#r{n}",
|
||||
f"{path}:{sheet.title}:row{n}")
|
||||
if e:
|
||||
entries.append(e)
|
||||
if not entries:
|
||||
raise ValueError(
|
||||
f"No sheet in {path} had a recognizable JSON-LD column. "
|
||||
f"Looked for: {', '.join(COLUMN_ALIASES['jsonld'][:6])} …"
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def _looks_like_schema(obj):
|
||||
"""True if a parsed object is itself JSON-LD (vs a wrapper row)."""
|
||||
if isinstance(obj, list):
|
||||
return True
|
||||
if isinstance(obj, dict):
|
||||
return any(k in obj for k in ("@context", "@type", "@graph"))
|
||||
return False
|
||||
|
||||
|
||||
def _wrapper_to_entry(obj, entry_id, source_ref):
|
||||
"""A JSONL/JSON wrapper object that carries url/lang + a jsonld payload."""
|
||||
cols = {k: k for k in obj.keys()}
|
||||
norm = {_norm_header(k): k for k in obj.keys()}
|
||||
def pick(role):
|
||||
for alias in COLUMN_ALIASES[role]:
|
||||
if alias in norm:
|
||||
v = obj[norm[alias]]
|
||||
return v
|
||||
return None
|
||||
payload = pick("jsonld")
|
||||
raw = payload if isinstance(payload, str) else json.dumps(payload, ensure_ascii=False)
|
||||
url = pick("url")
|
||||
return {
|
||||
"entry_id": entry_id,
|
||||
"url": str(url).strip() if url else "",
|
||||
"lang": str(pick("lang") or "").strip(),
|
||||
"device": str(pick("device") or "").strip(),
|
||||
"page_type": str(pick("page_type") or "").strip(),
|
||||
"raw": raw,
|
||||
"source_ref": source_ref,
|
||||
}
|
||||
|
||||
|
||||
def _load_jsonl(path):
|
||||
entries = []
|
||||
with open(path, encoding="utf-8") as f:
|
||||
for n, line in enumerate(f, start=1):
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
sref = f"{path}:line{n}"
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
# Keep the bad line so L1 reports it as a syntax error.
|
||||
entries.append({"entry_id": f"{Path(path).stem}#l{n}", "url": "",
|
||||
"lang": "", "device": "", "page_type": "",
|
||||
"raw": line, "source_ref": sref})
|
||||
continue
|
||||
eid = f"{Path(path).stem}#l{n}"
|
||||
if _looks_like_schema(obj):
|
||||
entries.append({"entry_id": eid, "url": "", "lang": "", "device": "",
|
||||
"page_type": "", "raw": line, "source_ref": sref})
|
||||
else:
|
||||
entries.append(_wrapper_to_entry(obj, eid, sref))
|
||||
return entries
|
||||
|
||||
|
||||
def _load_json(path):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
entries = []
|
||||
if isinstance(data, dict) and not _looks_like_schema(data) and all(
|
||||
isinstance(v, (dict, list, str)) for v in data.values()
|
||||
) and not any(k.startswith("@") for k in data):
|
||||
# url -> jsonld map
|
||||
for url, payload in data.items():
|
||||
raw = payload if isinstance(payload, str) else json.dumps(payload, ensure_ascii=False)
|
||||
entries.append({"entry_id": url, "url": url, "lang": "", "device": "",
|
||||
"page_type": "", "raw": raw, "source_ref": f"{path}:{url}"})
|
||||
elif isinstance(data, list):
|
||||
for n, item in enumerate(data, start=1):
|
||||
sref = f"{path}:[{n}]"
|
||||
eid = f"{Path(path).stem}#{n}"
|
||||
if _looks_like_schema(item) or not isinstance(item, dict):
|
||||
raw = item if isinstance(item, str) else json.dumps(item, ensure_ascii=False)
|
||||
entries.append({"entry_id": eid, "url": "", "lang": "", "device": "",
|
||||
"page_type": "", "raw": raw, "source_ref": sref})
|
||||
else:
|
||||
entries.append(_wrapper_to_entry(item, eid, sref))
|
||||
else:
|
||||
entries.append({"entry_id": Path(path).stem, "url": "", "lang": "", "device": "",
|
||||
"page_type": "", "raw": json.dumps(data, ensure_ascii=False),
|
||||
"source_ref": path})
|
||||
return entries
|
||||
|
||||
|
||||
def _load_dir(path):
|
||||
entries = []
|
||||
for p in sorted(Path(path).rglob("*")):
|
||||
if p.suffix.lower() in (".json", ".jsonld"):
|
||||
entries.append({"entry_id": p.stem, "url": "", "lang": "", "device": "",
|
||||
"page_type": "", "raw": p.read_text(encoding="utf-8"),
|
||||
"source_ref": str(p)})
|
||||
if not entries:
|
||||
raise ValueError(f"No .json/.jsonld files found under {path}")
|
||||
return entries
|
||||
|
||||
|
||||
def _load_live(urls):
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
raise SystemExit("Live mode (--live) needs requests: pip install requests")
|
||||
entries = []
|
||||
headers = {"User-Agent": "Mozilla/5.0 (compatible; SchemaValidator/1.0)"}
|
||||
for url in urls:
|
||||
try:
|
||||
resp = requests.get(url, headers=headers, timeout=20)
|
||||
resp.raise_for_status()
|
||||
except Exception as exc: # noqa: BLE001 — best-effort live fetch
|
||||
entries.append({"entry_id": url, "url": url, "lang": "", "device": "",
|
||||
"page_type": "", "raw": "", "source_ref": url,
|
||||
"_fetch_error": str(exc)})
|
||||
continue
|
||||
scripts = JSONLD_SCRIPT_RE.findall(resp.text)
|
||||
if not scripts:
|
||||
entries.append({"entry_id": url, "url": url, "lang": "", "device": "",
|
||||
"page_type": "", "raw": "", "source_ref": url,
|
||||
"_no_schema": True})
|
||||
continue
|
||||
for i, block in enumerate(scripts, start=1):
|
||||
entries.append({"entry_id": f"{url}#{i}", "url": url, "lang": "",
|
||||
"device": "", "page_type": "", "raw": block.strip(),
|
||||
"source_ref": f"{url} (script {i})"})
|
||||
return entries
|
||||
|
||||
|
||||
def load_entries(input_path, live_urls):
|
||||
if live_urls:
|
||||
return _load_live(live_urls)
|
||||
p = Path(input_path)
|
||||
if p.is_dir():
|
||||
return _load_dir(p)
|
||||
suffix = p.suffix.lower()
|
||||
if suffix == ".csv":
|
||||
return _load_csv(p)
|
||||
if suffix in (".xlsx", ".xlsm"):
|
||||
return _load_xlsx(p)
|
||||
if suffix == ".jsonl":
|
||||
return _load_jsonl(p)
|
||||
if suffix in (".json", ".jsonld"):
|
||||
return _load_json(p)
|
||||
raise ValueError(f"Unsupported input: {input_path} (suffix {suffix!r})")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Node helpers
|
||||
# --------------------------------------------------------------------------- #
|
||||
def type_of(node):
|
||||
"""Return the primary @type as a string (first if it's a list)."""
|
||||
t = node.get("@type")
|
||||
if isinstance(t, list):
|
||||
return t[0] if t else ""
|
||||
return t or ""
|
||||
|
||||
|
||||
def iter_typed_nodes(parsed):
|
||||
"""Yield every dict that has an @type, top-level and nested (recursively)."""
|
||||
seen = []
|
||||
|
||||
def walk(obj):
|
||||
if isinstance(obj, dict):
|
||||
if "@type" in obj:
|
||||
seen.append(obj)
|
||||
for v in obj.values():
|
||||
walk(v)
|
||||
elif isinstance(obj, list):
|
||||
for v in obj:
|
||||
walk(v)
|
||||
|
||||
# @graph documents: walk the graph; otherwise walk the object/array directly.
|
||||
if isinstance(parsed, dict) and "@graph" in parsed:
|
||||
walk(parsed["@graph"])
|
||||
else:
|
||||
walk(parsed)
|
||||
return seen
|
||||
|
||||
|
||||
def all_strings(obj):
|
||||
"""Yield (key, value) for every string value anywhere in the structure."""
|
||||
if isinstance(obj, dict):
|
||||
for k, v in obj.items():
|
||||
if isinstance(v, str):
|
||||
yield k, v
|
||||
else:
|
||||
yield from all_strings(v)
|
||||
elif isinstance(obj, list):
|
||||
for v in obj:
|
||||
yield from all_strings(v)
|
||||
|
||||
|
||||
def normalize_name(s):
|
||||
return re.sub(r"\s+", " ", str(s or "").strip().lower())
|
||||
|
||||
|
||||
def first_text(value):
|
||||
"""Coerce a property value to a comparable scalar (handles list/dict)."""
|
||||
if isinstance(value, list):
|
||||
return first_text(value[0]) if value else ""
|
||||
if isinstance(value, dict):
|
||||
return value.get("name") or value.get("@id") or value.get("streetAddress") or ""
|
||||
return value
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Layer 0 — Coverage
|
||||
# --------------------------------------------------------------------------- #
|
||||
def load_url_inventory(url_list_path):
|
||||
urls = set()
|
||||
p = Path(url_list_path)
|
||||
suffix = p.suffix.lower()
|
||||
if suffix in (".xlsx", ".xlsm"):
|
||||
from openpyxl import load_workbook
|
||||
wb = load_workbook(p, read_only=True, data_only=True)
|
||||
for sheet in wb.worksheets:
|
||||
for row in sheet.iter_rows(values_only=True):
|
||||
for cell in row:
|
||||
if isinstance(cell, str) and URL_RE.match(cell.strip()):
|
||||
urls.add(cell.strip())
|
||||
elif suffix == ".csv":
|
||||
with open(p, newline="", encoding="utf-8-sig") as f:
|
||||
for row in csv.reader(f):
|
||||
for cell in row:
|
||||
if isinstance(cell, str) and URL_RE.match(cell.strip()):
|
||||
urls.add(cell.strip())
|
||||
else: # plain text, one URL per line
|
||||
for line in p.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if URL_RE.match(line):
|
||||
urls.add(line)
|
||||
return urls
|
||||
|
||||
|
||||
def layer0_coverage(entries, inventory, defects):
|
||||
entry_urls = {e["url"] for e in entries if e.get("url")}
|
||||
missing = inventory - entry_urls
|
||||
for url in sorted(missing):
|
||||
defects.add("P1", "L0", "COVERAGE_MISSING",
|
||||
"Inventory URL has no authored schema entry.", url=url)
|
||||
orphans = entry_urls - inventory
|
||||
for url in sorted(orphans):
|
||||
defects.add("P2", "L0", "COVERAGE_ORPHAN",
|
||||
"Entry URL is not in the canonical URL inventory "
|
||||
"(typo, stale path, or missing from list).", url=url)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Layer 1 — Syntax
|
||||
# --------------------------------------------------------------------------- #
|
||||
def layer1_syntax(entry, rules, defects):
|
||||
"""Parse + structural checks. Returns parsed object or None (fatal)."""
|
||||
eid, url = entry["entry_id"], entry["url"]
|
||||
if entry.get("_fetch_error"):
|
||||
defects.add("P1", "L1", "FETCH_ERROR",
|
||||
f"Could not fetch live URL: {entry['_fetch_error']}", eid, url)
|
||||
return None
|
||||
if entry.get("_no_schema"):
|
||||
defects.add("P0", "L1", "NO_SCHEMA_IN_HTML",
|
||||
"Live page has no application/ld+json script block.", eid, url)
|
||||
return None
|
||||
raw = entry["raw"]
|
||||
if "<EFBFBD>" in raw:
|
||||
defects.add("P1", "L1", "ENCODING_CORRUPTION",
|
||||
"Replacement character (\\ufffd) present — encoding corruption.",
|
||||
eid, url)
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
defects.add("P0", "L1", "INVALID_JSON",
|
||||
f"JSON does not parse: {exc.msg} at line {exc.lineno} col {exc.colno}.",
|
||||
eid, url)
|
||||
return None
|
||||
|
||||
nodes = iter_typed_nodes(parsed)
|
||||
if not nodes:
|
||||
defects.add("P1", "L1", "NO_TYPE",
|
||||
"No @type found anywhere in the entry — not a usable schema object.",
|
||||
eid, url)
|
||||
|
||||
# @context lives at the top of the document; nested nodes inherit it.
|
||||
if isinstance(parsed, dict):
|
||||
ctx = parsed.get("@context")
|
||||
if ctx is None:
|
||||
defects.add("P1", "L1", "MISSING_CONTEXT",
|
||||
"Top-level @context is missing.", eid, url)
|
||||
else:
|
||||
ctx_urls = [ctx] if isinstance(ctx, str) else (
|
||||
[c for c in ctx if isinstance(c, str)] if isinstance(ctx, list) else []
|
||||
)
|
||||
valid = rules["valid_contexts"]
|
||||
if ctx_urls and not any(c.rstrip("/") in [v.rstrip("/") for v in valid]
|
||||
for c in ctx_urls):
|
||||
defects.add("P1", "L1", "WRONG_CONTEXT",
|
||||
f"@context is not schema.org: {ctx_urls}.", eid, url)
|
||||
return parsed
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Layer 2 — Vocabulary + value formats
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _check_value_formats(node, rules, defects, eid, url, ntype, severity):
|
||||
vf = rules["value_formats"]
|
||||
|
||||
def each(value):
|
||||
if isinstance(value, list):
|
||||
for v in value:
|
||||
yield from each(v)
|
||||
else:
|
||||
yield value
|
||||
|
||||
for prop, value in node.items():
|
||||
if prop.startswith("@"):
|
||||
continue
|
||||
if prop in vf["url_props"]:
|
||||
for v in each(value):
|
||||
if isinstance(v, str) and not URL_RE.match(v.strip()):
|
||||
defects.add(severity, "L2", "BAD_URL",
|
||||
f"'{prop}' is not an http(s) URL: {v!r}.", eid, url, ntype)
|
||||
if prop in vf["date_props"]:
|
||||
for v in each(value):
|
||||
if isinstance(v, str) and not DATE_RE.match(v.strip()):
|
||||
defects.add(severity, "L2", "BAD_DATE",
|
||||
f"'{prop}' is not ISO-8601: {v!r}.", eid, url, ntype)
|
||||
if prop in vf["lang_props"]:
|
||||
for v in each(value):
|
||||
if isinstance(v, str) and not LANG_RE.match(v.strip()):
|
||||
defects.add(severity, "L2", "BAD_LANG",
|
||||
f"'{prop}' is not a BCP-47 language code: {v!r}.",
|
||||
eid, url, ntype)
|
||||
if prop in vf["currency_props"]:
|
||||
for v in each(value):
|
||||
if isinstance(v, str) and not re.match(r"^[A-Z]{3}$", v.strip()):
|
||||
defects.add(severity, "L2", "BAD_CURRENCY",
|
||||
f"'{prop}' is not a 3-letter ISO-4217 code: {v!r}.",
|
||||
eid, url, ntype)
|
||||
if prop in vf["number_props"]:
|
||||
for v in each(value):
|
||||
if isinstance(v, str):
|
||||
try:
|
||||
float(v.replace(",", ""))
|
||||
except ValueError:
|
||||
defects.add(severity, "L2", "BAD_NUMBER",
|
||||
f"'{prop}' is not numeric: {v!r}.", eid, url, ntype)
|
||||
|
||||
|
||||
def layer2_vocabulary(node, rules, defects, eid, url, strict):
|
||||
ntype = type_of(node)
|
||||
known = rules["known_types"]
|
||||
containers = set(rules["container_types"])
|
||||
minor = "P1" if strict else "P2"
|
||||
|
||||
if ntype and ntype not in known and ntype not in containers:
|
||||
defects.add(minor, "L2", "UNKNOWN_TYPE",
|
||||
f"@type '{ntype}' is not in the curated rule set "
|
||||
"(treated as a warning — add it to schema_rules.json if intended).",
|
||||
eid, url, ntype)
|
||||
|
||||
_check_value_formats(node, rules, defects, eid, url, ntype, minor)
|
||||
|
||||
# Unexpected-property check is OPT-IN (--strict). Off by default to avoid the
|
||||
# exact noise explosion that makes clients say "too many errors".
|
||||
if strict and ntype in known:
|
||||
spec = known[ntype]
|
||||
allowed = set(spec["required"]) | set(spec["recommended"]) | set(spec["allowed"])
|
||||
allowed |= set(rules["global_properties"])
|
||||
for prop in node:
|
||||
if prop.startswith("@"):
|
||||
continue
|
||||
if prop not in allowed:
|
||||
defects.add("P1", "L2", "UNEXPECTED_PROPERTY",
|
||||
f"'{prop}' is not a known property of {ntype} (strict mode).",
|
||||
eid, url, ntype)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Layer 3 — Rich-result (required / recommended)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def layer3_richresult(node, rules, defects, eid, url, no_recommended):
|
||||
ntype = type_of(node)
|
||||
known = rules["known_types"]
|
||||
if ntype not in known:
|
||||
return # containers + unknown types have no required-property contract
|
||||
spec = known[ntype]
|
||||
|
||||
for prop in spec["required"]:
|
||||
if not node.get(prop):
|
||||
defects.add("P0", "L3", "MISSING_REQUIRED",
|
||||
f"{ntype} is missing required property '{prop}' "
|
||||
"(blocks the rich result).", eid, url, ntype)
|
||||
|
||||
if not no_recommended:
|
||||
missing_rec = [p for p in spec["recommended"] if not node.get(p)]
|
||||
if missing_rec:
|
||||
# Aggregate to ONE line per node — never one defect per property.
|
||||
defects.add("P2", "L3", "MISSING_RECOMMENDED",
|
||||
f"{ntype} is missing recommended properties: "
|
||||
f"{', '.join(missing_rec)}.", eid, url, ntype)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Layer 4 — Consistency (cross-node / cross-entry)
|
||||
# --------------------------------------------------------------------------- #
|
||||
NAP_TYPES = {"Organization", "Corporation", "LocalBusiness", "Hotel",
|
||||
"LodgingBusiness", "Resort", "Restaurant", "FoodEstablishment", "BarOrPub"}
|
||||
|
||||
|
||||
def _address_street(node):
|
||||
addr = node.get("address")
|
||||
if isinstance(addr, dict):
|
||||
return normalize_name(addr.get("streetAddress"))
|
||||
if isinstance(addr, list) and addr and isinstance(addr[0], dict):
|
||||
return normalize_name(addr[0].get("streetAddress"))
|
||||
return ""
|
||||
|
||||
|
||||
def _walk_ids(obj, defined, referenced):
|
||||
"""Collect @id definitions vs pure references by walking the whole document.
|
||||
|
||||
A *reference* is an object whose only key is @id (e.g. {"@id": "...#org"}).
|
||||
A *definition* is any object carrying @id plus other content. References live
|
||||
in untyped wrapper dicts, so this must walk the raw doc — not just typed nodes.
|
||||
"""
|
||||
if isinstance(obj, dict):
|
||||
nid = obj.get("@id")
|
||||
if nid:
|
||||
if set(obj.keys()) == {"@id"}:
|
||||
referenced.add(nid)
|
||||
else:
|
||||
defined.setdefault(nid, []).append(obj)
|
||||
for v in obj.values():
|
||||
_walk_ids(v, defined, referenced)
|
||||
elif isinstance(obj, list):
|
||||
for v in obj:
|
||||
_walk_ids(v, defined, referenced)
|
||||
|
||||
|
||||
def layer4_consistency(node_index, parsed_docs, rules, defects):
|
||||
"""node_index: (entry, node) for every TYPED node.
|
||||
parsed_docs: (entry, parsed) for every entry that parsed — used for @id scan."""
|
||||
# ---- placeholder text (P0) ----
|
||||
tokens = [t.lower() for t in rules["placeholder_tokens"]]
|
||||
for entry, node in node_index:
|
||||
ntype = type_of(node)
|
||||
for key, val in all_strings(node):
|
||||
low = val.lower()
|
||||
hit = next((t for t in tokens if t in low), None)
|
||||
if hit:
|
||||
defects.add("P0", "L4", "PLACEHOLDER_TEXT",
|
||||
f"Placeholder/boilerplate token {hit!r} in '{key}': {val[:60]!r}.",
|
||||
entry["entry_id"], entry["url"], ntype)
|
||||
break # one placeholder defect per node is enough signal
|
||||
|
||||
# ---- NAP consistency (P0) ----
|
||||
by_name = defaultdict(list)
|
||||
for entry, node in node_index:
|
||||
if type_of(node) in NAP_TYPES and node.get("name"):
|
||||
by_name[normalize_name(first_text(node.get("name")))].append((entry, node))
|
||||
for name, group in by_name.items():
|
||||
phones = {str(first_text(n.get("telephone"))).strip()
|
||||
for _, n in group if n.get("telephone")}
|
||||
streets = {_address_street(n) for _, n in group if _address_street(n)}
|
||||
if len(phones) > 1:
|
||||
defects.add("P0", "L4", "NAP_PHONE_MISMATCH",
|
||||
f"Business '{name}' has conflicting telephone values across "
|
||||
f"entries: {sorted(phones)}.", entry_id="(dataset)")
|
||||
if len(streets) > 1:
|
||||
defects.add("P0", "L4", "NAP_ADDRESS_MISMATCH",
|
||||
f"Business '{name}' has conflicting streetAddress values across "
|
||||
f"entries: {sorted(streets)}.", entry_id="(dataset)")
|
||||
|
||||
# ---- @id duplicates + dangling references (P1) ----
|
||||
defined = {} # @id -> list of definition dicts (walked across all docs)
|
||||
referenced = set() # @id values used purely as references
|
||||
for _, parsed in parsed_docs:
|
||||
_walk_ids(parsed, defined, referenced)
|
||||
for nid, defs in defined.items():
|
||||
if len(defs) > 1:
|
||||
# duplicate only matters if the definitions actually differ
|
||||
shapes = {json.dumps(n, sort_keys=True, ensure_ascii=False) for n in defs}
|
||||
if len(shapes) > 1:
|
||||
defects.add("P1", "L4", "DUPLICATE_ID",
|
||||
f"@id {nid!r} is defined {len(defs)} times with differing content.",
|
||||
entry_id="(dataset)")
|
||||
for nid in sorted(referenced - set(defined)):
|
||||
defects.add("P1", "L4", "DANGLING_ID",
|
||||
f"@id reference {nid!r} points to a node that is never defined.",
|
||||
entry_id="(dataset)")
|
||||
|
||||
# ---- swapped / out-of-range geo (P1) ----
|
||||
g = rules["geo"]
|
||||
for entry, node in node_index:
|
||||
if type_of(node) != "GeoCoordinates":
|
||||
continue
|
||||
try:
|
||||
lat = float(first_text(node.get("latitude")))
|
||||
lon = float(first_text(node.get("longitude")))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
lat_ok = g["lat_min"] <= lat <= g["lat_max"]
|
||||
lon_ok = g["lon_min"] <= lon <= g["lon_max"]
|
||||
if lat_ok and lon_ok:
|
||||
continue # both in valid global range
|
||||
# Invalid — distinguish a clean transposition from plain garbage.
|
||||
swap_ok = (g["lat_min"] <= lon <= g["lat_max"]) and (g["lon_min"] <= lat <= g["lon_max"])
|
||||
if swap_ok:
|
||||
defects.add("P1", "L4", "GEO_SWAPPED",
|
||||
f"GeoCoordinates look transposed (latitude={lat}, longitude={lon}) "
|
||||
"— swapping them yields valid coordinates.",
|
||||
entry["entry_id"], entry["url"], "GeoCoordinates")
|
||||
else:
|
||||
defects.add("P1", "L4", "GEO_OUT_OF_RANGE",
|
||||
f"GeoCoordinates out of range (latitude={lat}, longitude={lon}).",
|
||||
entry["entry_id"], entry["url"], "GeoCoordinates")
|
||||
|
||||
# ---- duplicate descriptions across entries (P1) ----
|
||||
desc_groups = defaultdict(set)
|
||||
for entry, node in node_index:
|
||||
d = first_text(node.get("description"))
|
||||
if isinstance(d, str) and len(d.strip()) >= 30:
|
||||
desc_groups[d.strip()].add(entry["entry_id"])
|
||||
for desc, eids in desc_groups.items():
|
||||
if len(eids) >= 3:
|
||||
defects.add("P1", "L4", "DUPLICATE_DESCRIPTION",
|
||||
f"Identical description reused across {len(eids)} entries "
|
||||
f"(e.g. {sorted(eids)[:3]}): {desc[:50]!r}…", entry_id="(dataset)")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Orchestration + output
|
||||
# --------------------------------------------------------------------------- #
|
||||
def run(entries, rules, inventory, strict, no_recommended):
|
||||
defects = DefectLog()
|
||||
if inventory is not None:
|
||||
layer0_coverage(entries, inventory, defects)
|
||||
|
||||
node_index = []
|
||||
parsed_docs = []
|
||||
valid_entries = 0
|
||||
for entry in entries:
|
||||
parsed = layer1_syntax(entry, rules, defects)
|
||||
if parsed is None:
|
||||
continue
|
||||
valid_entries += 1
|
||||
parsed_docs.append((entry, parsed))
|
||||
for node in iter_typed_nodes(parsed):
|
||||
layer2_vocabulary(node, rules, defects, entry["entry_id"], entry["url"], strict)
|
||||
layer3_richresult(node, rules, defects, entry["entry_id"], entry["url"],
|
||||
no_recommended)
|
||||
node_index.append((entry, node))
|
||||
|
||||
layer4_consistency(node_index, parsed_docs, rules, defects)
|
||||
return defects, valid_entries, len(node_index)
|
||||
|
||||
|
||||
def write_outputs(defects, outdir, meta):
|
||||
outdir = Path(outdir)
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# defect_log.csv — the client-facing triage artifact
|
||||
fields = ["entry_id", "url", "node_type", "layer", "code", "severity",
|
||||
"message", "status", "owner", "note"]
|
||||
rows = sorted(defects.rows, key=lambda r: (SEVERITY_ORDER[r["severity"]],
|
||||
r["layer"], r["code"]))
|
||||
with open(outdir / "defect_log.csv", "w", newline="", encoding="utf-8-sig") as f:
|
||||
w = csv.DictWriter(f, fieldnames=fields)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
|
||||
counts = defects.counts()
|
||||
gate = "PASS" if counts["P0"] == 0 else "FAIL"
|
||||
by_code = Counter((r["severity"], r["code"]) for r in defects.rows)
|
||||
|
||||
# results.json — machine-readable
|
||||
results = {
|
||||
"summary": {**meta, **counts, "total": len(rows), "gate": gate},
|
||||
"by_code": [{"severity": s, "code": c, "count": n}
|
||||
for (s, c), n in by_code.most_common()],
|
||||
"defects": rows,
|
||||
}
|
||||
(outdir / "results.json").write_text(
|
||||
json.dumps(results, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
|
||||
# report.md — human summary
|
||||
lines = [
|
||||
"# Schema Validation Report", "",
|
||||
f"- Entries read: **{meta['entries']}** | parsed OK: **{meta['valid_entries']}** "
|
||||
f"| nodes checked: **{meta['nodes']}**",
|
||||
f"- Defects: **P0 {counts['P0']}** · **P1 {counts['P1']}** · **P2 {counts['P2']}** "
|
||||
f"(total {len(rows)})",
|
||||
"",
|
||||
f"## Gate: **{gate}**",
|
||||
("> ✅ Zero P0 — entries may advance to client review."
|
||||
if gate == "PASS" else
|
||||
"> ⛔ P0 present — these entries must NOT reach client review. Fix P0 first."),
|
||||
"",
|
||||
"## Defects by code", "",
|
||||
"| Severity | Code | Count |", "|---|---|---|",
|
||||
]
|
||||
for (sev, code), n in by_code.most_common():
|
||||
lines.append(f"| {sev} | {code} | {n} |")
|
||||
|
||||
p0 = [r for r in rows if r["severity"] == "P0"]
|
||||
if p0:
|
||||
lines += ["", "## P0 blockers (top 15)", "",
|
||||
"| Entry | Type | Code | Message |", "|---|---|---|---|"]
|
||||
for r in p0[:15]:
|
||||
msg = r["message"].replace("|", "\\|")
|
||||
lines.append(f"| {r['entry_id']} | {r['node_type']} | {r['code']} | {msg} |")
|
||||
|
||||
lines += ["", "## Next step",
|
||||
("Triage P1 in `defect_log.csv`; client reviews the clean entries against this report."
|
||||
if gate == "PASS" else
|
||||
"Assign and fix every P0, re-run the validator, and only then open client review."),
|
||||
""]
|
||||
(outdir / "report.md").write_text("\n".join(lines), encoding="utf-8")
|
||||
return gate, counts
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description="5-layer offline JSON-LD schema validator.")
|
||||
ap.add_argument("dataset", nargs="?", help="xlsx/csv/jsonl/json file or a directory")
|
||||
ap.add_argument("--url-list", help="canonical URL inventory (xlsx/csv/txt) → enables Layer 0")
|
||||
ap.add_argument("--out", default="schema_qa_out", help="output directory")
|
||||
ap.add_argument("--strict", action="store_true",
|
||||
help="unexpected props on known types → P1; unknown types → P1")
|
||||
ap.add_argument("--no-recommended", action="store_true",
|
||||
help="drop L3 recommended (P2) findings — highest-signal gate")
|
||||
ap.add_argument("--live", nargs="+", metavar="URL",
|
||||
help="Mode B: validate live URLs (extract embedded JSON-LD)")
|
||||
ap.add_argument("--rules", default=str(RULES_DEFAULT), help="path to schema_rules.json")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if not args.dataset and not args.live:
|
||||
ap.error("provide a DATASET path or --live URL ...")
|
||||
|
||||
rules = json.loads(Path(args.rules).read_text(encoding="utf-8"))
|
||||
|
||||
try:
|
||||
entries = load_entries(args.dataset, args.live)
|
||||
except (ValueError, FileNotFoundError) as exc:
|
||||
print(f"ERROR loading input: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
inventory = load_url_inventory(args.url_list) if args.url_list else None
|
||||
|
||||
defects, valid_entries, nodes = run(entries, rules, inventory,
|
||||
args.strict, args.no_recommended)
|
||||
meta = {"entries": len(entries), "valid_entries": valid_entries, "nodes": nodes,
|
||||
"mode": "B-live" if args.live else "A-dataset", "strict": args.strict,
|
||||
"coverage": inventory is not None}
|
||||
gate, counts = write_outputs(defects, args.out, meta)
|
||||
|
||||
print(f"[{gate}] entries={len(entries)} nodes={nodes} "
|
||||
f"P0={counts['P0']} P1={counts['P1']} P2={counts['P2']} → {args.out}/")
|
||||
# Exit 1 when the gate fails so CI and `&&` chains stop on P0.
|
||||
return 0 if gate == "PASS" else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,57 @@
|
||||
# 구조화 데이터 QA 리포트 — {{프로젝트명}}
|
||||
|
||||
> 클라이언트 검토용. **원본 JSON이 아니라 결함 리포트를 검토합니다.**
|
||||
> 이 리포트에 오른 엔트리는 모두 기계 검증(Layer 0–4)을 통과한 **P0 0건** 상태입니다.
|
||||
|
||||
| 항목 | 값 |
|
||||
|---|---|
|
||||
| 데이터셋 | `{{dataset_파일명}}` |
|
||||
| 검증 일시 | {{YYYY-MM-DD HH:MM}} |
|
||||
| 검증 모드 | A — Dataset QA (배포 전) / B — Live audit (배포 후) |
|
||||
| 엔트리 수 | {{entries}} (파싱 성공 {{valid_entries}}, 노드 {{nodes}}) |
|
||||
| **게이트** | **{{PASS / FAIL}}** (PASS = P0 0건) |
|
||||
| 결함 | P0 {{n}} · P1 {{n}} · P2 {{n}} |
|
||||
| Audit ID | SCHEMA-{{YYYYMMDD}}-{{NNN}} |
|
||||
|
||||
## 1. 한눈에 보기
|
||||
|
||||
- ✅ **검토 가능 엔트리**: P0 0건을 통과한 {{n}}개 — 아래 판단 항목만 확인해 주세요.
|
||||
- ⛔ **보류 엔트리**(있다면): P0 {{n}}건으로 검토 대상에서 제외. 수정 후 재검증합니다.
|
||||
- 이번 검토에서 **사람의 판단이 필요한 것**은 기계가 잡지 못하는 두 가지뿐입니다:
|
||||
1. 페이지에 맞는 스키마 **타입**이 선택되었는가
|
||||
2. 표시되는 **문구(설명·이름)**가 사실과 정확히 일치하는가
|
||||
|
||||
## 2. 결함 요약 (코드별)
|
||||
|
||||
| 심각도 | 코드 | 건수 | 의미 |
|
||||
|---|---|---|---|
|
||||
| P0 | {{CODE}} | {{n}} | {{한 줄 설명}} |
|
||||
| P1 | {{CODE}} | {{n}} | {{한 줄 설명}} |
|
||||
| P2 | {{CODE}} | {{n}} | {{한 줄 설명}} |
|
||||
|
||||
> 코드 정의: `references/defect-taxonomy.md`. 전체 목록: 첨부 `defect_log.csv`.
|
||||
|
||||
## 3. P0 블로커 (있을 경우 — 검토 전 수정 필수)
|
||||
|
||||
| 엔트리 | 타입 | 코드 | 내용 | 담당 | 상태 |
|
||||
|---|---|---|---|---|---|
|
||||
| {{entry_id}} | {{type}} | {{CODE}} | {{message}} | {{owner}} | open |
|
||||
|
||||
## 4. 클라이언트 확인 요청 (판단 항목)
|
||||
|
||||
기계가 통과시킨 엔트리 중, 사람의 확인이 필요한 항목입니다.
|
||||
|
||||
| # | URL / 페이지 | 확인 요청 | 비고 |
|
||||
|---|---|---|---|
|
||||
| 1 | {{url}} | 이 페이지에 `{{@type}}` 타입이 맞습니까? | |
|
||||
| 2 | {{url}} | 설명/이름 문구가 정확합니까? | |
|
||||
|
||||
## 5. 다음 단계
|
||||
|
||||
- **PASS인 경우**: 위 4번 판단 항목 확정 → 배포 단계(G4 안정화)로 이동, 샘플을 Google
|
||||
Rich Results Test로 최종 확인.
|
||||
- **FAIL인 경우**: P0 담당 배정 → 수정 → 재검증(`validate_schema.py`) → 본 리포트 갱신.
|
||||
- P1 처리 방침(수정/수용)은 `decision-log.md`에 기록합니다.
|
||||
|
||||
---
|
||||
*생성: 16-seo-schema-validator · 첨부: `report.md`, `defect_log.csv`, `results.json`*
|
||||
@@ -0,0 +1,39 @@
|
||||
# P1 Decision Log — {{프로젝트명}}
|
||||
|
||||
P0 is non-negotiable: every P0 is fixed before launch (the gate enforces it). **P1 is
|
||||
where judgement lives** — some P1s get fixed, some get consciously accepted. This log
|
||||
records *which, by whom, and why*, so an accepted P1 is a decision on the record, not a
|
||||
silently ignored defect. It is the G3 (테스트) deliverable.
|
||||
|
||||
## How to use
|
||||
|
||||
1. Open `defect_log.csv`, filter to `severity = P1`.
|
||||
2. For each P1 (or each group of identical P1s), add a row below.
|
||||
3. Decision is one of: **Fix** (will correct before launch) / **Accept** (ship as-is,
|
||||
with rationale) / **Defer** (post-launch backlog).
|
||||
4. An `Accept`/`Defer` needs a named approver. `Fix` needs an owner + target date.
|
||||
5. Re-run the validator after the fixes; confirm the fixed P1s are gone.
|
||||
|
||||
## Log
|
||||
|
||||
| # | Code | Entry / scope | Summary | Decision | Owner / Approver | Target / Date | Rationale |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 1 | {{CODE}} | {{entry_id or (dataset)}} | {{one line}} | Fix / Accept / Defer | {{name}} | {{YYYY-MM-DD}} | {{why}} |
|
||||
| 2 | | | | | | | |
|
||||
| 3 | | | | | | | |
|
||||
|
||||
## Standing decisions (apply to all entries unless overridden)
|
||||
|
||||
Record cross-cutting calls once here instead of per row — e.g. "MISSING_RECOMMENDED for
|
||||
`starRating` is accepted group-wide: not contractually rated." Reduces log noise.
|
||||
|
||||
| Code | Standing decision | Approver | Date |
|
||||
|---|---|---|---|
|
||||
| {{CODE}} | Accept group-wide — {{reason}} | {{name}} | {{YYYY-MM-DD}} |
|
||||
|
||||
## Sign-off
|
||||
|
||||
| Stage gate | Condition | Confirmed by | Date |
|
||||
|---|---|---|---|
|
||||
| G3 테스트 | All P1 triaged (Fix/Accept/Defer), decisions logged above | {{name}} | {{date}} |
|
||||
| G4 안정화 | P0 = 0, all "Fix" P1 closed, online validator green on sample | {{name}} | {{date}} |
|
||||
126
custom-skills/17-seo-schema-generator/SKILL.md
Normal file
126
custom-skills/17-seo-schema-generator/SKILL.md
Normal file
@@ -0,0 +1,126 @@
|
||||
---
|
||||
name: 17-seo-schema-generator
|
||||
description: |
|
||||
Generates validation-ready JSON-LD structured data for a site, covering BOTH
|
||||
scenarios: (1) from an existing website — extract facts from live pages; and
|
||||
(2) from collected sources for a not-yet-published site — reconcile conflicting
|
||||
facts into a provenance-tracked claims register. Both modes emit the same claims
|
||||
register, build pruned drafts from type templates (no placeholders shipped), and
|
||||
hand off to 16-seo-schema-validator (generate -> validate, gate = zero P0).
|
||||
Triggers: generate schema, create JSON-LD, schema markup, structured data generator,
|
||||
source-to-schema, pre-launch schema, claims register, 스키마 생성, 스키마 저작,
|
||||
구조화 데이터 생성, 미발행 사이트 스키마, 기존 사이트 스키마 추출.
|
||||
version: "2.0"
|
||||
author: OurDigital / D.intelligence
|
||||
environment: Code
|
||||
---
|
||||
|
||||
# SEO Schema Generator (17)
|
||||
|
||||
Author JSON-LD for a site — whether the pages already exist or the site is not yet
|
||||
published. Both cases are error-prone for the same reason: facts must be turned into
|
||||
schema without leaking conflicts, gaps, or placeholders. This skill makes that reliable
|
||||
by routing **both scenarios through one pivot — a claims register** — then generating
|
||||
pruned drafts that hand off cleanly to the `16-seo-schema-validator` gate.
|
||||
|
||||
**Generate (17) → Validate (16).** This skill produces drafts; 16 is the QA gate.
|
||||
|
||||
## Two modes, one pipeline
|
||||
|
||||
The only thing that differs between the scenarios is **where facts come from**.
|
||||
Everything after the claims register is identical.
|
||||
|
||||
| | **Mode 1 — from an existing site** | **Mode 2 — from collected sources** |
|
||||
|---|---|---|
|
||||
| When | Site has pages but lacks (or needs better) schema | Site not published yet (no DOM) |
|
||||
| Source of truth | the live pages | scattered, conflicting sources (DART, Wikidata, brochures) |
|
||||
| Seed the register with | `scripts/extract_site_claims.py` | manual research → `templates/claims-register.csv` |
|
||||
| Hard part | extraction & mapping | authority hierarchy + entity reconciliation |
|
||||
| Conflicts | rare (one source) | frequent → resolve before shipping |
|
||||
|
||||
```
|
||||
Mode 1 (extract_site_claims.py)─┐
|
||||
├─▶ claims_register.csv ─▶ build_schema_drafts.py ─▶ drafts/*.jsonld
|
||||
Mode 2 (research + register)────┘ (the pivot) └─▶ schema_drafts_dataset.csv
|
||||
│
|
||||
16-seo-schema-validator ▼ (gate: zero P0)
|
||||
```
|
||||
|
||||
## The claims register — the core idea
|
||||
|
||||
A **claims register** is a provenance-tracked, conflict-resolved fact table. Columns:
|
||||
`entity_id, entity_type, property, value, lang, url, source_ids, authority, confidence,
|
||||
conflict, status, note`. Dotted `property` paths nest (`address.streetAddress`);
|
||||
pipe-separate array values (`a|b|c`).
|
||||
|
||||
**Only `CONFIRMED`, non-conflicting claims become schema.** Everything else (PENDING,
|
||||
CONFLICT, REJECTED, EMPTY) is excluded and reported — never shipped. An unfilled
|
||||
template slot is **deleted**, never emitted as `{{…}}` or `TODO` (placeholder leakage
|
||||
is the #1 pre-launch P0).
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
# Try the bundled sample first (Mode 2)
|
||||
python scripts/make_sample.py
|
||||
python scripts/build_schema_drafts.py fixtures/sample_claims.csv --out drafts_out
|
||||
|
||||
# MODE 1 — existing site → register (URLs, or local .html / a directory offline)
|
||||
python scripts/extract_site_claims.py https://example.com/ https://example.com/about \
|
||||
--out site_claims
|
||||
# review site_claims/claims_register.csv (confirm PENDING rows), then build:
|
||||
python scripts/build_schema_drafts.py site_claims/claims_register.csv --out drafts_out
|
||||
|
||||
# MODE 2 — collected sources → register (fill templates/claims-register.csv by hand)
|
||||
python scripts/build_schema_drafts.py path/to/claims_register.csv --out drafts_out
|
||||
|
||||
# HAND OFF TO THE GATE (must reach zero P0)
|
||||
python ../16-seo-schema-validator/scripts/validate_schema.py \
|
||||
drafts_out/schema_drafts_dataset.csv --out qa_out
|
||||
```
|
||||
|
||||
## Outputs
|
||||
|
||||
- `drafts/*.jsonld` — one pruned draft per entity (× language).
|
||||
- `schema_drafts_dataset.csv` — directly consumable by `16-seo-schema-validator`.
|
||||
- `build_report.md` — entities built + **excluded claims** (PENDING / CONFLICT / EMPTY) with reasons.
|
||||
- (Mode 1 also) `claims_register.csv` + `extraction_report.md`.
|
||||
|
||||
## Stage gates (설계→개발→테스트→안정화→런칭 후)
|
||||
|
||||
- **G1 설계** — Lock the entity→type map (`references/entity-and-type-map.md`). Mode 2: source
|
||||
register complete (≥2 sources/entity). *DoD:* every entity has an assigned type + required list.
|
||||
- **G2 개발** — Seed the register (Mode 1 extract / Mode 2 research), reconcile to `CONFIRMED`,
|
||||
conflicts = 0, run the builder → drafts have **zero placeholders**.
|
||||
- **G3 테스트** — Validate (16): **zero P0**; triage P1; fact-accuracy sign-off via `templates/review-guide.md` (report-based, not raw JSON).
|
||||
- **G4 안정화** — Google Rich Results Test green on a sample; re-run shows no regression.
|
||||
- **G5 런칭 후** — live schema == drafts; GSC "Rich results" no new errors.
|
||||
|
||||
## References & templates
|
||||
|
||||
- Mode 1 SOP: `references/site-extraction-methodology.md`.
|
||||
- Mode 2 SOP (9 steps): `references/source-to-schema-methodology.md`.
|
||||
- Source authority ranking: `references/source-authority-hierarchy.md`.
|
||||
- Entity→type scoping: `references/entity-and-type-map.md`.
|
||||
- Registers + review guide: `templates/claims-register.csv`, `templates/source-register.csv`, `templates/review-guide.md`.
|
||||
|
||||
## Templates included
|
||||
|
||||
`scripts/type_templates.json` covers Organization, WebSite, Hotel, Person, JobPosting,
|
||||
VideoObject, FAQPage. Required props are aligned with the validator's rule set, so a
|
||||
fully confirmed entity passes the gate. **Add a type = add a template block (edit JSON only).**
|
||||
|
||||
## Limits & honesty
|
||||
|
||||
- Quality of drafts == quality of the register. Garbage-in still produces gaps — but
|
||||
reported, never as placeholders.
|
||||
- Mode 1 inference (title/OpenGraph) is seeded as `PENDING` and will NOT ship until a
|
||||
human confirms it; existing JSON-LD is seeded `CONFIRMED`. If a site already has good
|
||||
JSON-LD, prefer auditing it directly with `16` Mode B.
|
||||
- Authoritative rich-result eligibility still needs Google's online test on a sample at G4.
|
||||
|
||||
## Integration
|
||||
|
||||
- **→ 16-seo-schema-validator**: the dataset CSV is the handoff; the gate is `zero P0`.
|
||||
- **→ seo-comprehensive-audit**: post-launch (G5) uses the validator's Mode B as audit stage 4.
|
||||
- This skill is a one-time-per-site authoring workflow, **not** an audit-pipeline stage.
|
||||
@@ -1,156 +0,0 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## Overview
|
||||
|
||||
Schema markup generator: create JSON-LD structured data from templates for various content types.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
pip install -r scripts/requirements.txt
|
||||
|
||||
# Generate Organization schema
|
||||
python scripts/schema_generator.py --type organization --url https://example.com
|
||||
|
||||
# Generate from template
|
||||
python scripts/schema_generator.py --template templates/article.json --data article_data.json
|
||||
```
|
||||
|
||||
## Scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `schema_generator.py` | Generate schema markup |
|
||||
| `base_client.py` | Shared utilities |
|
||||
|
||||
## Supported Schema Types
|
||||
|
||||
| Type | Template | Use Case |
|
||||
|------|----------|----------|
|
||||
| Organization | `organization.json` | Company/brand info |
|
||||
| LocalBusiness | `local_business.json` | Physical locations |
|
||||
| Article | `article.json` | Blog posts, news |
|
||||
| Product | `product.json` | E-commerce items |
|
||||
| FAQPage | `faq.json` | FAQ sections |
|
||||
| BreadcrumbList | `breadcrumb.json` | Navigation path |
|
||||
| WebSite | `website.json` | Site-level info |
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Organization
|
||||
```bash
|
||||
python scripts/schema_generator.py --type organization \
|
||||
--name "Company Name" \
|
||||
--url "https://example.com" \
|
||||
--logo "https://example.com/logo.png"
|
||||
```
|
||||
|
||||
### LocalBusiness
|
||||
```bash
|
||||
python scripts/schema_generator.py --type localbusiness \
|
||||
--name "Restaurant Name" \
|
||||
--address "123 Main St, City, State 12345" \
|
||||
--phone "+1-555-123-4567" \
|
||||
--hours "Mo-Fr 09:00-17:00"
|
||||
```
|
||||
|
||||
### Article
|
||||
```bash
|
||||
python scripts/schema_generator.py --type article \
|
||||
--headline "Article Title" \
|
||||
--author "Author Name" \
|
||||
--published "2024-01-15" \
|
||||
--image "https://example.com/image.jpg"
|
||||
```
|
||||
|
||||
### FAQPage
|
||||
```bash
|
||||
python scripts/schema_generator.py --type faq \
|
||||
--questions questions.json
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Generated JSON-LD ready for insertion:
|
||||
|
||||
```html
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
"name": "Company Name",
|
||||
"url": "https://example.com",
|
||||
"logo": "https://example.com/logo.png"
|
||||
}
|
||||
</script>
|
||||
```
|
||||
|
||||
## Template Customization
|
||||
|
||||
Templates in `templates/` can be modified. Required fields are marked:
|
||||
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Article",
|
||||
"headline": "{{REQUIRED}}",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "{{REQUIRED}}"
|
||||
},
|
||||
"datePublished": "{{REQUIRED}}",
|
||||
"image": "{{RECOMMENDED}}"
|
||||
}
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
Generated schemas are validated before output:
|
||||
- Syntax correctness
|
||||
- Required properties present
|
||||
- Schema.org vocabulary compliance
|
||||
|
||||
Use skill 13 (schema-validator) for additional validation.
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
jsonschema>=4.21.0
|
||||
requests>=2.31.0
|
||||
python-dotenv>=1.0.0
|
||||
```
|
||||
|
||||
## Notion Output (Required)
|
||||
|
||||
**IMPORTANT**: All audit reports MUST be saved to the OurDigital SEO Audit Log database.
|
||||
|
||||
### Database Configuration
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Database ID | `2c8581e5-8a1e-8035-880b-e38cefc2f3ef` |
|
||||
| URL | https://www.notion.so/dintelligence/2c8581e58a1e8035880be38cefc2f3ef |
|
||||
|
||||
### Required Properties
|
||||
|
||||
| Property | Type | Description |
|
||||
|----------|------|-------------|
|
||||
| Issue | Title | Report title (Korean + date) |
|
||||
| Site | URL | Audited website URL |
|
||||
| Category | Select | Technical SEO, On-page SEO, Performance, Schema/Structured Data, Sitemap, Robots.txt, Content, Local SEO |
|
||||
| Priority | Select | Critical, High, Medium, Low |
|
||||
| Found Date | Date | Audit date (YYYY-MM-DD) |
|
||||
| Audit ID | Rich Text | Format: [TYPE]-YYYYMMDD-NNN |
|
||||
|
||||
### Language Guidelines
|
||||
|
||||
- Report content in Korean (한국어)
|
||||
- Keep technical English terms as-is (e.g., SEO Audit, Core Web Vitals, Schema Markup)
|
||||
- URLs and code remain unchanged
|
||||
|
||||
### Example MCP Call
|
||||
|
||||
```bash
|
||||
mcp-cli call notion/API-post-page '{"parent": {"database_id": "2c8581e5-8a1e-8035-880b-e38cefc2f3ef"}, "properties": {...}}'
|
||||
```
|
||||
|
||||
@@ -1,207 +0,0 @@
|
||||
"""
|
||||
Base Client - Shared async client utilities
|
||||
===========================================
|
||||
Purpose: Rate-limited async operations for API clients
|
||||
Python: 3.10+
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from asyncio import Semaphore
|
||||
from datetime import datetime
|
||||
from typing import Any, Callable, TypeVar
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from tenacity import (
|
||||
retry,
|
||||
stop_after_attempt,
|
||||
wait_exponential,
|
||||
retry_if_exception_type,
|
||||
)
|
||||
|
||||
# Load environment variables
|
||||
load_dotenv()
|
||||
|
||||
# Logging setup
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
|
||||
T = TypeVar("T")
|
||||
|
||||
|
||||
class RateLimiter:
|
||||
"""Rate limiter using token bucket algorithm."""
|
||||
|
||||
def __init__(self, rate: float, per: float = 1.0):
|
||||
"""
|
||||
Initialize rate limiter.
|
||||
|
||||
Args:
|
||||
rate: Number of requests allowed
|
||||
per: Time period in seconds (default: 1 second)
|
||||
"""
|
||||
self.rate = rate
|
||||
self.per = per
|
||||
self.tokens = rate
|
||||
self.last_update = datetime.now()
|
||||
self._lock = asyncio.Lock()
|
||||
|
||||
async def acquire(self) -> None:
|
||||
"""Acquire a token, waiting if necessary."""
|
||||
async with self._lock:
|
||||
now = datetime.now()
|
||||
elapsed = (now - self.last_update).total_seconds()
|
||||
self.tokens = min(self.rate, self.tokens + elapsed * (self.rate / self.per))
|
||||
self.last_update = now
|
||||
|
||||
if self.tokens < 1:
|
||||
wait_time = (1 - self.tokens) * (self.per / self.rate)
|
||||
await asyncio.sleep(wait_time)
|
||||
self.tokens = 0
|
||||
else:
|
||||
self.tokens -= 1
|
||||
|
||||
|
||||
class BaseAsyncClient:
|
||||
"""Base class for async API clients with rate limiting."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
max_concurrent: int = 5,
|
||||
requests_per_second: float = 3.0,
|
||||
logger: logging.Logger | None = None,
|
||||
):
|
||||
"""
|
||||
Initialize base client.
|
||||
|
||||
Args:
|
||||
max_concurrent: Maximum concurrent requests
|
||||
requests_per_second: Rate limit
|
||||
logger: Logger instance
|
||||
"""
|
||||
self.semaphore = Semaphore(max_concurrent)
|
||||
self.rate_limiter = RateLimiter(requests_per_second)
|
||||
self.logger = logger or logging.getLogger(self.__class__.__name__)
|
||||
self.stats = {
|
||||
"requests": 0,
|
||||
"success": 0,
|
||||
"errors": 0,
|
||||
"retries": 0,
|
||||
}
|
||||
|
||||
@retry(
|
||||
stop=stop_after_attempt(3),
|
||||
wait=wait_exponential(multiplier=1, min=2, max=10),
|
||||
retry=retry_if_exception_type(Exception),
|
||||
)
|
||||
async def _rate_limited_request(
|
||||
self,
|
||||
coro: Callable[[], Any],
|
||||
) -> Any:
|
||||
"""Execute a request with rate limiting and retry."""
|
||||
async with self.semaphore:
|
||||
await self.rate_limiter.acquire()
|
||||
self.stats["requests"] += 1
|
||||
try:
|
||||
result = await coro()
|
||||
self.stats["success"] += 1
|
||||
return result
|
||||
except Exception as e:
|
||||
self.stats["errors"] += 1
|
||||
self.logger.error(f"Request failed: {e}")
|
||||
raise
|
||||
|
||||
async def batch_requests(
|
||||
self,
|
||||
requests: list[Callable[[], Any]],
|
||||
desc: str = "Processing",
|
||||
) -> list[Any]:
|
||||
"""Execute multiple requests concurrently."""
|
||||
try:
|
||||
from tqdm.asyncio import tqdm
|
||||
has_tqdm = True
|
||||
except ImportError:
|
||||
has_tqdm = False
|
||||
|
||||
async def execute(req: Callable) -> Any:
|
||||
try:
|
||||
return await self._rate_limited_request(req)
|
||||
except Exception as e:
|
||||
return {"error": str(e)}
|
||||
|
||||
tasks = [execute(req) for req in requests]
|
||||
|
||||
if has_tqdm:
|
||||
results = []
|
||||
for coro in tqdm.as_completed(tasks, total=len(tasks), desc=desc):
|
||||
result = await coro
|
||||
results.append(result)
|
||||
return results
|
||||
else:
|
||||
return await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
def print_stats(self) -> None:
|
||||
"""Print request statistics."""
|
||||
self.logger.info("=" * 40)
|
||||
self.logger.info("Request Statistics:")
|
||||
self.logger.info(f" Total Requests: {self.stats['requests']}")
|
||||
self.logger.info(f" Successful: {self.stats['success']}")
|
||||
self.logger.info(f" Errors: {self.stats['errors']}")
|
||||
self.logger.info("=" * 40)
|
||||
|
||||
|
||||
class ConfigManager:
|
||||
"""Manage API configuration and credentials."""
|
||||
|
||||
def __init__(self):
|
||||
load_dotenv()
|
||||
|
||||
@property
|
||||
def google_credentials_path(self) -> str | None:
|
||||
"""Get Google service account credentials path."""
|
||||
# Prefer SEO-specific credentials, fallback to general credentials
|
||||
seo_creds = os.path.expanduser("~/.credential/ourdigital-seo-agent.json")
|
||||
if os.path.exists(seo_creds):
|
||||
return seo_creds
|
||||
return os.getenv("GOOGLE_APPLICATION_CREDENTIALS")
|
||||
|
||||
@property
|
||||
def pagespeed_api_key(self) -> str | None:
|
||||
"""Get PageSpeed Insights API key."""
|
||||
return os.getenv("PAGESPEED_API_KEY")
|
||||
|
||||
@property
|
||||
def custom_search_api_key(self) -> str | None:
|
||||
"""Get Custom Search API key."""
|
||||
return os.getenv("CUSTOM_SEARCH_API_KEY")
|
||||
|
||||
@property
|
||||
def custom_search_engine_id(self) -> str | None:
|
||||
"""Get Custom Search Engine ID."""
|
||||
return os.getenv("CUSTOM_SEARCH_ENGINE_ID")
|
||||
|
||||
@property
|
||||
def notion_token(self) -> str | None:
|
||||
"""Get Notion API token."""
|
||||
return os.getenv("NOTION_TOKEN") or os.getenv("NOTION_API_KEY")
|
||||
|
||||
def validate_google_credentials(self) -> bool:
|
||||
"""Validate Google credentials are configured."""
|
||||
creds_path = self.google_credentials_path
|
||||
if not creds_path:
|
||||
return False
|
||||
return os.path.exists(creds_path)
|
||||
|
||||
def get_required(self, key: str) -> str:
|
||||
"""Get required environment variable or raise error."""
|
||||
value = os.getenv(key)
|
||||
if not value:
|
||||
raise ValueError(f"Missing required environment variable: {key}")
|
||||
return value
|
||||
|
||||
|
||||
# Singleton config instance
|
||||
config = ConfigManager()
|
||||
@@ -1,6 +0,0 @@
|
||||
# 14-seo-schema-generator dependencies
|
||||
jsonschema>=4.21.0
|
||||
requests>=2.31.0
|
||||
python-dotenv>=1.0.0
|
||||
rich>=13.7.0
|
||||
typer>=0.9.0
|
||||
@@ -1,490 +0,0 @@
|
||||
"""
|
||||
Schema Generator - Generate JSON-LD structured data markup
|
||||
==========================================================
|
||||
Purpose: Generate schema.org structured data in JSON-LD format
|
||||
Python: 3.10+
|
||||
Usage:
|
||||
python schema_generator.py --type organization --name "Company Name" --url "https://example.com"
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Template directory relative to this script
|
||||
TEMPLATE_DIR = Path(__file__).parent.parent / "templates" / "schema_templates"
|
||||
|
||||
|
||||
class SchemaGenerator:
|
||||
"""Generate JSON-LD schema markup from templates."""
|
||||
|
||||
SCHEMA_TYPES = {
|
||||
"organization": "organization.json",
|
||||
"local_business": "local_business.json",
|
||||
"product": "product.json",
|
||||
"article": "article.json",
|
||||
"faq": "faq.json",
|
||||
"breadcrumb": "breadcrumb.json",
|
||||
"website": "website.json",
|
||||
}
|
||||
|
||||
# Business type mappings for LocalBusiness
|
||||
BUSINESS_TYPES = {
|
||||
"restaurant": "Restaurant",
|
||||
"cafe": "CafeOrCoffeeShop",
|
||||
"bar": "BarOrPub",
|
||||
"hotel": "Hotel",
|
||||
"store": "Store",
|
||||
"medical": "MedicalBusiness",
|
||||
"dental": "Dentist",
|
||||
"legal": "LegalService",
|
||||
"real_estate": "RealEstateAgent",
|
||||
"auto": "AutoRepair",
|
||||
"beauty": "BeautySalon",
|
||||
"gym": "HealthClub",
|
||||
"spa": "DaySpa",
|
||||
}
|
||||
|
||||
# Article type mappings
|
||||
ARTICLE_TYPES = {
|
||||
"article": "Article",
|
||||
"blog": "BlogPosting",
|
||||
"news": "NewsArticle",
|
||||
"tech": "TechArticle",
|
||||
"scholarly": "ScholarlyArticle",
|
||||
}
|
||||
|
||||
def __init__(self, template_dir: Path = TEMPLATE_DIR):
|
||||
self.template_dir = template_dir
|
||||
|
||||
def load_template(self, schema_type: str) -> dict:
|
||||
"""Load a schema template file."""
|
||||
if schema_type not in self.SCHEMA_TYPES:
|
||||
raise ValueError(f"Unknown schema type: {schema_type}. "
|
||||
f"Available: {list(self.SCHEMA_TYPES.keys())}")
|
||||
|
||||
template_file = self.template_dir / self.SCHEMA_TYPES[schema_type]
|
||||
if not template_file.exists():
|
||||
raise FileNotFoundError(f"Template not found: {template_file}")
|
||||
|
||||
with open(template_file, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
def fill_template(self, template: dict, data: dict[str, Any]) -> dict:
|
||||
"""Fill template placeholders with actual data."""
|
||||
template_str = json.dumps(template, ensure_ascii=False)
|
||||
|
||||
# Replace placeholders {{key}} with values
|
||||
for key, value in data.items():
|
||||
placeholder = f"{{{{{key}}}}}"
|
||||
if value is not None:
|
||||
template_str = template_str.replace(placeholder, str(value))
|
||||
|
||||
# Remove unfilled placeholders and their parent objects if empty
|
||||
result = json.loads(template_str)
|
||||
return self._clean_empty_values(result)
|
||||
|
||||
def _clean_empty_values(self, obj: Any) -> Any:
|
||||
"""Remove empty values and unfilled placeholders."""
|
||||
if isinstance(obj, dict):
|
||||
cleaned = {}
|
||||
for key, value in obj.items():
|
||||
cleaned_value = self._clean_empty_values(value)
|
||||
# Skip if value is empty, None, or unfilled placeholder
|
||||
if cleaned_value is None:
|
||||
continue
|
||||
if isinstance(cleaned_value, str) and cleaned_value.startswith("{{"):
|
||||
continue
|
||||
if isinstance(cleaned_value, (list, dict)) and not cleaned_value:
|
||||
continue
|
||||
cleaned[key] = cleaned_value
|
||||
return cleaned if cleaned else None
|
||||
elif isinstance(obj, list):
|
||||
cleaned = []
|
||||
for item in obj:
|
||||
cleaned_item = self._clean_empty_values(item)
|
||||
if cleaned_item is not None:
|
||||
if isinstance(cleaned_item, str) and cleaned_item.startswith("{{"):
|
||||
continue
|
||||
cleaned.append(cleaned_item)
|
||||
return cleaned if cleaned else None
|
||||
elif isinstance(obj, str):
|
||||
if obj.startswith("{{") and obj.endswith("}}"):
|
||||
return None
|
||||
return obj
|
||||
return obj
|
||||
|
||||
def generate_organization(
|
||||
self,
|
||||
name: str,
|
||||
url: str,
|
||||
logo_url: str | None = None,
|
||||
description: str | None = None,
|
||||
founding_date: str | None = None,
|
||||
phone: str | None = None,
|
||||
address: dict | None = None,
|
||||
social_links: list[str] | None = None,
|
||||
) -> dict:
|
||||
"""Generate Organization schema."""
|
||||
template = self.load_template("organization")
|
||||
|
||||
data = {
|
||||
"name": name,
|
||||
"url": url,
|
||||
"logo_url": logo_url,
|
||||
"description": description,
|
||||
"founding_date": founding_date,
|
||||
"phone": phone,
|
||||
}
|
||||
|
||||
if address:
|
||||
data.update({
|
||||
"street_address": address.get("street"),
|
||||
"city": address.get("city"),
|
||||
"region": address.get("region"),
|
||||
"postal_code": address.get("postal_code"),
|
||||
"country": address.get("country", "KR"),
|
||||
})
|
||||
|
||||
if social_links:
|
||||
# Handle social links specially
|
||||
pass
|
||||
|
||||
return self.fill_template(template, data)
|
||||
|
||||
def generate_local_business(
|
||||
self,
|
||||
name: str,
|
||||
business_type: str,
|
||||
address: dict,
|
||||
phone: str | None = None,
|
||||
url: str | None = None,
|
||||
description: str | None = None,
|
||||
hours: dict | None = None,
|
||||
geo: dict | None = None,
|
||||
price_range: str | None = None,
|
||||
rating: float | None = None,
|
||||
review_count: int | None = None,
|
||||
) -> dict:
|
||||
"""Generate LocalBusiness schema."""
|
||||
template = self.load_template("local_business")
|
||||
|
||||
schema_business_type = self.BUSINESS_TYPES.get(
|
||||
business_type.lower(), "LocalBusiness"
|
||||
)
|
||||
|
||||
data = {
|
||||
"business_type": schema_business_type,
|
||||
"name": name,
|
||||
"url": url,
|
||||
"description": description,
|
||||
"phone": phone,
|
||||
"price_range": price_range,
|
||||
"street_address": address.get("street"),
|
||||
"city": address.get("city"),
|
||||
"region": address.get("region"),
|
||||
"postal_code": address.get("postal_code"),
|
||||
"country": address.get("country", "KR"),
|
||||
}
|
||||
|
||||
if geo:
|
||||
data["latitude"] = geo.get("lat")
|
||||
data["longitude"] = geo.get("lng")
|
||||
|
||||
if hours:
|
||||
data.update({
|
||||
"weekday_opens": hours.get("weekday_opens", "09:00"),
|
||||
"weekday_closes": hours.get("weekday_closes", "18:00"),
|
||||
"weekend_opens": hours.get("weekend_opens"),
|
||||
"weekend_closes": hours.get("weekend_closes"),
|
||||
})
|
||||
|
||||
if rating is not None:
|
||||
data["rating"] = str(rating)
|
||||
data["review_count"] = str(review_count or 0)
|
||||
|
||||
return self.fill_template(template, data)
|
||||
|
||||
def generate_product(
|
||||
self,
|
||||
name: str,
|
||||
description: str,
|
||||
price: float,
|
||||
currency: str = "KRW",
|
||||
brand: str | None = None,
|
||||
sku: str | None = None,
|
||||
images: list[str] | None = None,
|
||||
availability: str = "InStock",
|
||||
condition: str = "NewCondition",
|
||||
rating: float | None = None,
|
||||
review_count: int | None = None,
|
||||
url: str | None = None,
|
||||
seller: str | None = None,
|
||||
) -> dict:
|
||||
"""Generate Product schema."""
|
||||
template = self.load_template("product")
|
||||
|
||||
data = {
|
||||
"name": name,
|
||||
"description": description,
|
||||
"price": str(int(price)),
|
||||
"currency": currency,
|
||||
"brand_name": brand,
|
||||
"sku": sku,
|
||||
"product_url": url,
|
||||
"availability": availability,
|
||||
"condition": condition,
|
||||
"seller_name": seller,
|
||||
}
|
||||
|
||||
if images:
|
||||
for i, img in enumerate(images[:3], 1):
|
||||
data[f"image_url_{i}"] = img
|
||||
|
||||
if rating is not None:
|
||||
data["rating"] = str(rating)
|
||||
data["review_count"] = str(review_count or 0)
|
||||
|
||||
return self.fill_template(template, data)
|
||||
|
||||
def generate_article(
|
||||
self,
|
||||
headline: str,
|
||||
description: str,
|
||||
author_name: str,
|
||||
date_published: str,
|
||||
publisher_name: str,
|
||||
article_type: str = "article",
|
||||
date_modified: str | None = None,
|
||||
images: list[str] | None = None,
|
||||
page_url: str | None = None,
|
||||
publisher_logo: str | None = None,
|
||||
author_url: str | None = None,
|
||||
section: str | None = None,
|
||||
word_count: int | None = None,
|
||||
keywords: str | None = None,
|
||||
) -> dict:
|
||||
"""Generate Article schema."""
|
||||
template = self.load_template("article")
|
||||
|
||||
schema_article_type = self.ARTICLE_TYPES.get(
|
||||
article_type.lower(), "Article"
|
||||
)
|
||||
|
||||
data = {
|
||||
"article_type": schema_article_type,
|
||||
"headline": headline,
|
||||
"description": description,
|
||||
"author_name": author_name,
|
||||
"author_url": author_url,
|
||||
"date_published": date_published,
|
||||
"date_modified": date_modified or date_published,
|
||||
"publisher_name": publisher_name,
|
||||
"publisher_logo_url": publisher_logo,
|
||||
"page_url": page_url,
|
||||
"section": section,
|
||||
"word_count": str(word_count) if word_count else None,
|
||||
"keywords": keywords,
|
||||
}
|
||||
|
||||
if images:
|
||||
for i, img in enumerate(images[:2], 1):
|
||||
data[f"image_url_{i}"] = img
|
||||
|
||||
return self.fill_template(template, data)
|
||||
|
||||
def generate_faq(self, questions: list[dict[str, str]]) -> dict:
|
||||
"""Generate FAQPage schema."""
|
||||
schema = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"mainEntity": [],
|
||||
}
|
||||
|
||||
for qa in questions:
|
||||
schema["mainEntity"].append({
|
||||
"@type": "Question",
|
||||
"name": qa["question"],
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": qa["answer"],
|
||||
},
|
||||
})
|
||||
|
||||
return schema
|
||||
|
||||
def generate_breadcrumb(self, items: list[dict[str, str]]) -> dict:
|
||||
"""Generate BreadcrumbList schema."""
|
||||
schema = {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "BreadcrumbList",
|
||||
"itemListElement": [],
|
||||
}
|
||||
|
||||
for i, item in enumerate(items, 1):
|
||||
schema["itemListElement"].append({
|
||||
"@type": "ListItem",
|
||||
"position": i,
|
||||
"name": item["name"],
|
||||
"item": item["url"],
|
||||
})
|
||||
|
||||
return schema
|
||||
|
||||
def generate_website(
|
||||
self,
|
||||
name: str,
|
||||
url: str,
|
||||
search_url_template: str | None = None,
|
||||
description: str | None = None,
|
||||
language: str = "ko-KR",
|
||||
publisher_name: str | None = None,
|
||||
logo_url: str | None = None,
|
||||
alternate_name: str | None = None,
|
||||
) -> dict:
|
||||
"""Generate WebSite schema."""
|
||||
template = self.load_template("website")
|
||||
|
||||
data = {
|
||||
"site_name": name,
|
||||
"url": url,
|
||||
"description": description,
|
||||
"language": language,
|
||||
"search_url_template": search_url_template,
|
||||
"publisher_name": publisher_name or name,
|
||||
"logo_url": logo_url,
|
||||
"alternate_name": alternate_name,
|
||||
}
|
||||
|
||||
return self.fill_template(template, data)
|
||||
|
||||
def to_json_ld(self, schema: dict, pretty: bool = True) -> str:
|
||||
"""Convert schema dict to JSON-LD string."""
|
||||
indent = 2 if pretty else None
|
||||
return json.dumps(schema, ensure_ascii=False, indent=indent)
|
||||
|
||||
def to_html_script(self, schema: dict) -> str:
|
||||
"""Wrap schema in HTML script tag."""
|
||||
json_ld = self.to_json_ld(schema)
|
||||
return f'<script type="application/ld+json">\n{json_ld}\n</script>'
|
||||
|
||||
|
||||
def main():
|
||||
"""Main entry point for CLI usage."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate JSON-LD schema markup",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Generate Organization schema
|
||||
python schema_generator.py --type organization --name "My Company" --url "https://example.com"
|
||||
|
||||
# Generate Product schema
|
||||
python schema_generator.py --type product --name "Widget" --price 29900 --currency KRW
|
||||
|
||||
# Generate Article schema
|
||||
python schema_generator.py --type article --headline "Article Title" --author "John Doe"
|
||||
""",
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
"--type", "-t",
|
||||
required=True,
|
||||
choices=SchemaGenerator.SCHEMA_TYPES.keys(),
|
||||
help="Schema type to generate",
|
||||
)
|
||||
parser.add_argument("--name", help="Name/title")
|
||||
parser.add_argument("--url", help="URL")
|
||||
parser.add_argument("--description", help="Description")
|
||||
parser.add_argument("--price", type=float, help="Price (for product)")
|
||||
parser.add_argument("--currency", default="KRW", help="Currency code")
|
||||
parser.add_argument("--headline", help="Headline (for article)")
|
||||
parser.add_argument("--author", help="Author name")
|
||||
parser.add_argument("--output", "-o", help="Output file path")
|
||||
parser.add_argument("--html", action="store_true", help="Output as HTML script tag")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
generator = SchemaGenerator()
|
||||
|
||||
try:
|
||||
if args.type == "organization":
|
||||
schema = generator.generate_organization(
|
||||
name=args.name or "Organization Name",
|
||||
url=args.url or "https://example.com",
|
||||
description=args.description,
|
||||
)
|
||||
elif args.type == "product":
|
||||
schema = generator.generate_product(
|
||||
name=args.name or "Product Name",
|
||||
description=args.description or "Product description",
|
||||
price=args.price or 0,
|
||||
currency=args.currency,
|
||||
)
|
||||
elif args.type == "article":
|
||||
schema = generator.generate_article(
|
||||
headline=args.headline or args.name or "Article Title",
|
||||
description=args.description or "Article description",
|
||||
author_name=args.author or "Author",
|
||||
date_published=datetime.now().strftime("%Y-%m-%d"),
|
||||
publisher_name="Publisher",
|
||||
)
|
||||
elif args.type == "website":
|
||||
schema = generator.generate_website(
|
||||
name=args.name or "Website Name",
|
||||
url=args.url or "https://example.com",
|
||||
description=args.description,
|
||||
)
|
||||
elif args.type == "faq":
|
||||
# Example FAQ
|
||||
schema = generator.generate_faq([
|
||||
{"question": "Question 1?", "answer": "Answer 1"},
|
||||
{"question": "Question 2?", "answer": "Answer 2"},
|
||||
])
|
||||
elif args.type == "breadcrumb":
|
||||
# Example breadcrumb
|
||||
schema = generator.generate_breadcrumb([
|
||||
{"name": "Home", "url": "https://example.com/"},
|
||||
{"name": "Category", "url": "https://example.com/category/"},
|
||||
])
|
||||
elif args.type == "local_business":
|
||||
schema = generator.generate_local_business(
|
||||
name=args.name or "Business Name",
|
||||
business_type="store",
|
||||
address={"street": "123 Main St", "city": "Seoul", "country": "KR"},
|
||||
url=args.url,
|
||||
description=args.description,
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"Unsupported type: {args.type}")
|
||||
|
||||
if args.html:
|
||||
output = generator.to_html_script(schema)
|
||||
else:
|
||||
output = generator.to_json_ld(schema)
|
||||
|
||||
if args.output:
|
||||
with open(args.output, "w", encoding="utf-8") as f:
|
||||
f.write(output)
|
||||
logger.info(f"Schema written to {args.output}")
|
||||
else:
|
||||
print(output)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error generating schema: {e}")
|
||||
raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,32 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "{{article_type}}",
|
||||
"headline": "{{headline}}",
|
||||
"description": "{{description}}",
|
||||
"image": [
|
||||
"{{image_url_1}}",
|
||||
"{{image_url_2}}"
|
||||
],
|
||||
"datePublished": "{{date_published}}",
|
||||
"dateModified": "{{date_modified}}",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "{{author_name}}",
|
||||
"url": "{{author_url}}"
|
||||
},
|
||||
"publisher": {
|
||||
"@type": "Organization",
|
||||
"name": "{{publisher_name}}",
|
||||
"logo": {
|
||||
"@type": "ImageObject",
|
||||
"url": "{{publisher_logo_url}}"
|
||||
}
|
||||
},
|
||||
"mainEntityOfPage": {
|
||||
"@type": "WebPage",
|
||||
"@id": "{{page_url}}"
|
||||
},
|
||||
"articleSection": "{{section}}",
|
||||
"wordCount": "{{word_count}}",
|
||||
"keywords": "{{keywords}}"
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "BreadcrumbList",
|
||||
"itemListElement": [
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 1,
|
||||
"name": "{{level_1_name}}",
|
||||
"item": "{{level_1_url}}"
|
||||
},
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 2,
|
||||
"name": "{{level_2_name}}",
|
||||
"item": "{{level_2_url}}"
|
||||
},
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 3,
|
||||
"name": "{{level_3_name}}",
|
||||
"item": "{{level_3_url}}"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"mainEntity": [
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_1}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_1}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_2}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_2}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_3}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_3}}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "{{business_type}}",
|
||||
"name": "{{name}}",
|
||||
"description": "{{description}}",
|
||||
"url": "{{url}}",
|
||||
"telephone": "{{phone}}",
|
||||
"email": "{{email}}",
|
||||
"image": "{{image_url}}",
|
||||
"priceRange": "{{price_range}}",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{street_address}}",
|
||||
"addressLocality": "{{city}}",
|
||||
"addressRegion": "{{region}}",
|
||||
"postalCode": "{{postal_code}}",
|
||||
"addressCountry": "{{country}}"
|
||||
},
|
||||
"geo": {
|
||||
"@type": "GeoCoordinates",
|
||||
"latitude": "{{latitude}}",
|
||||
"longitude": "{{longitude}}"
|
||||
},
|
||||
"openingHoursSpecification": [
|
||||
{
|
||||
"@type": "OpeningHoursSpecification",
|
||||
"dayOfWeek": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"],
|
||||
"opens": "{{weekday_opens}}",
|
||||
"closes": "{{weekday_closes}}"
|
||||
},
|
||||
{
|
||||
"@type": "OpeningHoursSpecification",
|
||||
"dayOfWeek": ["Saturday", "Sunday"],
|
||||
"opens": "{{weekend_opens}}",
|
||||
"closes": "{{weekend_closes}}"
|
||||
}
|
||||
],
|
||||
"aggregateRating": {
|
||||
"@type": "AggregateRating",
|
||||
"ratingValue": "{{rating}}",
|
||||
"reviewCount": "{{review_count}}"
|
||||
},
|
||||
"sameAs": [
|
||||
"{{facebook_url}}",
|
||||
"{{instagram_url}}"
|
||||
]
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
"name": "{{name}}",
|
||||
"url": "{{url}}",
|
||||
"logo": "{{logo_url}}",
|
||||
"description": "{{description}}",
|
||||
"foundingDate": "{{founding_date}}",
|
||||
"founders": [
|
||||
{
|
||||
"@type": "Person",
|
||||
"name": "{{founder_name}}"
|
||||
}
|
||||
],
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{street_address}}",
|
||||
"addressLocality": "{{city}}",
|
||||
"addressRegion": "{{region}}",
|
||||
"postalCode": "{{postal_code}}",
|
||||
"addressCountry": "{{country}}"
|
||||
},
|
||||
"contactPoint": [
|
||||
{
|
||||
"@type": "ContactPoint",
|
||||
"telephone": "{{phone}}",
|
||||
"contactType": "customer service",
|
||||
"availableLanguage": ["Korean", "English"]
|
||||
}
|
||||
],
|
||||
"sameAs": [
|
||||
"{{facebook_url}}",
|
||||
"{{twitter_url}}",
|
||||
"{{linkedin_url}}",
|
||||
"{{instagram_url}}"
|
||||
]
|
||||
}
|
||||
@@ -1,76 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Product",
|
||||
"name": "{{name}}",
|
||||
"description": "{{description}}",
|
||||
"image": [
|
||||
"{{image_url_1}}",
|
||||
"{{image_url_2}}",
|
||||
"{{image_url_3}}"
|
||||
],
|
||||
"sku": "{{sku}}",
|
||||
"mpn": "{{mpn}}",
|
||||
"gtin13": "{{gtin13}}",
|
||||
"brand": {
|
||||
"@type": "Brand",
|
||||
"name": "{{brand_name}}"
|
||||
},
|
||||
"offers": {
|
||||
"@type": "Offer",
|
||||
"url": "{{product_url}}",
|
||||
"price": "{{price}}",
|
||||
"priceCurrency": "{{currency}}",
|
||||
"priceValidUntil": "{{price_valid_until}}",
|
||||
"availability": "https://schema.org/{{availability}}",
|
||||
"itemCondition": "https://schema.org/{{condition}}",
|
||||
"seller": {
|
||||
"@type": "Organization",
|
||||
"name": "{{seller_name}}"
|
||||
},
|
||||
"shippingDetails": {
|
||||
"@type": "OfferShippingDetails",
|
||||
"shippingRate": {
|
||||
"@type": "MonetaryAmount",
|
||||
"value": "{{shipping_cost}}",
|
||||
"currency": "{{currency}}"
|
||||
},
|
||||
"deliveryTime": {
|
||||
"@type": "ShippingDeliveryTime",
|
||||
"handlingTime": {
|
||||
"@type": "QuantitativeValue",
|
||||
"minValue": "{{handling_min_days}}",
|
||||
"maxValue": "{{handling_max_days}}",
|
||||
"unitCode": "DAY"
|
||||
},
|
||||
"transitTime": {
|
||||
"@type": "QuantitativeValue",
|
||||
"minValue": "{{transit_min_days}}",
|
||||
"maxValue": "{{transit_max_days}}",
|
||||
"unitCode": "DAY"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"aggregateRating": {
|
||||
"@type": "AggregateRating",
|
||||
"ratingValue": "{{rating}}",
|
||||
"reviewCount": "{{review_count}}",
|
||||
"bestRating": "5",
|
||||
"worstRating": "1"
|
||||
},
|
||||
"review": [
|
||||
{
|
||||
"@type": "Review",
|
||||
"reviewRating": {
|
||||
"@type": "Rating",
|
||||
"ratingValue": "{{review_rating}}",
|
||||
"bestRating": "5"
|
||||
},
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "{{reviewer_name}}"
|
||||
},
|
||||
"reviewBody": "{{review_text}}"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "WebSite",
|
||||
"name": "{{site_name}}",
|
||||
"alternateName": "{{alternate_name}}",
|
||||
"url": "{{url}}",
|
||||
"description": "{{description}}",
|
||||
"inLanguage": "{{language}}",
|
||||
"potentialAction": {
|
||||
"@type": "SearchAction",
|
||||
"target": {
|
||||
"@type": "EntryPoint",
|
||||
"urlTemplate": "{{search_url_template}}"
|
||||
},
|
||||
"query-input": "required name=search_term_string"
|
||||
},
|
||||
"publisher": {
|
||||
"@type": "Organization",
|
||||
"name": "{{publisher_name}}",
|
||||
"logo": {
|
||||
"@type": "ImageObject",
|
||||
"url": "{{logo_url}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
---
|
||||
name: seo-schema-generator
|
||||
description: |
|
||||
JSON-LD structured data generator from templates for various content types.
|
||||
Triggers: generate schema, create JSON-LD, schema markup, structured data generator.
|
||||
---
|
||||
|
||||
# SEO Schema Generator
|
||||
|
||||
## Purpose
|
||||
|
||||
Generate JSON-LD structured data markup for various content types using templates.
|
||||
|
||||
## Core Capabilities
|
||||
|
||||
1. **Organization** - Company/brand information
|
||||
2. **LocalBusiness** - Physical location businesses
|
||||
3. **Article** - Blog posts and news articles
|
||||
4. **Product** - E-commerce products
|
||||
5. **FAQPage** - FAQ sections
|
||||
6. **BreadcrumbList** - Navigation breadcrumbs
|
||||
7. **WebSite** - Site-level with search action
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Identify content type
|
||||
2. Gather required information
|
||||
3. Generate JSON-LD from template
|
||||
4. Validate output
|
||||
5. Provide implementation instructions
|
||||
|
||||
## Schema Templates
|
||||
|
||||
### Organization
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
"name": "[Company Name]",
|
||||
"url": "[Website URL]",
|
||||
"logo": "[Logo URL]",
|
||||
"sameAs": [
|
||||
"[Social Media URLs]"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### LocalBusiness
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "LocalBusiness",
|
||||
"name": "[Business Name]",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "[Street]",
|
||||
"addressLocality": "[City]",
|
||||
"addressRegion": "[State]",
|
||||
"postalCode": "[ZIP]",
|
||||
"addressCountry": "[Country]"
|
||||
},
|
||||
"telephone": "[Phone]",
|
||||
"openingHours": ["Mo-Fr 09:00-17:00"]
|
||||
}
|
||||
```
|
||||
|
||||
### Article
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Article",
|
||||
"headline": "[Title]",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "[Author Name]"
|
||||
},
|
||||
"datePublished": "[YYYY-MM-DD]",
|
||||
"dateModified": "[YYYY-MM-DD]",
|
||||
"image": "[Image URL]",
|
||||
"publisher": {
|
||||
"@type": "Organization",
|
||||
"name": "[Publisher]",
|
||||
"logo": "[Logo URL]"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### FAQPage
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"mainEntity": [
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "[Question]",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "[Answer]"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Product
|
||||
```json
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Product",
|
||||
"name": "[Product Name]",
|
||||
"image": "[Image URL]",
|
||||
"description": "[Description]",
|
||||
"offers": {
|
||||
"@type": "Offer",
|
||||
"price": "[Price]",
|
||||
"priceCurrency": "[Currency]",
|
||||
"availability": "https://schema.org/InStock"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
Place generated JSON-LD in `<head>` section:
|
||||
|
||||
```html
|
||||
<head>
|
||||
<script type="application/ld+json">
|
||||
[Generated Schema Here]
|
||||
</script>
|
||||
</head>
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
After generating:
|
||||
1. Use schema validator skill (13) to verify
|
||||
2. Test with Google Rich Results Test
|
||||
3. Monitor in Search Console
|
||||
|
||||
## Limitations
|
||||
|
||||
- Templates cover common types only
|
||||
- Complex nested schemas may need manual adjustment
|
||||
- Some Rich Results require additional properties
|
||||
|
||||
## Notion Output (Required)
|
||||
|
||||
All audit reports MUST be saved to OurDigital SEO Audit Log:
|
||||
- **Database ID**: `2c8581e5-8a1e-8035-880b-e38cefc2f3ef`
|
||||
- **Properties**: Issue (title), Site (url), Category, Priority, Found Date, Audit ID
|
||||
- **Language**: Korean with English technical terms
|
||||
- **Audit ID Format**: [TYPE]-YYYYMMDD-NNN
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
# Skill metadata (extracted from SKILL.md frontmatter)
|
||||
|
||||
name: seo-schema-generator
|
||||
description: |
|
||||
Schema markup generator for JSON-LD structured data. Triggers: generate schema, create JSON-LD, add structured data, schema markup.
|
||||
|
||||
# Optional fields
|
||||
allowed-tools:
|
||||
- mcp__firecrawl__*
|
||||
- mcp__perplexity__*
|
||||
|
||||
# triggers: [] # TODO: Extract from description
|
||||
@@ -1,32 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "{{article_type}}",
|
||||
"headline": "{{headline}}",
|
||||
"description": "{{description}}",
|
||||
"image": [
|
||||
"{{image_url_1}}",
|
||||
"{{image_url_2}}"
|
||||
],
|
||||
"datePublished": "{{date_published}}",
|
||||
"dateModified": "{{date_modified}}",
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "{{author_name}}",
|
||||
"url": "{{author_url}}"
|
||||
},
|
||||
"publisher": {
|
||||
"@type": "Organization",
|
||||
"name": "{{publisher_name}}",
|
||||
"logo": {
|
||||
"@type": "ImageObject",
|
||||
"url": "{{publisher_logo_url}}"
|
||||
}
|
||||
},
|
||||
"mainEntityOfPage": {
|
||||
"@type": "WebPage",
|
||||
"@id": "{{page_url}}"
|
||||
},
|
||||
"articleSection": "{{section}}",
|
||||
"wordCount": "{{word_count}}",
|
||||
"keywords": "{{keywords}}"
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "BreadcrumbList",
|
||||
"itemListElement": [
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 1,
|
||||
"name": "{{level_1_name}}",
|
||||
"item": "{{level_1_url}}"
|
||||
},
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 2,
|
||||
"name": "{{level_2_name}}",
|
||||
"item": "{{level_2_url}}"
|
||||
},
|
||||
{
|
||||
"@type": "ListItem",
|
||||
"position": 3,
|
||||
"name": "{{level_3_name}}",
|
||||
"item": "{{level_3_url}}"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"mainEntity": [
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_1}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_1}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_2}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_2}}"
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "{{question_3}}",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "{{answer_3}}"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "{{business_type}}",
|
||||
"name": "{{name}}",
|
||||
"description": "{{description}}",
|
||||
"url": "{{url}}",
|
||||
"telephone": "{{phone}}",
|
||||
"email": "{{email}}",
|
||||
"image": "{{image_url}}",
|
||||
"priceRange": "{{price_range}}",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{street_address}}",
|
||||
"addressLocality": "{{city}}",
|
||||
"addressRegion": "{{region}}",
|
||||
"postalCode": "{{postal_code}}",
|
||||
"addressCountry": "{{country}}"
|
||||
},
|
||||
"geo": {
|
||||
"@type": "GeoCoordinates",
|
||||
"latitude": "{{latitude}}",
|
||||
"longitude": "{{longitude}}"
|
||||
},
|
||||
"openingHoursSpecification": [
|
||||
{
|
||||
"@type": "OpeningHoursSpecification",
|
||||
"dayOfWeek": ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday"],
|
||||
"opens": "{{weekday_opens}}",
|
||||
"closes": "{{weekday_closes}}"
|
||||
},
|
||||
{
|
||||
"@type": "OpeningHoursSpecification",
|
||||
"dayOfWeek": ["Saturday", "Sunday"],
|
||||
"opens": "{{weekend_opens}}",
|
||||
"closes": "{{weekend_closes}}"
|
||||
}
|
||||
],
|
||||
"aggregateRating": {
|
||||
"@type": "AggregateRating",
|
||||
"ratingValue": "{{rating}}",
|
||||
"reviewCount": "{{review_count}}"
|
||||
},
|
||||
"sameAs": [
|
||||
"{{facebook_url}}",
|
||||
"{{instagram_url}}"
|
||||
]
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
"name": "{{name}}",
|
||||
"url": "{{url}}",
|
||||
"logo": "{{logo_url}}",
|
||||
"description": "{{description}}",
|
||||
"foundingDate": "{{founding_date}}",
|
||||
"founders": [
|
||||
{
|
||||
"@type": "Person",
|
||||
"name": "{{founder_name}}"
|
||||
}
|
||||
],
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{street_address}}",
|
||||
"addressLocality": "{{city}}",
|
||||
"addressRegion": "{{region}}",
|
||||
"postalCode": "{{postal_code}}",
|
||||
"addressCountry": "{{country}}"
|
||||
},
|
||||
"contactPoint": [
|
||||
{
|
||||
"@type": "ContactPoint",
|
||||
"telephone": "{{phone}}",
|
||||
"contactType": "customer service",
|
||||
"availableLanguage": ["Korean", "English"]
|
||||
}
|
||||
],
|
||||
"sameAs": [
|
||||
"{{facebook_url}}",
|
||||
"{{twitter_url}}",
|
||||
"{{linkedin_url}}",
|
||||
"{{instagram_url}}"
|
||||
]
|
||||
}
|
||||
@@ -1,76 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Product",
|
||||
"name": "{{name}}",
|
||||
"description": "{{description}}",
|
||||
"image": [
|
||||
"{{image_url_1}}",
|
||||
"{{image_url_2}}",
|
||||
"{{image_url_3}}"
|
||||
],
|
||||
"sku": "{{sku}}",
|
||||
"mpn": "{{mpn}}",
|
||||
"gtin13": "{{gtin13}}",
|
||||
"brand": {
|
||||
"@type": "Brand",
|
||||
"name": "{{brand_name}}"
|
||||
},
|
||||
"offers": {
|
||||
"@type": "Offer",
|
||||
"url": "{{product_url}}",
|
||||
"price": "{{price}}",
|
||||
"priceCurrency": "{{currency}}",
|
||||
"priceValidUntil": "{{price_valid_until}}",
|
||||
"availability": "https://schema.org/{{availability}}",
|
||||
"itemCondition": "https://schema.org/{{condition}}",
|
||||
"seller": {
|
||||
"@type": "Organization",
|
||||
"name": "{{seller_name}}"
|
||||
},
|
||||
"shippingDetails": {
|
||||
"@type": "OfferShippingDetails",
|
||||
"shippingRate": {
|
||||
"@type": "MonetaryAmount",
|
||||
"value": "{{shipping_cost}}",
|
||||
"currency": "{{currency}}"
|
||||
},
|
||||
"deliveryTime": {
|
||||
"@type": "ShippingDeliveryTime",
|
||||
"handlingTime": {
|
||||
"@type": "QuantitativeValue",
|
||||
"minValue": "{{handling_min_days}}",
|
||||
"maxValue": "{{handling_max_days}}",
|
||||
"unitCode": "DAY"
|
||||
},
|
||||
"transitTime": {
|
||||
"@type": "QuantitativeValue",
|
||||
"minValue": "{{transit_min_days}}",
|
||||
"maxValue": "{{transit_max_days}}",
|
||||
"unitCode": "DAY"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"aggregateRating": {
|
||||
"@type": "AggregateRating",
|
||||
"ratingValue": "{{rating}}",
|
||||
"reviewCount": "{{review_count}}",
|
||||
"bestRating": "5",
|
||||
"worstRating": "1"
|
||||
},
|
||||
"review": [
|
||||
{
|
||||
"@type": "Review",
|
||||
"reviewRating": {
|
||||
"@type": "Rating",
|
||||
"ratingValue": "{{review_rating}}",
|
||||
"bestRating": "5"
|
||||
},
|
||||
"author": {
|
||||
"@type": "Person",
|
||||
"name": "{{reviewer_name}}"
|
||||
},
|
||||
"reviewBody": "{{review_text}}"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "WebSite",
|
||||
"name": "{{site_name}}",
|
||||
"alternateName": "{{alternate_name}}",
|
||||
"url": "{{url}}",
|
||||
"description": "{{description}}",
|
||||
"inLanguage": "{{language}}",
|
||||
"potentialAction": {
|
||||
"@type": "SearchAction",
|
||||
"target": {
|
||||
"@type": "EntryPoint",
|
||||
"urlTemplate": "{{search_url_template}}"
|
||||
},
|
||||
"query-input": "required name=search_term_string"
|
||||
},
|
||||
"publisher": {
|
||||
"@type": "Organization",
|
||||
"name": "{{publisher_name}}",
|
||||
"logo": {
|
||||
"@type": "ImageObject",
|
||||
"url": "{{logo_url}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
# Firecrawl
|
||||
|
||||
> TODO: Document tool usage for this skill
|
||||
|
||||
## Available Commands
|
||||
|
||||
- [ ] List commands
|
||||
|
||||
## Configuration
|
||||
|
||||
- [ ] Add configuration details
|
||||
|
||||
## Examples
|
||||
|
||||
- [ ] Add usage examples
|
||||
@@ -1,15 +0,0 @@
|
||||
# Perplexity
|
||||
|
||||
> TODO: Document tool usage for this skill
|
||||
|
||||
## Available Commands
|
||||
|
||||
- [ ] List commands
|
||||
|
||||
## Configuration
|
||||
|
||||
- [ ] Add configuration details
|
||||
|
||||
## Examples
|
||||
|
||||
- [ ] Add usage examples
|
||||
@@ -0,0 +1,47 @@
|
||||
entity_id,entity_type,property,value,lang,url,source_ids,authority,confidence,conflict,status,note
|
||||
org:shilla,Organization,@id,https://www.shillahotels.com/#org,,,,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,name,The Shilla Hotels & Resorts,,,S-OFF|S-WIKI,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,legalName,주식회사 호텔신라,,,S-DART,1,high,,CONFIRMED,DART 법인명
|
||||
org:shilla,Organization,url,https://www.shillahotels.com/,,,S-OFF,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,foundingDate,1973-05-09,,,S-DART,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,sameAs,https://www.wikidata.org/wiki/Q494845|https://en.wikipedia.org/wiki/The_Shilla,,,S-WIKI|S-WD,2,high,,CONFIRMED,array via pipe
|
||||
org:shilla,Organization,address.streetAddress,동호로 249,,,S-DART,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,address.addressLocality,중구,,,S-DART,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,address.addressRegion,서울,,,S-DART,1,high,,CONFIRMED,
|
||||
org:shilla,Organization,address.addressCountry,KR,,,S-DART,1,high,,CONFIRMED,
|
||||
site:ko,WebSite,@id,https://www.shillahotels.com/ko#website,ko,https://www.shillahotels.com/ko/,S-OFF,1,high,,CONFIRMED,
|
||||
site:ko,WebSite,name,신라호텔,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
site:ko,WebSite,url,https://www.shillahotels.com/ko/,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
site:ko,WebSite,inLanguage,ko,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
site:ko,WebSite,publisher.@id,https://www.shillahotels.com/#org,ko,,,1,high,,CONFIRMED,ref to org
|
||||
hotel:theshilla-seoul,Hotel,@id,https://www.shillahotels.com/ko/theshilla/seoul#hotel,ko,https://www.shillahotels.com/ko/theshilla/seoul/index.do,S-OFF|S-BROCH,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,name,The Shilla Seoul,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,telephone,+82-2-2233-3131,ko,,S-OFF|S-GBP,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,priceRange,$$$$,ko,,S-OFF,2,med,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,brand.name,The Shilla,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,parentOrganization.@id,https://www.shillahotels.com/#org,ko,,,1,high,,CONFIRMED,entity graph link
|
||||
hotel:theshilla-seoul,Hotel,address.streetAddress,동호로 249,ko,,S-OFF|S-GBP,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,address.addressLocality,서울,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,address.addressCountry,KR,ko,,S-OFF,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,geo.latitude,37.5564,ko,,S-GBP,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,geo.longitude,127.0058,ko,,S-GBP,1,high,,CONFIRMED,
|
||||
person:ceo,Person,@id,https://www.shillahotels.com/#ceo,,,S-DART|S-NEWS,1,high,,CONFIRMED,
|
||||
person:ceo,Person,name,이부진,,,S-DART,1,high,,CONFIRMED,
|
||||
person:ceo,Person,jobTitle,대표이사 사장,,,S-DART,1,high,,CONFIRMED,
|
||||
person:ceo,Person,worksFor.@id,https://www.shillahotels.com/#org,,,,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,title,프런트오피스 매니저,ko,https://recruit.shilla.net/job/1234,S-RECRUIT,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,description,더 신라 서울 프런트오피스 운영 총괄 및 VIP 응대.,ko,,S-RECRUIT,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,datePosted,2026-05-01,ko,,S-RECRUIT,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,employmentType,FULL_TIME,ko,,S-RECRUIT,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,hiringOrganization.@id,https://www.shillahotels.com/#org,ko,,,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,jobLocation.addressLocality,서울,ko,,S-RECRUIT,1,high,,CONFIRMED,
|
||||
job:fo-manager,JobPosting,jobLocation.addressCountry,KR,ko,,S-RECRUIT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,name,The Shilla — Authentic Indulgence,,https://www.youtube.com/watch?v=XXXX,S-YT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,description,더 신라 브랜드 필름.,,,S-YT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,thumbnailUrl,https://i.ytimg.com/vi/XXXX/maxresdefault.jpg,,,S-YT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,uploadDate,2025-11-20,,,S-YT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,duration,PT1M45S,,,S-YT,1,high,,CONFIRMED,
|
||||
video:brand-film,VideoObject,publisher.@id,https://www.shillahotels.com/#org,,,,1,high,,CONFIRMED,
|
||||
hotel:theshilla-seoul,Hotel,image,https://example.com/seoul.jpg,ko,,S-OFF,3,low,,PENDING,이미지 최종본 미확정
|
||||
org:shilla,Organization,telephone,+82-2-2233-3131,,,S-OFF,2,med,Y,CONFIRMED,대표번호 vs IR번호 출처 충돌
|
||||
person:ceo,Person,image,,,,,,,,CONFIRMED,값 공란 -> 제외
|
||||
|
@@ -0,0 +1,14 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="ko">
|
||||
<head>
|
||||
<!-- No JSON-LD here → only meta/OG, seeded as PENDING (must be confirmed) -->
|
||||
<title>그랜드 조선 부산 — 객실 및 예약</title>
|
||||
<meta name="description" content="해운대 해변에 위치한 럭셔리 호텔, 그랜드 조선 부산.">
|
||||
<meta property="og:type" content="business.business">
|
||||
<meta property="og:title" content="그랜드 조선 부산">
|
||||
<meta property="og:url" content="https://www.josunhotel.com/grand-busan">
|
||||
<meta property="og:image" content="https://www.josunhotel.com/grand-busan.jpg">
|
||||
<link rel="canonical" href="https://www.josunhotel.com/grand-busan">
|
||||
</head>
|
||||
<body><h1>그랜드 조선 부산</h1></body>
|
||||
</html>
|
||||
@@ -0,0 +1,45 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="ko">
|
||||
<head>
|
||||
<title>조선호텔앤리조트 — 공식 홈페이지</title>
|
||||
<meta property="og:site_name" content="조선호텔앤리조트">
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:url" content="https://www.josunhotel.com/">
|
||||
<link rel="canonical" href="https://www.josunhotel.com/">
|
||||
<!-- Existing JSON-LD on the live page → extracted as CONFIRMED claims -->
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
{
|
||||
"@type": "Organization",
|
||||
"@id": "https://www.josunhotel.com/#org",
|
||||
"name": "조선호텔앤리조트",
|
||||
"url": "https://www.josunhotel.com/",
|
||||
"logo": "https://www.josunhotel.com/logo.png",
|
||||
"sameAs": [
|
||||
"https://www.instagram.com/josunhotelsandresorts/",
|
||||
"https://www.wikidata.org/wiki/Q567458"
|
||||
]
|
||||
},
|
||||
{
|
||||
"@type": "Hotel",
|
||||
"@id": "https://www.josunhotel.com/westin#hotel",
|
||||
"name": "웨스틴 조선 서울",
|
||||
"url": "https://www.josunhotel.com/westin",
|
||||
"telephone": "+82-2-771-0500",
|
||||
"priceRange": "$$$$",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "소공로 106",
|
||||
"addressLocality": "서울",
|
||||
"addressRegion": "중구",
|
||||
"addressCountry": "KR"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
</script>
|
||||
</head>
|
||||
<body><h1>조선호텔앤리조트</h1></body>
|
||||
</html>
|
||||
@@ -0,0 +1,59 @@
|
||||
# Entity → Schema Type Map (source-to-schema, pre-launch)
|
||||
|
||||
Maps the source/entity kinds collected in steps 1–2 to schema.org types, and lists the
|
||||
Google rich-result requirements each type must satisfy. Required/recommended columns are
|
||||
kept in sync with `16-seo-schema-validator/scripts/schema_rules.json` so drafts pass the gate.
|
||||
|
||||
## Type map
|
||||
|
||||
| Source / entity | Type | Google required | Key recommended | Cardinality |
|
||||
|-----------------|------|-----------------|-----------------|-------------|
|
||||
| Corporate/legal (DART, official About) | `Organization` | `name`, `url` | `logo`, `sameAs`, `address`, `contactPoint` | 1 per legal entity |
|
||||
| Per-language site | `WebSite` | `name`, `url` | `inLanguage`, `publisher` | 1 per language |
|
||||
| Hotel property | `Hotel` | `name`, `address` | `telephone`, `priceRange`, `geo`, `image`, `brand` | 1 per property × language |
|
||||
| Executive / person | `Person` | `name` | `jobTitle`, `worksFor`, `sameAs`, `url` | 1 per person |
|
||||
| Recruitment posting | `JobPosting` | `title`, `description`, `datePosted`, `hiringOrganization`, `jobLocation` | `validThrough`, `employmentType`, `baseSalary` | 1 per open role |
|
||||
| Official video | `VideoObject` | `name`, `description`, `thumbnailUrl`, `uploadDate` | `duration`, `contentUrl`, `embedUrl` | 1 per featured video |
|
||||
| FAQ / press-kit Q&A | `FAQPage` | `mainEntity` (Question→Answer) | `inLanguage`, `url` | 1 per FAQ page |
|
||||
| Site navigation | `BreadcrumbList` | `itemListElement` | — | 1 per page |
|
||||
|
||||
Note: `JobPosting` and `VideoObject` were added to the validator rule set for this workflow,
|
||||
since recruitment sites and the official YouTube channel are listed source types.
|
||||
|
||||
## Address requirement nuance (avoid false P0)
|
||||
|
||||
`PostalAddress` requires only `addressLocality` + `addressCountry` as the context-safe minimum.
|
||||
`streetAddress` is **recommended** (P2), because it is expected for `Hotel`/`LocalBusiness` NAP
|
||||
but NOT required by Google for `JobPosting.jobLocation` (city/region/country suffices). For
|
||||
hotels, the L4 NAP-consistency check still enforces a complete, identical street address across
|
||||
all pages of a property — so the street address signal is not lost where it matters.
|
||||
|
||||
## Brand-tier rule (Shilla)
|
||||
|
||||
- The Shilla, Shilla Monogram → `Hotel`
|
||||
- Shilla Stay → `Hotel` (preferred; it is lodging, not a generic LocalBusiness)
|
||||
- Always set `brand.name` (e.g. "The Shilla", "Shilla Monogram", "Shilla Stay") and link
|
||||
`parentOrganization.@id` back to the single `Organization` node.
|
||||
|
||||
## @id entity graph (critical for pre-launch coherence)
|
||||
|
||||
Author stable `@id` URIs and cross-reference them so search engines read the portfolio as one
|
||||
connected entity graph rather than disconnected snippets:
|
||||
|
||||
```
|
||||
Organization @id = https://www.shillahotels.com/#org
|
||||
▲ parentOrganization ▲ publisher ▲ hiringOrganization ▲ worksFor / publisher
|
||||
│ │ │ │
|
||||
Hotel (per property) WebSite (per lang) JobPosting Person / VideoObject
|
||||
```
|
||||
|
||||
Rules
|
||||
- One `Organization` node, one `@id`, referenced by every other node via `@id`.
|
||||
- `@id` must be an absolute, stable URI (the provisional production URL + a `#fragment`).
|
||||
- Every `@id` referenced must also be defined somewhere in the dataset (the validator's L4
|
||||
checks for dangling `@id` references).
|
||||
|
||||
## When NOT to author schema
|
||||
- Authenticated/transactional pages (mypage, login, booking cart) → no schema.
|
||||
- Thin or duplicate pages → no schema until content exists.
|
||||
- Any entity whose facts are still `conflict`/`PENDING` → resolve first (builder excludes them).
|
||||
@@ -0,0 +1,76 @@
|
||||
# Site-Extraction Methodology (Mode 1 — from an existing website)
|
||||
|
||||
How to turn an existing site into a claims register, and why this mode is *easier*
|
||||
than Mode 2 (collected sources) but still must not blindly trust what it scrapes.
|
||||
|
||||
Pair skill: extraction is `scripts/extract_site_claims.py`; the build engine and the
|
||||
QA gate are shared with Mode 2. See `source-to-schema-methodology.md` for Mode 2.
|
||||
|
||||
## Why a website is the easy case (and where it still bites)
|
||||
|
||||
A published site has a **single source of truth** — the pages themselves — so there is
|
||||
little to reconcile. The risks are different from Mode 2:
|
||||
|
||||
| Risk on an existing site | What it causes | Countermeasure |
|
||||
|---|---|---|
|
||||
| Trusting inferred meta as fact | wrong/old values shipped as schema | meta/OG seeded **PENDING**, never auto-shipped |
|
||||
| Existing JSON-LD is partial or stale | gaps, outdated facts | extracted as CONFIRMED but **spot-checked** at review |
|
||||
| Many near-identical pages | duplicate descriptions, bloated register | one entity per real thing; let Layer 4 catch dupes |
|
||||
| JS-rendered schema not in raw HTML | "nothing extracted" | use a rendered snapshot / live fetch, or fall to Mode 2 |
|
||||
|
||||
## The 5 steps
|
||||
|
||||
### 1. Choose the pages
|
||||
Pick the canonical page per entity (home, about/company, each property/location, key
|
||||
product/FAQ pages). One representative page per entity is enough to seed it; you don't
|
||||
need the whole crawl.
|
||||
|
||||
### 2. Extract
|
||||
Run the adapter on URLs, local `.html` files, or a directory (offline):
|
||||
```bash
|
||||
python scripts/extract_site_claims.py https://site/ https://site/about --out site_claims
|
||||
python scripts/extract_site_claims.py ./snapshot/ --out site_claims # offline
|
||||
```
|
||||
It produces two tiers of claims:
|
||||
- **Existing JSON-LD → `CONFIRMED` (authority 1).** The site already published these
|
||||
facts about itself; flattened to dotted-path claims.
|
||||
- **`<title>` / meta description / OpenGraph / `<html lang>` / canonical → `PENDING`
|
||||
(authority 2).** Inferred, not authoritative. These will **not** ship until confirmed.
|
||||
|
||||
### 3. Review the register (the critical human step)
|
||||
Open `site_claims/claims_register.csv`:
|
||||
- **Spot-check CONFIRMED rows** — extraction is faithful, but the site's own JSON-LD can
|
||||
be wrong/stale. Correct values; clear nothing silently.
|
||||
- **Confirm or drop PENDING rows** — set `status=CONFIRMED` only for facts you've verified;
|
||||
delete the rest. PENDING rows are excluded by the builder by design.
|
||||
- **Add what the page didn't expose** — telephone, full address, `geo`, `sameAs`,
|
||||
`priceRange`. The richest schema usually needs facts no single page renders.
|
||||
- Set `conflict=Y` on any value you're unsure about to keep it out until resolved.
|
||||
|
||||
### 4. Build
|
||||
```bash
|
||||
python scripts/build_schema_drafts.py site_claims/claims_register.csv --out drafts_out
|
||||
```
|
||||
Unfilled slots are pruned; only CONFIRMED, non-conflicting claims become schema. Read
|
||||
`drafts_out/build_report.md` for everything excluded and why.
|
||||
|
||||
### 5. Validate (the gate)
|
||||
```bash
|
||||
python ../16-seo-schema-validator/scripts/validate_schema.py \
|
||||
drafts_out/schema_drafts_dataset.csv --out qa_out
|
||||
```
|
||||
Gate = **zero P0**. Fix P0, re-build, re-validate, then open client review against the
|
||||
report (not raw JSON).
|
||||
|
||||
## When NOT to use Mode 1
|
||||
|
||||
If the existing site **already has good, complete JSON-LD**, you don't need to regenerate
|
||||
it — **audit it in place** with `16-seo-schema-validator` Mode B
|
||||
(`validate_schema.py --live <URL>`). Mode 1 is for sites whose pages carry the *facts* but
|
||||
not yet the *structured data*, or whose schema needs a rebuild.
|
||||
|
||||
## entity_id convention
|
||||
|
||||
The adapter assigns `prefix:slug` ids (`org:`, `site:`, `hotel:`, `dining:`, `page:`, …)
|
||||
derived from each node's `@id` fragment or page URL. Rename them to stable, human ids
|
||||
during review (e.g. `hotel:theshilla-seoul`) so re-runs and Mode 2 additions line up.
|
||||
@@ -0,0 +1,62 @@
|
||||
# 출처 권위 위계 · 출처추적(Provenance) · 충돌 해소
|
||||
|
||||
미발행 사이트 스키마 저작의 성패는 "사실을 스키마로 굳히기 전에 단일 확정값을 만들 수 있는가"에 달려 있다. 그 판단 규칙을 명문화한다.
|
||||
|
||||
## 1. 출처 권위 위계 (authority rank)
|
||||
|
||||
값이 충돌할 때 **상위 권위 출처가 이긴다.** 클레임 레지스터의 `authority` 열에 1~5로 기록한다.
|
||||
|
||||
| 순위 | 출처 유형 | 신뢰 대상(어떤 사실에 권위) |
|
||||
|:---:|-----------|------------------------------|
|
||||
| **1** | 기업공시(DART), 사업자등록 정보 | 법인명·설립일·본사주소·대표자 (법적 사실) |
|
||||
| **1** | 공식 홈페이지/IR, 프레스킷 | 브랜드 표기·연락처·URL·로고 (공식 표기) |
|
||||
| **2** | 지속가능경영보고서, 사보, 공식 발간물 | 서사·정책·시설 스펙 |
|
||||
| **2** | Wikidata, 위키백과 | 엔티티 식별자(Q-ID)·sameAs·국제 표기 |
|
||||
| **3** | 주요 미디어 기사 | 사건·인용·맥락 (교차검증용) |
|
||||
| **4** | 인물정보/집계 사이트, 소셜 | 보조·후보값 (단독 근거 불가) |
|
||||
|
||||
규칙
|
||||
- **법적 사실**(법인명/설립일/주소)은 순위 1(공시) 우선.
|
||||
- **공식 표기**(브랜드명/전화/URL)는 순위 1(공식 채널) 우선.
|
||||
- **국제 식별/연결**(sameAs, 외국어 표기)은 Wikidata 우선.
|
||||
- 순위 4 단독으로는 CONFIRMED 불가 → 상위 출처로 교차검증 필수.
|
||||
|
||||
## 2. 출처추적(provenance)을 남기는 이유
|
||||
|
||||
발행된 페이지 기반 작업은 "그 페이지가 근거"라는 자명한 출처가 있다. 미발행 작업은 그렇지 않으므로 **모든 클레임에 출처를 명시**해야 한다.
|
||||
|
||||
- `source_ids` — 소스 레지스터의 출처 ID(파이프로 복수). 예: `S-DART|S-OFF`
|
||||
- 효용:
|
||||
1. 충돌 시 권위 비교의 근거가 된다.
|
||||
2. 럭셔리 브랜드 특성상 **사실 오류는 PR/법적 리스크** — 근거 추적이 방어선.
|
||||
3. 런칭 후 사실 변경 시 어느 클레임을 갱신할지 즉시 특정 가능.
|
||||
|
||||
## 3. 충돌 해소 절차
|
||||
|
||||
```
|
||||
값 충돌 발견
|
||||
│
|
||||
├─ 권위 순위가 다른가? ──예──▶ 상위 출처 채택, 하위는 note에 기록, status=CONFIRMED
|
||||
│
|
||||
└─ 동순위 충돌인가?
|
||||
├─ 최신성(retrieved_date) 우선 적용 가능? ──예──▶ 최신 채택
|
||||
└─ 판단 불가 ──▶ conflict=Y 유지, status=PENDING
|
||||
→ 빌더가 자동 제외하고 build_report에 보고
|
||||
→ 고객/이해관계자 질의로 확정 (최소 단위 질문)
|
||||
```
|
||||
|
||||
원칙: **충돌이 미해소면 스키마에 넣지 않는다.** 모순된 사실로 만든 스키마는 NAP 불일치·KG 혼선으로 직결된다. 비우는 편이 틀리는 것보다 낫다.
|
||||
|
||||
## 4. 엔티티 정합(reconciliation) — 동명 함정
|
||||
|
||||
- 모든 핵심 엔티티는 **Wikidata Q-ID**로 못박는다(예: 호텔 법인 vs 동명 역사·지명).
|
||||
- `sameAs`에는 검증된 식별 URL만: Wikidata, 위키백과, 공식 소셜.
|
||||
- 미디어 기사 URL은 sameAs가 아님(엔티티 식별자가 아니라 언급).
|
||||
- Knowledge Panel이 이미 있으면 그 표기를 공식 표기와 대조해 일치시킨다.
|
||||
|
||||
## 5. CONFIRMED 승격 체크리스트
|
||||
- [ ] 값이 정규화됨(전화 E.164 / 날짜 ISO 8601 / 언어 BCP-47 / 국가 ISO alpha-2)
|
||||
- [ ] `source_ids` 1개 이상, 핵심 사실은 권위 순위 1~2
|
||||
- [ ] 동일 속성 충돌 없음(`conflict` 비어 있음)
|
||||
- [ ] 엔티티는 `sameAs`/Q-ID로 식별 정합 완료
|
||||
- 위 충족 시에만 `status=CONFIRMED` → 빌더가 스키마로 채택.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Source-to-Schema 표준 프로세스 (미발행 사이트용 Schema 저작)
|
||||
|
||||
> 대상: 아직 발행되지 않은 웹사이트의 구조화 데이터(JSON-LD)를 **텍스트 소스로부터 저작**하는 작업.
|
||||
> 짝 스킬: 생성/저작은 `17-seo-schema-generator`(본 문서 = Mode 2 소스 기반), 검증은 `16-seo-schema-validator`.
|
||||
|
||||
---
|
||||
|
||||
## 0. 왜 이 작업이 "발행된 페이지 기반"보다 어려운가
|
||||
|
||||
발행된 페이지 기반 스키마는 **DOM이라는 단일 진실원본(single source of truth)** 이 이미 있고, 거기서 *추출*만 하면 된다. 미발행 사이트는 그 진실원본이 존재하지 않기 때문에 다음 네 가지 난점이 동시에 발생한다.
|
||||
|
||||
| # | 난점 | 결과적으로 생기는 결함 | 대응 원칙 |
|
||||
|---|------|----------------------|-----------|
|
||||
| 1 | 사실이 여러 출처에 흩어져 있고 서로 **충돌** (DART vs 위키 vs 브로셔) | NAP 불일치, 값 모순 | **출처 권위 위계**로 단일 값 확정 |
|
||||
| 2 | 붙일 **URL이 아직 없음** | placeholder/TODO 누출 (최다 P0) | 확정값 없으면 **키 자체를 생성하지 않음** |
|
||||
| 3 | 엔티티 식별을 **사람이 수동**으로 (어느 "신라"인가) | 잘못된 sameAs, 엔티티 혼선 | **Wikidata/Wikipedia 정합(reconciliation)** |
|
||||
| 4 | 무엇을 만들지 **범위 자체가 미정** | 누락/과잉 엔티티 | **엔티티-타입 맵**으로 범위 선확정 |
|
||||
|
||||
**결론적 설계 원칙**: 스키마로 굳히기 *전에* "출처 → 클레임(claim) 확정"을 먼저 끝낸다. 정제되지 않은 사실을 곧장 JSON-LD에 부으면 모든 충돌·공백이 그대로 스키마 결함이 된다. 그래서 본 프로세스의 중심축은 **클레임 레지스터(claims register)** — 출처가 추적되고 충돌이 해소된 사실 대장 — 이며, **CONFIRMED 클레임만 스키마가 된다.**
|
||||
|
||||
---
|
||||
|
||||
## 9단계 표준 프로세스
|
||||
|
||||
각 단계는 목적 / 입력 / 절차 / 산출 / 완료기준(AC) / 리스크로 명세한다.
|
||||
|
||||
### 1단계. 온라인 정통 소스(authentic source) 수집
|
||||
- **목적**: 권위 있는 1차 출처를 폭넓게 확보한다.
|
||||
- **입력**: 대상 기업/브랜드명, 법인명, 도메인.
|
||||
- **절차**: 다음을 수집·기록 → `templates/source-register.csv`
|
||||
- 기업공시(**DART**) — 법인명/설립일/주소/대표자 (법적 사실의 최상위 권위)
|
||||
- **공식 홈페이지**(About/푸터/IR), 지속가능경영보고서, 뉴스룸/미디어, 뉴스레터
|
||||
- **Wikidata / 위키백과** — 엔티티 식별자(Q-ID)와 sameAs 후보
|
||||
- 채용 사이트 — JobPosting 원천
|
||||
- 공식 **YouTube 채널** — VideoObject 원천
|
||||
- 공식 소셜/디지털 채널 — sameAs 후보
|
||||
- 주요 미디어 기사, 인물정보 사이트 — 보조/교차검증용
|
||||
- **산출**: 소스 레지스터(출처별 1행, 권위순위·언어·커버 엔티티 기록).
|
||||
- **AC**: 각 핵심 엔티티에 대해 **최소 2개 독립 출처** 확보(교차검증 가능).
|
||||
- **리스크**: 비공식·오래된 출처를 1차로 오인 → 권위순위를 반드시 명시.
|
||||
|
||||
### 2단계. 오프라인 콘텐츠 수집
|
||||
- **목적**: 온라인에 없는 1차 사실(브랜드 서사, 시설 스펙, 공식 표기) 확보.
|
||||
- **입력**: 브로셔 PDF, 사보/경영보고서, 프레스 킷, 보도자료 모음, 발간물.
|
||||
- **절차**: 파일을 소스 레지스터에 등록(파일 경로·발행일·언어). PDF는 텍스트 추출(스캔본은 OCR) 후 출처 표기 유지.
|
||||
- **산출**: 오프라인 출처도 동일 레지스터에 통합.
|
||||
- **AC**: 모든 오프라인 소스에 `retrieved_date`와 `authority` 기재.
|
||||
- **리스크**: PDF 추출 시 인코딩 깨짐/표 붕괴 → 추출 직후 육안 점검.
|
||||
|
||||
### 3단계. 정규화 & 정제(distill)
|
||||
- **목적**: 수집물을 **클레임 단위**로 분해하고 노이즈·중복·빈값을 제거한다.
|
||||
- **입력**: 1·2단계 소스.
|
||||
- **절차**:
|
||||
1. 각 소스에서 사실을 (엔티티, 속성, 값, 출처) 단위로 추출 → `templates/claims-register.csv`
|
||||
2. 동일 (엔티티, 속성)의 **중복 통합**, 빈값/노이즈 제거
|
||||
3. 출처 충돌 시 `conflict=Y`로 표시(해소 전까지 스키마 진입 차단)
|
||||
4. 표기 정규화: 전화(E.164), 날짜(ISO 8601), 언어코드(BCP-47), 국가(ISO 3166-1 alpha-2)
|
||||
- **산출**: 1차 클레임 레지스터(아직 미확정 포함).
|
||||
- **AC**: 모든 핵심 속성이 단일 정규화 값 후보를 가짐(또는 conflict/pending로 명시).
|
||||
- **리스크**: 정규화 누락이 다운스트림 VAL 결함으로 직결 → 3단계에서 포맷 확정.
|
||||
|
||||
### 4단계. 텍스트 분석 & Knowledge Graph 교차검증
|
||||
- **목적**: 클레임을 외부 KG와 대조해 **이중 점검**하고, 동시에 현황·개선과제를 도출한다.
|
||||
- **입력**: 1차 클레임 레지스터.
|
||||
- **절차**:
|
||||
1. **엔티티 정합**: Wikidata Q-ID, 위키백과, Google Knowledge Panel과 대조 → `sameAs` 확정
|
||||
2. 핵심 사실(설립일, 법인명, 대표자)을 KG 값과 비교 → 불일치는 `conflict`
|
||||
3. KG에 **존재하지 않거나 빈약한 엔티티**를 식별 → *개선과제 자료*로 별도 기록(런칭 후 KG 강화 목표)
|
||||
4. 충돌·미확정을 권위 위계로 해소 → `status=CONFIRMED` 승격
|
||||
- **산출**: 확정 클레임 레지스터 + **KG 현황/개선과제 메모**(별도 컨설팅 산출물로 활용).
|
||||
- **AC**: 모든 핵심 엔티티에 검증된 `sameAs` 1개 이상; conflict 0건.
|
||||
- **리스크**: 동명 엔티티 오정합(예: "신라" 왕조 vs 호텔) → Q-ID로 못박기.
|
||||
- 참고: `references/source-authority-hierarchy.md`
|
||||
|
||||
### 5단계. 유형별 활용 스키마 타입 분류
|
||||
- **목적**: 어떤 엔티티에 어떤 schema.org 타입을 쓸지 범위를 확정한다.
|
||||
- **입력**: 확정 클레임 + 페이지/엔티티 목록.
|
||||
- **절차**: 엔티티·소스 유형을 타입에 매핑(아래는 본 스킬 기본 매핑).
|
||||
|
||||
| 소스/엔티티 | schema.org 타입 |
|
||||
|-------------|-----------------|
|
||||
| 법인·브랜드(공시/공식) | `Organization` |
|
||||
| 언어별 사이트 | `WebSite` |
|
||||
| 호텔 프로퍼티 | `Hotel` (= LocalBusiness 계열) |
|
||||
| 임원/인물 | `Person` |
|
||||
| 채용공고 | `JobPosting` |
|
||||
| 공식 영상 | `VideoObject` |
|
||||
| FAQ/프레스킷 Q&A | `FAQPage` |
|
||||
| 사이트 내비게이션 | `BreadcrumbList` |
|
||||
|
||||
- **산출**: 엔티티-타입 맵(엔티티별 타입 + 필수 속성 목록).
|
||||
- **AC**: 모든 대상 엔티티에 타입과 Google 필수 속성 목록이 배정됨.
|
||||
- **리스크**: 과잉 타입 부여(불필요한 타입은 검증 부담만 가중) → "리치결과 가치 있는 타입" 우선.
|
||||
- 참고: `references/entity-and-type-map.md`
|
||||
|
||||
### 6단계. 타입별 템플릿 설정 & 초안 추출 (자동화)
|
||||
- **목적**: 확정 클레임 + 타입 템플릿으로 JSON-LD 초안을 **자동 생성**한다.
|
||||
- **입력**: 확정 클레임 레지스터(`status=CONFIRMED`), `scripts/type_templates.json`.
|
||||
- **절차**:
|
||||
```bash
|
||||
python scripts/build_schema_drafts.py path/to/claims_register.csv \
|
||||
--templates scripts/type_templates.json --out drafts_out
|
||||
```
|
||||
- CONFIRMED·비충돌 클레임만 스키마가 됨. PENDING/REJECTED/conflict/공란은 **제외 후 보고**.
|
||||
- 미충족 슬롯은 **키 자체를 삭제**(placeholder 누출 원천 차단).
|
||||
- 엔티티 간 `@id` 참조로 엔티티 그래프 형성(Hotel→parentOrganization 등).
|
||||
- **산출**: `drafts/*.jsonld`, 검증기 입력용 `schema_drafts_dataset.csv`, `build_report.md`(제외 클레임 목록).
|
||||
- **AC**: 초안에 placeholder/빈 객체 0건; 모든 제외 클레임이 보고서에 사유와 함께 기재.
|
||||
- **리스크**: 템플릿 누락 타입은 건너뜀(보고됨) → 5단계 맵과 템플릿 동기화.
|
||||
|
||||
### 7단계. 리뷰·검토·수정 + 리뷰 가이드
|
||||
- **목적**: 초안을 사람·고객이 검토하되, **원본 JSON이 아니라 결함 리포트**로 검토하게 한다.
|
||||
- **입력**: 6단계 초안 + 검증기 결과.
|
||||
- **절차**:
|
||||
1. 초안을 **즉시 검증기에 통과**(아래 8단계)시켜 P0=0 게이트부터 확보
|
||||
2. 남은 항목을 `templates/review-guide.md` 기준으로 검토(사실 정확성·표기·번역)
|
||||
3. 고객 검토는 P0가 0인 깨끗한 초안에 대해서만, 결함 리포트 기준으로 진행
|
||||
- **산출**: 수정 반영 초안 + 리뷰 가이드 체크 결과.
|
||||
- **AC**: 모든 P0 해소; 사실 정확성 검토 서명(저작자·검수자).
|
||||
- **리스크**: 사람이 원본 JSON을 직접 보면 "오류 과다" 문제 재발 → 반드시 리포트 기반 검토.
|
||||
- 참고: `templates/review-guide.md`
|
||||
|
||||
### 8단계. 수정 초안의 rich result 적격성 점검
|
||||
- **목적**: 리치결과 적격성을 (1) 오프라인 게이트 + (2) Google 온라인 테스트로 이중 확인.
|
||||
- **입력**: 수정 초안 데이터셋.
|
||||
- **절차**:
|
||||
```bash
|
||||
# 오프라인 게이트 (반드시 zero P0)
|
||||
python ../16-seo-schema-validator/scripts/validate_schema.py drafts_out/schema_drafts_dataset.csv --out qa_out
|
||||
```
|
||||
- 게이트 PASS 후, **표본 엔트리**를 Google Rich Results Test에 통과(이 런타임은 오프라인이라 온라인 테스트는 사용자가 수행)시켜 캡처.
|
||||
- **산출**: 검증기 리포트(Gate PASS) + Rich Results Test 표본 통과 캡처.
|
||||
- **AC**: P0=0, P1 트리아지 완료, 온라인 테스트 표본 green.
|
||||
- **리스크**: 오프라인 규칙은 호텔 도메인 큐레이션 부분집합 → 온라인 표본 검사로 보완.
|
||||
|
||||
### 9단계. 발행 후 유효성 검증 & KG 변화 측정
|
||||
- **목적**: 배포된 스키마가 저작 초안과 일치하는지 확인하고, 4단계의 KG 개선과제 달성도를 측정한다.
|
||||
- **입력**: 라이브 URL.
|
||||
- **절차**:
|
||||
1. 검증기 **Mode B**(라이브 URL)로 렌더링된 스키마 재검증 → `seo-comprehensive-audit` 4단계 연계
|
||||
2. GSC "리치 결과" 리포트 모니터링(신규 오류 0 유지)
|
||||
3. 4단계 KG 메모 대비 Knowledge Panel/Wikidata 노출·정확도 변화 측정
|
||||
- **산출**: 발행 후 검증 리포트 + KG 변화 측정(전/후 비교).
|
||||
- **AC**: 라이브 스키마 = 저작 초안; GSC 신규 구조화데이터 오류 0; KG 개선과제 진척 기록.
|
||||
- **리스크**: 렌더링 단계에서 JS로 스키마 누락/변형 → Mode B로 실측.
|
||||
|
||||
---
|
||||
|
||||
## 스테이지 게이트 (설계→개발→테스트→안정화→런칭 후)
|
||||
|
||||
| 게이트 | 단계 | DoD(완료 정의) |
|
||||
|--------|------|----------------|
|
||||
| **G1 설계** | 1·2·5 | 소스 레지스터 완료(엔티티당 ≥2출처), 엔티티-타입 맵 확정(타입+필수속성) |
|
||||
| **G2 개발** | 3·4·6 | 클레임 레지스터 CONFIRMED·conflict 0, 빌더 실행 → 초안 placeholder 0 |
|
||||
| **G3 테스트** | 7·8 | 검증기 **zero P0**, P1 트리아지(`decision-log`), 사실정확성 검수 서명 |
|
||||
| **G4 안정화** | 8 | Google Rich Results Test 표본 green, 재실행 무회귀 |
|
||||
| **G5 런칭 후** | 9 | 라이브=초안 일치, GSC 신규오류 0, KG 변화 측정 기록 |
|
||||
|
||||
---
|
||||
|
||||
## 산출물 일람
|
||||
- `templates/source-register.csv` — 1·2단계 출처 대장
|
||||
- `templates/claims-register.csv` — 3·4단계 사실 대장(스키마의 원천)
|
||||
- `templates/review-guide.md` — 7단계 검토 기준
|
||||
- `scripts/type_templates.json` — 6단계 타입별 JSON-LD 템플릿
|
||||
- `scripts/build_schema_drafts.py` — 6단계 초안 자동 생성
|
||||
- (검증) `16-seo-schema-validator` — 7·8·9단계 게이트
|
||||
@@ -0,0 +1,305 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
build_schema_drafts.py — Source-to-Schema draft generator (skill 17, pre-launch)
|
||||
|
||||
WHAT IT DOES
|
||||
Turns a *claims register* (reconciled, provenance-tracked facts) into JSON-LD
|
||||
drafts, then writes a dataset CSV that feeds straight into
|
||||
16-seo-schema-validator/scripts/validate_schema.py.
|
||||
|
||||
WHY A CLAIMS REGISTER FIRST (the core idea)
|
||||
Authoring schema for a site that does not exist yet is error-prone because the
|
||||
facts live in many conflicting sources (DART, Wikipedia, brochures...). If you
|
||||
pour raw, unreconciled facts straight into JSON-LD you reproduce every conflict
|
||||
and gap as a schema defect. So we reconcile facts FIRST (one confirmed value per
|
||||
property, with sources recorded), and only CONFIRMED, non-conflicting claims are
|
||||
allowed to become schema. Everything else is reported, not shipped.
|
||||
|
||||
THE PRUNING RULE (placeholder leakage is the #1 pre-launch defect)
|
||||
A template slot that has no confirmed value is DELETED — never emitted as
|
||||
"{{...}}" or "TODO". An empty nested object (only @type left) is dropped too.
|
||||
This guarantees drafts contain only backed facts.
|
||||
|
||||
INPUT (claims register): .csv or .xlsx with columns (case-insensitive, KR/EN aliases ok)
|
||||
entity_id e.g. org:shilla, hotel:theshilla-seoul (groups rows into one node)
|
||||
entity_type Organization | Hotel | Person | JobPosting | VideoObject | WebSite | FAQPage
|
||||
property schema.org property, dotted for nesting: address.streetAddress
|
||||
append nothing for scalars; arrays are handled automatically
|
||||
value the confirmed value (pipe-separate multiple values: a|b|c)
|
||||
lang optional (ko/en/ja/zh) -> produces one draft per language
|
||||
url optional target URL (provisional ok) -> carried to validator
|
||||
source_ids optional pipe-separated refs into the source register (provenance)
|
||||
authority optional 1..n (1 = most authoritative) — for your audit trail
|
||||
confidence optional high|med|low
|
||||
conflict optional — any truthy value (Y/1/true/충돌) EXCLUDES the claim
|
||||
status CONFIRMED (default if blank) | PENDING | REJECTED — only CONFIRMED ships
|
||||
note optional
|
||||
|
||||
USAGE
|
||||
python scripts/build_schema_drafts.py path/to/claims_register.csv \
|
||||
--templates scripts/type_templates.json --out drafts_out
|
||||
# then hand off to the validator:
|
||||
python ../16-seo-schema-validator/scripts/validate_schema.py \
|
||||
drafts_out/schema_drafts_dataset.csv --out qa_out
|
||||
"""
|
||||
|
||||
import argparse, csv, json, os, sys, copy, re
|
||||
from collections import defaultdict, OrderedDict
|
||||
|
||||
# ----------------------------------------------------------------------------- #
|
||||
# Column aliasing — accept Korean/English header variants #
|
||||
# ----------------------------------------------------------------------------- #
|
||||
COL_ALIASES = {
|
||||
"entity_id": ["entity_id", "entity", "엔티티", "엔티티id", "id"],
|
||||
"entity_type": ["entity_type", "type", "타입", "유형", "스키마타입"],
|
||||
"property": ["property", "prop", "속성", "프로퍼티", "path"],
|
||||
"value": ["value", "값", "내용"],
|
||||
"lang": ["lang", "language", "언어", "언어코드"],
|
||||
"url": ["url", "메뉴 url", "메뉴url", "주소"],
|
||||
"source_ids": ["source_ids", "source", "sources", "출처", "출처id"],
|
||||
"authority": ["authority", "권위", "권위순위"],
|
||||
"confidence": ["confidence", "신뢰도"],
|
||||
"conflict": ["conflict", "충돌", "conflict_flag"],
|
||||
"status": ["status", "상태"],
|
||||
"note": ["note", "notes", "비고", "메모"],
|
||||
}
|
||||
TRUTHY = {"y", "yes", "1", "true", "t", "충돌", "conflict", "o"}
|
||||
PLACEHOLDER_RE = re.compile(r"^\{\{(.+?)\}\}$") # matches an entire-string slot
|
||||
UNFILLED = object() # sentinel: slot had no value
|
||||
|
||||
|
||||
def _norm(s):
|
||||
return (s or "").strip().lower().replace(" ", "")
|
||||
|
||||
|
||||
def map_columns(headers):
|
||||
"""Return {canonical_name: actual_header} using the alias table."""
|
||||
lookup = {_norm(h): h for h in headers}
|
||||
out = {}
|
||||
for canon, aliases in COL_ALIASES.items():
|
||||
for a in aliases:
|
||||
if _norm(a) in lookup:
|
||||
out[canon] = lookup[_norm(a)]
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- #
|
||||
# Loading the claims register #
|
||||
# ----------------------------------------------------------------------------- #
|
||||
def load_rows(path):
|
||||
"""Yield dict rows from .csv or .xlsx. Keeps original header names."""
|
||||
ext = os.path.splitext(path)[1].lower()
|
||||
if ext in (".csv", ".tsv"):
|
||||
delim = "\t" if ext == ".tsv" else ","
|
||||
with open(path, encoding="utf-8-sig", newline="") as f:
|
||||
for row in csv.DictReader(f, delimiter=delim):
|
||||
yield row
|
||||
elif ext in (".xlsx", ".xlsm"):
|
||||
try:
|
||||
from openpyxl import load_workbook
|
||||
except ImportError:
|
||||
sys.exit("openpyxl required for .xlsx — pip install openpyxl")
|
||||
wb = load_workbook(path, read_only=True, data_only=True)
|
||||
for ws in wb.worksheets:
|
||||
rows = ws.iter_rows(values_only=True)
|
||||
try:
|
||||
headers = [str(h) if h is not None else "" for h in next(rows)]
|
||||
except StopIteration:
|
||||
continue
|
||||
if not map_columns(headers).get("entity_id"):
|
||||
continue # sheet without our schema -> skip
|
||||
for r in rows:
|
||||
yield {headers[i]: ("" if v is None else str(v))
|
||||
for i, v in enumerate(r) if i < len(headers)}
|
||||
else:
|
||||
sys.exit(f"Unsupported claims register format: {ext}")
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- #
|
||||
# Distil rows -> per-(entity, lang) confirmed claim maps + exclusion log #
|
||||
# ----------------------------------------------------------------------------- #
|
||||
def collect_claims(path):
|
||||
rows = list(load_rows(path))
|
||||
if not rows:
|
||||
sys.exit("Claims register is empty.")
|
||||
cmap = map_columns(rows[0].keys())
|
||||
for req in ("entity_id", "entity_type", "property", "value"):
|
||||
if req not in cmap:
|
||||
sys.exit(f"Missing required column '{req}'. Found: {list(rows[0].keys())}")
|
||||
|
||||
def g(row, key):
|
||||
col = cmap.get(key)
|
||||
return (row.get(col, "") if col else "").strip()
|
||||
|
||||
# claims[(entity_id, lang)] -> {"type":..., "url":..., "props": {path: [values]}}
|
||||
claims = OrderedDict()
|
||||
excluded = [] # (entity, prop, reason, detail)
|
||||
for row in rows:
|
||||
eid = g(row, "entity_id")
|
||||
if not eid:
|
||||
continue
|
||||
etype = g(row, "entity_type")
|
||||
prop = g(row, "property")
|
||||
val = g(row, "value")
|
||||
lang = g(row, "lang") or ""
|
||||
status = (g(row, "status") or "CONFIRMED").upper()
|
||||
conflict = _norm(g(row, "conflict")) in TRUTHY
|
||||
|
||||
if conflict:
|
||||
excluded.append((eid, prop, "CONFLICT", f"sources disagree -> resolve first"))
|
||||
continue
|
||||
if status == "REJECTED":
|
||||
excluded.append((eid, prop, "REJECTED", g(row, "note")))
|
||||
continue
|
||||
if status == "PENDING":
|
||||
excluded.append((eid, prop, "PENDING", "not yet confirmed by an authoritative source"))
|
||||
continue
|
||||
if not val:
|
||||
excluded.append((eid, prop, "EMPTY", "confirmed row but value is blank"))
|
||||
continue
|
||||
|
||||
key = (eid, lang)
|
||||
node = claims.setdefault(key, {"type": etype, "url": g(row, "url"),
|
||||
"props": defaultdict(list)})
|
||||
if etype and not node["type"]:
|
||||
node["type"] = etype
|
||||
if g(row, "url") and not node["url"]:
|
||||
node["url"] = g(row, "url")
|
||||
# pipe-separated value -> multiple values (array support)
|
||||
for v in (val.split("|") if "|" in val else [val]):
|
||||
v = v.strip()
|
||||
if v:
|
||||
node["props"][prop].append(v)
|
||||
return claims, excluded
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- #
|
||||
# Fill a template, pruning every unfilled slot #
|
||||
# ----------------------------------------------------------------------------- #
|
||||
def fill(node_template, props):
|
||||
"""Recursively fill {{slots}}; return UNFILLED when a branch has no real data."""
|
||||
if isinstance(node_template, str):
|
||||
m = PLACEHOLDER_RE.match(node_template.strip())
|
||||
if not m:
|
||||
return node_template # literal (e.g. "@type":"Hotel")
|
||||
path = m.group(1)
|
||||
is_array = path.endswith("[]")
|
||||
if is_array:
|
||||
path = path[:-2]
|
||||
vals = props.get(path, [])
|
||||
if not vals:
|
||||
return UNFILLED
|
||||
if is_array:
|
||||
return list(vals)
|
||||
if len(vals) > 1:
|
||||
print(f" ! multiple values for scalar '{path}' — using first ({len(vals)} given)")
|
||||
return vals[0]
|
||||
|
||||
if isinstance(node_template, dict):
|
||||
out = OrderedDict()
|
||||
for k, v in node_template.items():
|
||||
filled = fill(v, props)
|
||||
if filled is UNFILLED:
|
||||
continue
|
||||
out[k] = filled
|
||||
# an object that only carries @type/@context (no real data, no @id ref) is empty
|
||||
meaningful = [k for k in out if k not in ("@type", "@context")]
|
||||
if not meaningful:
|
||||
return UNFILLED
|
||||
return out
|
||||
|
||||
if isinstance(node_template, list):
|
||||
out = [x for x in (fill(i, props) for i in node_template) if x is not UNFILLED]
|
||||
return out if out else UNFILLED
|
||||
|
||||
return node_template
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------------- #
|
||||
# Main #
|
||||
# ----------------------------------------------------------------------------- #
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="Build JSON-LD drafts from a claims register.")
|
||||
ap.add_argument("claims", help="claims register .csv/.xlsx")
|
||||
ap.add_argument("--templates", default=os.path.join(os.path.dirname(__file__), "type_templates.json"))
|
||||
ap.add_argument("--out", default="drafts_out")
|
||||
args = ap.parse_args()
|
||||
|
||||
templates = json.load(open(args.templates, encoding="utf-8"))["templates"]
|
||||
claims, excluded = collect_claims(args.claims)
|
||||
|
||||
os.makedirs(os.path.join(args.out, "drafts"), exist_ok=True)
|
||||
dataset_rows = []
|
||||
built, skipped_type = 0, []
|
||||
|
||||
for (eid, lang), node in claims.items():
|
||||
etype = node["type"]
|
||||
if etype not in templates:
|
||||
skipped_type.append((eid, etype))
|
||||
continue
|
||||
filled = fill(copy.deepcopy(templates[etype]["tpl"]), node["props"])
|
||||
if filled is UNFILLED or not filled:
|
||||
skipped_type.append((eid, f"{etype} (no usable claims)"))
|
||||
continue
|
||||
jsonld = json.dumps(filled, ensure_ascii=False, indent=2)
|
||||
safe = re.sub(r"[^A-Za-z0-9]+", "_", eid).strip("_")
|
||||
fname = f"{safe}__{lang}.jsonld" if lang else f"{safe}.jsonld"
|
||||
with open(os.path.join(args.out, "drafts", fname), "w", encoding="utf-8") as f:
|
||||
f.write(jsonld)
|
||||
dataset_rows.append(OrderedDict([
|
||||
("entity_id", eid), ("entity_type", etype),
|
||||
("url", node["url"]), ("lang", lang),
|
||||
("jsonld", json.dumps(filled, ensure_ascii=False)),
|
||||
]))
|
||||
built += 1
|
||||
|
||||
# dataset CSV — directly consumable by validate_schema.py (auto-detects 'jsonld')
|
||||
ds_path = os.path.join(args.out, "schema_drafts_dataset.csv")
|
||||
with open(ds_path, "w", encoding="utf-8-sig", newline="") as f:
|
||||
w = csv.DictWriter(f, fieldnames=["entity_id", "entity_type", "url", "lang", "jsonld"])
|
||||
w.writeheader()
|
||||
w.writerows(dataset_rows)
|
||||
|
||||
# build report
|
||||
rep = [
|
||||
"# Schema Draft Build Report",
|
||||
"",
|
||||
f"- Entities built: **{built}**",
|
||||
f"- Claims excluded (not shipped): **{len(excluded)}**",
|
||||
f"- Entities skipped (no template / no usable claims): **{len(skipped_type)}**",
|
||||
"",
|
||||
"## Excluded claims (resolve before they can become schema)",
|
||||
]
|
||||
if excluded:
|
||||
rep.append("| entity | property | reason | detail |")
|
||||
rep.append("|--------|----------|--------|--------|")
|
||||
for eid, prop, reason, detail in excluded:
|
||||
rep.append(f"| {eid} | {prop} | **{reason}** | {detail} |")
|
||||
else:
|
||||
rep.append("_None._")
|
||||
if skipped_type:
|
||||
rep += ["", "## Skipped entities", ""]
|
||||
for eid, why in skipped_type:
|
||||
rep.append(f"- {eid} — {why}")
|
||||
rep += [
|
||||
"", "## Next step",
|
||||
"1. Resolve every excluded claim (confirm an authoritative value, clear conflicts).",
|
||||
"2. Re-run this builder.",
|
||||
"3. Validate the output (the QA gate):",
|
||||
" ```bash",
|
||||
f" python ../16-seo-schema-validator/scripts/validate_schema.py {ds_path} --out qa_out",
|
||||
" ```",
|
||||
"4. Fix all P0 from the validator, then proceed to client review.",
|
||||
]
|
||||
rep_path = os.path.join(args.out, "build_report.md")
|
||||
open(rep_path, "w", encoding="utf-8").write("\n".join(rep))
|
||||
|
||||
print(f"Built {built} drafts | excluded {len(excluded)} claims | skipped {len(skipped_type)} entities")
|
||||
print(f"Wrote: {ds_path}")
|
||||
print(f" {rep_path}")
|
||||
print(f" {os.path.join(args.out, 'drafts')}/*.jsonld")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,346 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
extract_site_claims.py — Scenario 1 adapter: an EXISTING website → claims register.
|
||||
|
||||
THE MERGE, IN ONE SENTENCE
|
||||
Schema generation has two scenarios that differ only in WHERE facts come from:
|
||||
1) from a given website — the live pages ARE the source of truth (this script)
|
||||
2) from collected sources — a not-yet-published site, facts scattered & conflicting
|
||||
Both emit the SAME claims_register.csv, which build_schema_drafts.py then turns into
|
||||
drafts and 16-seo-schema-validator gates. The claims register is the shared pivot
|
||||
that lets one skill cover both scenarios.
|
||||
|
||||
WHAT THIS DOES
|
||||
Reads pages of an existing site and seeds a claims register from them:
|
||||
- existing JSON-LD (<script type="application/ld+json">) -> CONFIRMED (authority 1):
|
||||
the site already published these facts about itself.
|
||||
- <title> / meta description / OpenGraph / <html lang> / canonical -> PENDING:
|
||||
inferred, not authoritative. The builder EXCLUDES PENDING claims until a human
|
||||
confirms them — so inference never silently ships (the skill's core principle).
|
||||
|
||||
You then review/edit claims_register.csv, run build_schema_drafts.py, and validate.
|
||||
|
||||
INPUT (any mix): URLs (needs `requests`), local .html files, or a directory of .html
|
||||
OUTPUT (in --out): claims_register.csv + extraction_report.md
|
||||
|
||||
USAGE
|
||||
python extract_site_claims.py https://example.com/ https://example.com/about --out site_claims
|
||||
python extract_site_claims.py ./snapshot/ --out site_claims # offline, local HTML
|
||||
# then:
|
||||
python build_schema_drafts.py site_claims/claims_register.csv --out drafts_out
|
||||
python ../16-seo-schema-validator/scripts/validate_schema.py drafts_out/schema_drafts_dataset.csv --out qa_out
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from collections import OrderedDict
|
||||
from html.parser import HTMLParser
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
JSONLD_RE = re.compile(
|
||||
r'<script[^>]+type=["\']application/ld\+json["\'][^>]*>(.*?)</script>',
|
||||
re.IGNORECASE | re.DOTALL,
|
||||
)
|
||||
|
||||
# entity_id prefix per schema type — keeps ids readable and groupable.
|
||||
TYPE_PREFIX = {
|
||||
"Organization": "org", "Corporation": "org", "LocalBusiness": "biz",
|
||||
"WebSite": "site", "WebPage": "page",
|
||||
"Hotel": "hotel", "LodgingBusiness": "hotel", "Resort": "hotel",
|
||||
"Restaurant": "dining", "FoodEstablishment": "dining", "BarOrPub": "dining",
|
||||
"Person": "person", "Product": "product", "Article": "article",
|
||||
"NewsArticle": "article", "BlogPosting": "article", "Event": "event",
|
||||
"FAQPage": "faq", "BreadcrumbList": "crumb",
|
||||
}
|
||||
# OpenGraph og:type -> our seed @type for the meta-only fallback.
|
||||
OG_TYPE_MAP = {"website": "WebSite", "article": "Article", "product": "Product",
|
||||
"business.business": "LocalBusiness", "profile": "Person"}
|
||||
|
||||
CLAIM_FIELDS = ["entity_id", "entity_type", "property", "value", "lang", "url",
|
||||
"source_ids", "authority", "confidence", "conflict", "status", "note"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Lightweight HTML meta extraction (stdlib only)
|
||||
# --------------------------------------------------------------------------- #
|
||||
class MetaParser(HTMLParser):
|
||||
"""Pull <title>, <html lang>, <link rel=canonical>, and <meta> name/property."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.title_parts = []
|
||||
self._in_title = False
|
||||
self.lang = ""
|
||||
self.canonical = ""
|
||||
self.meta = {} # name/property (lowercased) -> content
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
a = {k.lower(): (v or "") for k, v in attrs}
|
||||
if tag == "html" and a.get("lang"):
|
||||
self.lang = a["lang"].strip()
|
||||
elif tag == "title":
|
||||
self._in_title = True
|
||||
elif tag == "link" and a.get("rel", "").lower() == "canonical" and a.get("href"):
|
||||
self.canonical = a["href"].strip()
|
||||
elif tag == "meta":
|
||||
key = (a.get("property") or a.get("name") or "").lower().strip()
|
||||
if key and "content" in a:
|
||||
self.meta.setdefault(key, a["content"].strip())
|
||||
|
||||
def handle_endtag(self, tag):
|
||||
if tag == "title":
|
||||
self._in_title = False
|
||||
|
||||
def handle_data(self, data):
|
||||
if self._in_title:
|
||||
self.title_parts.append(data)
|
||||
|
||||
@property
|
||||
def title(self):
|
||||
return re.sub(r"\s+", " ", "".join(self.title_parts)).strip()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Page acquisition
|
||||
# --------------------------------------------------------------------------- #
|
||||
def gather_pages(inputs):
|
||||
"""Yield (url, html). Accepts http(s) URLs, .html files, and directories."""
|
||||
for item in inputs:
|
||||
if re.match(r"^https?://", item, re.IGNORECASE):
|
||||
try:
|
||||
import requests
|
||||
except ImportError:
|
||||
sys.exit("Fetching URLs needs requests: pip install requests "
|
||||
"(or pass local .html files / a directory instead).")
|
||||
try:
|
||||
r = requests.get(item, timeout=20,
|
||||
headers={"User-Agent": "Mozilla/5.0 (SchemaGen/1.0)"})
|
||||
r.raise_for_status()
|
||||
yield item, r.text
|
||||
except Exception as exc: # noqa: BLE001 — best-effort fetch
|
||||
print(f" ! could not fetch {item}: {exc}", file=sys.stderr)
|
||||
else:
|
||||
p = Path(item)
|
||||
if p.is_dir():
|
||||
for hp in sorted(p.rglob("*.html")):
|
||||
hp = hp.resolve()
|
||||
yield hp.as_uri(), hp.read_text(encoding="utf-8", errors="replace")
|
||||
elif p.is_file():
|
||||
p = p.resolve()
|
||||
yield p.as_uri(), p.read_text(encoding="utf-8", errors="replace")
|
||||
else:
|
||||
print(f" ! not a URL/file/dir, skipped: {item}", file=sys.stderr)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# JSON-LD node -> flat (dotted-path, value) claims
|
||||
# --------------------------------------------------------------------------- #
|
||||
def primary_type(node):
|
||||
t = node.get("@type")
|
||||
if isinstance(t, list):
|
||||
return t[0] if t else ""
|
||||
return t or ""
|
||||
|
||||
|
||||
def top_level_nodes(parsed):
|
||||
"""Return the ENTITY nodes only — @graph members, array items, or a single object.
|
||||
|
||||
Deliberately NOT recursive: nested objects (PostalAddress, GeoCoordinates, …) belong
|
||||
to their parent and are captured by flatten() as dotted paths. Recursing here would
|
||||
wrongly promote a nested address into its own entity.
|
||||
"""
|
||||
if isinstance(parsed, dict) and "@graph" in parsed:
|
||||
graph = parsed["@graph"]
|
||||
return [n for n in graph if isinstance(n, dict) and "@type" in n]
|
||||
if isinstance(parsed, list):
|
||||
return [n for n in parsed if isinstance(n, dict) and "@type" in n]
|
||||
if isinstance(parsed, dict) and "@type" in parsed:
|
||||
return [parsed]
|
||||
return []
|
||||
|
||||
|
||||
def flatten(node, prefix=""):
|
||||
"""Yield (property_path, value) pairs matching the template {{dotted.path}} slots.
|
||||
|
||||
- scalars -> ("name", "X")
|
||||
- scalar arrays -> ("sameAs", "a|b|c") (pipe-joined; builder splits on '|')
|
||||
- nested objects -> recurse with dotted prefix ("address.streetAddress", ...)
|
||||
- @type inside a nested object is structural (templates hard-code it) -> skipped
|
||||
- a bare {"@id": "..."} reference -> ("parentOrganization.@id", "...")
|
||||
"""
|
||||
for key, val in node.items():
|
||||
if key == "@type":
|
||||
continue
|
||||
path = f"{prefix}{key}"
|
||||
if isinstance(val, (str, int, float, bool)):
|
||||
yield path, str(val)
|
||||
elif isinstance(val, list):
|
||||
scalars = [str(v) for v in val if isinstance(v, (str, int, float, bool))]
|
||||
if scalars:
|
||||
yield path, "|".join(scalars)
|
||||
for v in val: # objects inside arrays -> recurse (best-effort)
|
||||
if isinstance(v, dict):
|
||||
yield from flatten(v, prefix=f"{path}.")
|
||||
elif isinstance(val, dict):
|
||||
yield from flatten(val, prefix=f"{path}.")
|
||||
|
||||
|
||||
def slugify(text, maxlen=40):
|
||||
s = re.sub(r"^https?://", "", text or "")
|
||||
s = re.sub(r"[^A-Za-z0-9]+", "-", s).strip("-").lower()
|
||||
return s[:maxlen] or "node"
|
||||
|
||||
|
||||
def entity_id_for(node, url, idx):
|
||||
"""Readable, groupable id like `hotel:westin-hotel` from an @id or URL."""
|
||||
etype = primary_type(node)
|
||||
prefix = TYPE_PREFIX.get(etype, "node")
|
||||
nid = node.get("@id")
|
||||
if nid:
|
||||
pr = urlparse(str(nid))
|
||||
tail = [s for s in pr.path.split("/") if s]
|
||||
parts = (tail[-1:] if tail else []) + ([pr.fragment] if pr.fragment else [])
|
||||
base = "-".join(parts) or pr.netloc or str(nid)
|
||||
else:
|
||||
base = url or f"n{idx}"
|
||||
return f"{prefix}:{slugify(base)}"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Build claims rows from one page
|
||||
# --------------------------------------------------------------------------- #
|
||||
def claims_from_page(url, html, default_lang, rows, seen_props):
|
||||
"""Append claim rows for one page. seen_props tracks (entity_id, property)
|
||||
already taken from authoritative JSON-LD, so meta only fills genuine gaps."""
|
||||
page_lang = ""
|
||||
found_jsonld = False
|
||||
|
||||
# --- 1) existing JSON-LD -> CONFIRMED (authority 1) ---
|
||||
for block in JSONLD_RE.findall(html):
|
||||
try:
|
||||
parsed = json.loads(block.strip())
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
for i, node in enumerate(top_level_nodes(parsed)):
|
||||
etype = primary_type(node)
|
||||
if not etype:
|
||||
continue
|
||||
found_jsonld = True
|
||||
eid = entity_id_for(node, url, i)
|
||||
node_lang = node.get("inLanguage") if isinstance(node.get("inLanguage"), str) else ""
|
||||
lang = node_lang or default_lang
|
||||
for prop, value in flatten(node):
|
||||
rows.append(_row(eid, etype, prop, value, lang, url,
|
||||
"S-SITE", 1, "high", "CONFIRMED",
|
||||
"extracted from existing JSON-LD"))
|
||||
seen_props.add((eid, prop))
|
||||
|
||||
# --- 2) meta / OpenGraph -> PENDING (inferred, needs confirmation) ---
|
||||
mp = MetaParser()
|
||||
try:
|
||||
mp.feed(html)
|
||||
except Exception: # noqa: BLE001 — tolerate malformed HTML
|
||||
pass
|
||||
page_lang = mp.lang or default_lang
|
||||
og_type = mp.meta.get("og:type", "").lower()
|
||||
etype = OG_TYPE_MAP.get(og_type, "WebPage")
|
||||
eid = f"{TYPE_PREFIX.get(etype, 'page')}:{slugify(mp.canonical or url)}"
|
||||
|
||||
inferred = {
|
||||
"name": mp.meta.get("og:title") or mp.title,
|
||||
"url": mp.meta.get("og:url") or mp.canonical or (url if url.startswith("http") else ""),
|
||||
"description": mp.meta.get("og:description") or mp.meta.get("description"),
|
||||
"image": mp.meta.get("og:image"),
|
||||
"inLanguage": page_lang,
|
||||
}
|
||||
# Only emit meta claims when this page contributed NO JSON-LD (else JSON-LD wins).
|
||||
if not found_jsonld:
|
||||
for prop, value in inferred.items():
|
||||
if value and (eid, prop) not in seen_props:
|
||||
rows.append(_row(eid, etype, prop, value, page_lang, url,
|
||||
"S-SITE-META", 2, "med", "PENDING",
|
||||
"inferred from <title>/OpenGraph — confirm before shipping"))
|
||||
|
||||
|
||||
def _row(eid, etype, prop, value, lang, url, src, authority, conf, status, note):
|
||||
return OrderedDict([
|
||||
("entity_id", eid), ("entity_type", etype), ("property", prop),
|
||||
("value", value), ("lang", lang or ""), ("url", url if url.startswith("http") else ""),
|
||||
("source_ids", src), ("authority", authority), ("confidence", conf),
|
||||
("conflict", ""), ("status", status), ("note", note),
|
||||
])
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Main
|
||||
# --------------------------------------------------------------------------- #
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(
|
||||
description="Scenario-1 adapter: existing website -> claims register.")
|
||||
ap.add_argument("inputs", nargs="+", help="URLs, .html files, or a directory")
|
||||
ap.add_argument("--out", default="site_claims", help="output directory")
|
||||
ap.add_argument("--default-lang", default="", help="fallback language code (e.g. ko)")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
rows, seen = [], set()
|
||||
pages = 0
|
||||
for url, html in gather_pages(args.inputs):
|
||||
pages += 1
|
||||
before = len(rows)
|
||||
claims_from_page(url, html, args.default_lang, rows, seen)
|
||||
print(f" · {url} → {len(rows) - before} claims")
|
||||
|
||||
if not rows:
|
||||
print("No claims extracted (no JSON-LD or usable meta on the given pages).",
|
||||
file=sys.stderr)
|
||||
return 1
|
||||
|
||||
outdir = Path(args.out)
|
||||
outdir.mkdir(parents=True, exist_ok=True)
|
||||
reg = outdir / "claims_register.csv"
|
||||
with open(reg, "w", encoding="utf-8-sig", newline="") as f:
|
||||
w = csv.DictWriter(f, fieldnames=CLAIM_FIELDS)
|
||||
w.writeheader()
|
||||
w.writerows(rows)
|
||||
|
||||
confirmed = sum(1 for r in rows if r["status"] == "CONFIRMED")
|
||||
pending = sum(1 for r in rows if r["status"] == "PENDING")
|
||||
entities = sorted({r["entity_id"] for r in rows})
|
||||
rep = [
|
||||
"# Site Extraction Report", "",
|
||||
f"- Pages read: **{pages}**",
|
||||
f"- Claims extracted: **{len(rows)}** "
|
||||
f"(CONFIRMED from JSON-LD: {confirmed} · PENDING from meta: {pending})",
|
||||
f"- Entities seeded: **{len(entities)}**", "",
|
||||
"## Entities", "",
|
||||
]
|
||||
rep += [f"- `{e}`" for e in entities]
|
||||
rep += [
|
||||
"", "## Review before building",
|
||||
"1. **CONFIRMED** rows came from the site's own JSON-LD — spot-check accuracy.",
|
||||
"2. **PENDING** rows were inferred from `<title>`/OpenGraph and will NOT ship until "
|
||||
"you set `status=CONFIRMED` (and clear any `conflict`).",
|
||||
"3. Add anything the pages didn't expose (telephone, address, geo, sameAs).",
|
||||
"", "## Next step",
|
||||
"```bash",
|
||||
f"python build_schema_drafts.py {reg} --out drafts_out",
|
||||
"python ../16-seo-schema-validator/scripts/validate_schema.py "
|
||||
"drafts_out/schema_drafts_dataset.csv --out qa_out",
|
||||
"```",
|
||||
]
|
||||
(outdir / "extraction_report.md").write_text("\n".join(rep), encoding="utf-8")
|
||||
|
||||
print(f"\nWrote {len(rows)} claims ({confirmed} CONFIRMED, {pending} PENDING) "
|
||||
f"for {len(entities)} entities → {reg}")
|
||||
print(f" {outdir / 'extraction_report.md'}")
|
||||
print("Review the register (confirm PENDING rows), then run build_schema_drafts.py.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
87
custom-skills/17-seo-schema-generator/scripts/make_sample.py
Normal file
87
custom-skills/17-seo-schema-generator/scripts/make_sample.py
Normal file
@@ -0,0 +1,87 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
make_sample.py — generate fixtures/sample_claims.csv
|
||||
|
||||
A small, realistic claims register for the Shilla context. It exercises:
|
||||
- 5 entity types: Organization, WebSite, Hotel, Person, JobPosting, VideoObject
|
||||
- dotted nested paths (address.*, geo.*) and an array (sameAs[])
|
||||
- @id cross-references between entities (Hotel.parentOrganization -> org:shilla)
|
||||
- the EXCLUSION gate: one PENDING claim, one CONFLICT claim, one EMPTY value
|
||||
Run, then: python scripts/build_schema_drafts.py fixtures/sample_claims.csv
|
||||
"""
|
||||
import csv, os
|
||||
|
||||
ROWS = [
|
||||
# entity_id, entity_type, property, value, lang, url, source_ids, authority, confidence, conflict, status, note
|
||||
# ---- Organization (DART + official + Wikidata) ----
|
||||
("org:shilla", "Organization", "@id", "https://www.shillahotels.com/#org", "", "", "", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "name", "The Shilla Hotels & Resorts", "", "", "S-OFF|S-WIKI", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "legalName", "주식회사 호텔신라", "", "", "S-DART", "1", "high", "", "CONFIRMED", "DART 법인명"),
|
||||
("org:shilla", "Organization", "url", "https://www.shillahotels.com/", "", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "foundingDate", "1973-05-09", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "sameAs", "https://www.wikidata.org/wiki/Q494845|https://en.wikipedia.org/wiki/The_Shilla", "", "", "S-WIKI|S-WD", "2", "high", "", "CONFIRMED", "array via pipe"),
|
||||
("org:shilla", "Organization", "address.streetAddress", "동호로 249", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "address.addressLocality", "중구", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "address.addressRegion", "서울", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("org:shilla", "Organization", "address.addressCountry", "KR", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
# ---- WebSite (per-language) ----
|
||||
("site:ko", "WebSite", "@id", "https://www.shillahotels.com/ko#website", "ko", "https://www.shillahotels.com/ko/", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("site:ko", "WebSite", "name", "신라호텔", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("site:ko", "WebSite", "url", "https://www.shillahotels.com/ko/", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("site:ko", "WebSite", "inLanguage", "ko", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("site:ko", "WebSite", "publisher.@id", "https://www.shillahotels.com/#org", "ko", "", "", "1", "high", "", "CONFIRMED", "ref to org"),
|
||||
# ---- Hotel (property; @id ref back to org) ----
|
||||
("hotel:theshilla-seoul", "Hotel", "@id", "https://www.shillahotels.com/ko/theshilla/seoul#hotel", "ko", "https://www.shillahotels.com/ko/theshilla/seoul/index.do", "S-OFF|S-BROCH", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "name", "The Shilla Seoul", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "telephone", "+82-2-2233-3131", "ko", "", "S-OFF|S-GBP", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "priceRange", "$$$$", "ko", "", "S-OFF", "2", "med", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "brand.name", "The Shilla", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "parentOrganization.@id", "https://www.shillahotels.com/#org", "ko", "", "", "1", "high", "", "CONFIRMED", "entity graph link"),
|
||||
("hotel:theshilla-seoul", "Hotel", "address.streetAddress", "동호로 249", "ko", "", "S-OFF|S-GBP", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "address.addressLocality", "서울", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "address.addressCountry", "KR", "ko", "", "S-OFF", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "geo.latitude", "37.5564", "ko", "", "S-GBP", "1", "high", "", "CONFIRMED", ""),
|
||||
("hotel:theshilla-seoul", "Hotel", "geo.longitude", "127.0058", "ko", "", "S-GBP", "1", "high", "", "CONFIRMED", ""),
|
||||
# ---- Person (executive bio) ----
|
||||
("person:ceo", "Person", "@id", "https://www.shillahotels.com/#ceo", "", "", "S-DART|S-NEWS", "1", "high", "", "CONFIRMED", ""),
|
||||
("person:ceo", "Person", "name", "이부진", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("person:ceo", "Person", "jobTitle", "대표이사 사장", "", "", "S-DART", "1", "high", "", "CONFIRMED", ""),
|
||||
("person:ceo", "Person", "worksFor.@id", "https://www.shillahotels.com/#org", "", "", "", "1", "high", "", "CONFIRMED", ""),
|
||||
# ---- JobPosting (recruitment site) ----
|
||||
("job:fo-manager", "JobPosting", "title", "프런트오피스 매니저", "ko", "https://recruit.shilla.net/job/1234", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "description", "더 신라 서울 프런트오피스 운영 총괄 및 VIP 응대.", "ko", "", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "datePosted", "2026-05-01", "ko", "", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "employmentType", "FULL_TIME", "ko", "", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "hiringOrganization.@id", "https://www.shillahotels.com/#org", "ko", "", "", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "jobLocation.addressLocality", "서울", "ko", "", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
("job:fo-manager", "JobPosting", "jobLocation.addressCountry", "KR", "ko", "", "S-RECRUIT", "1", "high", "", "CONFIRMED", ""),
|
||||
# ---- VideoObject (official YouTube) ----
|
||||
("video:brand-film", "VideoObject", "name", "The Shilla — Authentic Indulgence", "", "https://www.youtube.com/watch?v=XXXX", "S-YT", "1", "high", "", "CONFIRMED", ""),
|
||||
("video:brand-film", "VideoObject", "description", "더 신라 브랜드 필름.", "", "", "S-YT", "1", "high", "", "CONFIRMED", ""),
|
||||
("video:brand-film", "VideoObject", "thumbnailUrl", "https://i.ytimg.com/vi/XXXX/maxresdefault.jpg", "", "", "S-YT", "1", "high", "", "CONFIRMED", ""),
|
||||
("video:brand-film", "VideoObject", "uploadDate", "2025-11-20", "", "", "S-YT", "1", "high", "", "CONFIRMED", ""),
|
||||
("video:brand-film", "VideoObject", "duration", "PT1M45S", "", "", "S-YT", "1", "high", "", "CONFIRMED", ""),
|
||||
("video:brand-film", "VideoObject", "publisher.@id", "https://www.shillahotels.com/#org", "", "", "", "1", "high", "", "CONFIRMED", ""),
|
||||
# ---- EXCLUSION GATE demonstrations (these must NOT appear in drafts) ----
|
||||
("hotel:theshilla-seoul", "Hotel", "image", "https://example.com/seoul.jpg", "ko", "", "S-OFF", "3", "low", "", "PENDING", "이미지 최종본 미확정"),
|
||||
("org:shilla", "Organization", "telephone", "+82-2-2233-3131", "", "", "S-OFF", "2", "med", "Y", "CONFIRMED", "대표번호 vs IR번호 출처 충돌"),
|
||||
("person:ceo", "Person", "image", "", "", "", "", "", "", "", "CONFIRMED", "값 공란 -> 제외"),
|
||||
]
|
||||
|
||||
HEADERS = ["entity_id", "entity_type", "property", "value", "lang", "url",
|
||||
"source_ids", "authority", "confidence", "conflict", "status", "note"]
|
||||
|
||||
|
||||
def main():
|
||||
here = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
out = os.path.join(here, "fixtures", "sample_claims.csv")
|
||||
os.makedirs(os.path.dirname(out), exist_ok=True)
|
||||
with open(out, "w", encoding="utf-8-sig", newline="") as f:
|
||||
w = csv.writer(f)
|
||||
w.writerow(HEADERS)
|
||||
w.writerows(ROWS)
|
||||
print("Wrote", out, f"({len(ROWS)} claim rows)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
# build_schema_drafts.py and extract_site_claims.py run on the Python standard
|
||||
# library alone for CSV input and local-HTML extraction (the offline default).
|
||||
#
|
||||
# Optional extras, installed only when you need them:
|
||||
openpyxl>=3.1 # read .xlsx claims registers
|
||||
requests>=2.31 # fetch live URLs in extract_site_claims.py (Mode 1 over the network)
|
||||
@@ -0,0 +1,136 @@
|
||||
{
|
||||
"_meta": {
|
||||
"purpose": "JSON-LD draft templates for source-to-schema authoring (pre-launch).",
|
||||
"placeholder_syntax": "{{property.path}} — dotted paths map to claims-register 'property' column. Lines whose value stays unfilled are dropped (never shipped as placeholder).",
|
||||
"aligned_with": "16-seo-schema-validator/scripts/schema_rules.json (required props match Google rich-result requirements)"
|
||||
},
|
||||
"templates": {
|
||||
"Organization": {
|
||||
"_source_hint": "DART, official site footer/about, sustainability report, Wikidata, newsroom",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Organization",
|
||||
"@id": "{{@id}}",
|
||||
"name": "{{name}}",
|
||||
"legalName": "{{legalName}}",
|
||||
"url": "{{url}}",
|
||||
"logo": "{{logo}}",
|
||||
"sameAs": "{{sameAs[]}}",
|
||||
"foundingDate": "{{foundingDate}}",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{address.streetAddress}}",
|
||||
"addressLocality": "{{address.addressLocality}}",
|
||||
"addressRegion": "{{address.addressRegion}}",
|
||||
"postalCode": "{{address.postalCode}}",
|
||||
"addressCountry": "{{address.addressCountry}}"
|
||||
},
|
||||
"contactPoint": {
|
||||
"@type": "ContactPoint",
|
||||
"telephone": "{{contactPoint.telephone}}",
|
||||
"contactType": "{{contactPoint.contactType}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
"WebSite": {
|
||||
"_source_hint": "official homepage; one per language site",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "WebSite",
|
||||
"@id": "{{@id}}",
|
||||
"name": "{{name}}",
|
||||
"url": "{{url}}",
|
||||
"inLanguage": "{{inLanguage}}",
|
||||
"publisher": { "@id": "{{publisher.@id}}" }
|
||||
}
|
||||
},
|
||||
"Hotel": {
|
||||
"_source_hint": "property pages, brochure PDF, GBP, booking data; one per property per language",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Hotel",
|
||||
"@id": "{{@id}}",
|
||||
"name": "{{name}}",
|
||||
"url": "{{url}}",
|
||||
"telephone": "{{telephone}}",
|
||||
"priceRange": "{{priceRange}}",
|
||||
"image": "{{image[]}}",
|
||||
"brand": { "@type": "Brand", "name": "{{brand.name}}" },
|
||||
"parentOrganization": { "@id": "{{parentOrganization.@id}}" },
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"streetAddress": "{{address.streetAddress}}",
|
||||
"addressLocality": "{{address.addressLocality}}",
|
||||
"addressRegion": "{{address.addressRegion}}",
|
||||
"postalCode": "{{address.postalCode}}",
|
||||
"addressCountry": "{{address.addressCountry}}"
|
||||
},
|
||||
"geo": {
|
||||
"@type": "GeoCoordinates",
|
||||
"latitude": "{{geo.latitude}}",
|
||||
"longitude": "{{geo.longitude}}"
|
||||
}
|
||||
}
|
||||
},
|
||||
"Person": {
|
||||
"_source_hint": "executive bios, people-info sites, Wikipedia, press kit; one per person",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "Person",
|
||||
"@id": "{{@id}}",
|
||||
"name": "{{name}}",
|
||||
"jobTitle": "{{jobTitle}}",
|
||||
"worksFor": { "@id": "{{worksFor.@id}}" },
|
||||
"url": "{{url}}",
|
||||
"image": "{{image}}",
|
||||
"sameAs": "{{sameAs[]}}"
|
||||
}
|
||||
},
|
||||
"JobPosting": {
|
||||
"_source_hint": "recruitment sites (채용공고); one per open role",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "JobPosting",
|
||||
"title": "{{title}}",
|
||||
"description": "{{description}}",
|
||||
"datePosted": "{{datePosted}}",
|
||||
"validThrough": "{{validThrough}}",
|
||||
"employmentType": "{{employmentType}}",
|
||||
"hiringOrganization": { "@id": "{{hiringOrganization.@id}}" },
|
||||
"jobLocation": {
|
||||
"@type": "Place",
|
||||
"address": {
|
||||
"@type": "PostalAddress",
|
||||
"addressLocality": "{{jobLocation.addressLocality}}",
|
||||
"addressCountry": "{{jobLocation.addressCountry}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"VideoObject": {
|
||||
"_source_hint": "official YouTube channel; one per featured video",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "VideoObject",
|
||||
"name": "{{name}}",
|
||||
"description": "{{description}}",
|
||||
"thumbnailUrl": "{{thumbnailUrl[]}}",
|
||||
"uploadDate": "{{uploadDate}}",
|
||||
"duration": "{{duration}}",
|
||||
"contentUrl": "{{contentUrl}}",
|
||||
"embedUrl": "{{embedUrl}}",
|
||||
"publisher": { "@id": "{{publisher.@id}}" }
|
||||
}
|
||||
},
|
||||
"FAQPage": {
|
||||
"_source_hint": "press kit FAQ, newsroom, distilled common questions; one per FAQ page",
|
||||
"tpl": {
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"url": "{{url}}",
|
||||
"inLanguage": "{{inLanguage}}",
|
||||
"mainEntity": "{{mainEntity[]}}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
entity_id,entity_type,property,value,lang,url,source_ids,authority,confidence,conflict,status,note
|
||||
org:example,Organization,@id,https://www.example.com/#org,,,S-OFF,1,high,,CONFIRMED,
|
||||
org:example,Organization,name,Example Corp,,,S-OFF|S-DART,1,high,,CONFIRMED,
|
||||
org:example,Organization,url,https://www.example.com/,,,S-OFF,1,high,,CONFIRMED,
|
||||
org:example,Organization,address.addressLocality,Seoul,,,S-DART,1,high,,CONFIRMED,
|
||||
org:example,Organization,address.addressCountry,KR,,,S-DART,1,high,,CONFIRMED,
|
||||
org:example,Organization,sameAs,https://www.wikidata.org/wiki/Q000|https://en.wikipedia.org/wiki/Example,,,S-WD|S-WIKI,2,high,,CONFIRMED,array via pipe
|
||||
org:example,Organization,foundingDate,1998-01-01,,,S-DART,1,high,Y,PENDING,두 출처 연도 충돌 -> 해소 필요
|
||||
|
@@ -0,0 +1,40 @@
|
||||
# Schema 초안 리뷰 가이드 (7단계)
|
||||
|
||||
> 원칙: **사람은 원본 JSON을 직접 보지 않는다.** 기계가 잡을 수 있는 결함은 검증기 게이트(8단계)가
|
||||
> 먼저 0건으로 만들고, 사람은 "기계가 못 잡는 것"만 본다. 그래야 "오류 과다" 문제가 재발하지 않는다.
|
||||
|
||||
## 검토 순서
|
||||
1. **먼저 검증기 통과** → `zero P0` 확보 (P0가 남은 초안은 검토 대상 아님)
|
||||
2. 아래 항목은 사람만 판단 가능 → 검토
|
||||
3. 고객 검토는 P0=0인 깨끗한 초안 + 결함 리포트로만 진행
|
||||
|
||||
## 사람이 검토할 항목 (기계가 못 잡는 것)
|
||||
|
||||
### A. 사실 정확성 (출처 대조)
|
||||
- [ ] `name` / `legalName` 이 공식 표기와 정확히 일치하는가 (공백·영문병기 포함)
|
||||
- [ ] 주소·전화가 **현재 유효한** 값인가 (출처가 오래되지 않았는가)
|
||||
- [ ] `foundingDate` 등 날짜가 공시값과 일치하는가
|
||||
- [ ] 인물의 `jobTitle` 이 **현직** 기준인가
|
||||
|
||||
### B. 엔티티 정합
|
||||
- [ ] `sameAs` 가 **정확히 그 엔티티**를 가리키는가 (동명 오정합 없는가)
|
||||
- [ ] `@id` 참조가 의도한 엔티티로 연결되는가 (Hotel→올바른 Organization)
|
||||
- [ ] 브랜드 티어(`brand.name`)가 프로퍼티와 일치하는가 (The Shilla/Monogram/Stay)
|
||||
|
||||
### C. 언어·번역
|
||||
- [ ] 언어별 초안의 `inLanguage` 와 실제 값 언어가 일치하는가
|
||||
- [ ] 번역값이 공식 다국어 표기와 일치하는가 (임의 번역 아님)
|
||||
|
||||
### D. 범위 적절성
|
||||
- [ ] 스키마를 붙이면 안 되는 페이지(mypage/login/booking)에 초안이 없는가
|
||||
- [ ] 누락된 핵심 엔티티가 없는가 (엔티티-타입 맵 대조)
|
||||
|
||||
## 검토 결과 처리
|
||||
- 수정 필요 → 클레임 레지스터에서 값 수정 후 **빌더 재실행**(JSON 직접 수정 금지: 원천은 항상 클레임)
|
||||
- 충돌 발견 → `conflict=Y`, `status=PENDING` → 출처 권위로 해소
|
||||
- 검토 완료 → 저작자·검수자 서명, P1은 `decision-log`에 기록
|
||||
|
||||
## 서명
|
||||
- 저작(빌드): ______ / 일자: 2026-__-__
|
||||
- 검수(사실확인): ______ / 일자: 2026-__-__
|
||||
- 게이트(검증기 PASS) 확인: ______ / 일자: 2026-__-__
|
||||
@@ -0,0 +1,11 @@
|
||||
source_id,source_type,title_or_name,url_or_filepath,retrieved_date,authority,language,entities_covered,note
|
||||
S-DART,corporate_disclosure,DART 사업보고서,https://dart.fss.or.kr/...,2026-05-__,1,ko,org:shilla,법인명/설립일/주소/대표자
|
||||
S-OFF,official_site,공식 홈페이지 About/푸터,https://www.shillahotels.com/,2026-05-__,1,ko,org:shilla|hotel:*|site:ko,공식 표기/연락처/URL
|
||||
S-SUSTAIN,sustainability_report,지속가능경영보고서 2025,/path/to/esg.pdf,2026-05-__,2,ko,org:shilla,서사/정책
|
||||
S-WD,wikidata,Wikidata 항목,https://www.wikidata.org/wiki/Q______,2026-05-__,2,en,org:shilla,Q-ID/sameAs
|
||||
S-WIKI,wikipedia,위키백과,https://en.wikipedia.org/wiki/______,2026-05-__,2,en,org:shilla,sameAs/국제표기
|
||||
S-RECRUIT,recruitment,채용 사이트 공고,https://recruit._____,2026-05-__,1,ko,job:*,JobPosting 원천
|
||||
S-YT,youtube,공식 YouTube 채널,https://www.youtube.com/@______,2026-05-__,1,ko,video:*,VideoObject 원천
|
||||
S-GBP,google_business_profile,Google Business Profile,,2026-05-__,1,ko,hotel:*,NAP/geo
|
||||
S-BROCH,brochure_pdf,프로퍼티 브로셔,/path/to/brochure.pdf,2026-05-__,2,ko,hotel:*,시설 스펙
|
||||
S-NEWS,media_article,주요 미디어 기사,https://_____,2026-05-__,3,ko,person:ceo,교차검증
|
||||
|
152
custom-skills/95-ourdigital-presales-seo/DESIGN.md
Normal file
152
custom-skills/95-ourdigital-presales-seo/DESIGN.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Design — `ourdigital-presales-seo` skill
|
||||
|
||||
- **Status**: Approved design (2026-05-27). Ready for implementation planning.
|
||||
- **Author**: OurDigital (andrew.yim@ourdigital.org)
|
||||
- **Origin**: Standardizes the Sono Hotels & Resorts pre-sales diagnostic (`~/Workspaces/shr-workspace/audits/2026-05-27-presales/`) into a reusable skill, adding estimate generation and a sales slide deck.
|
||||
|
||||
## 1. Purpose & scope
|
||||
|
||||
A single OurDigital Claude Code skill that runs a **pre-sales SEO + entity diagnostic** for any prospect domain and produces, as **confirmed step-by-step stages**:
|
||||
1. Technical / on-page scan
|
||||
2. Knowledge Graph / entity analysis (Korean market, Naver-aware)
|
||||
3. Consolidated opportunity brief
|
||||
4. **Rate-card-based cost estimate (견적)**
|
||||
5. **Editable PPTX sales-briefing deck** + short client PDF
|
||||
|
||||
**Public-data only** by default (no client GSC/GA4/GTM access assumed) — suited to prospecting where access isn't yet granted.
|
||||
|
||||
### Non-goals (YAGNI)
|
||||
- Not a full post-contract audit (that stays with `seo-comprehensive-audit`).
|
||||
- No fixed 3-tier package output (user chose findings→rate-card line items only).
|
||||
- No automated sending/emailing of deliverables.
|
||||
- No fully-autonomous mode — execution is step-by-step gated.
|
||||
|
||||
## 2. Invocation & inputs (Stage 0)
|
||||
|
||||
Invoked as `/ourdigital-presales-seo` (optionally with a domain arg). Stage 0 gathers:
|
||||
|
||||
| Input | Req | Default |
|
||||
|---|:---:|---|
|
||||
| `domain` | ✅ | — |
|
||||
| `brand_name` + `aliases[]` (for KG; e.g. 앤/& /EN variants) | ✅ | derived from site `<title>`/og |
|
||||
| `sub_brands[]`, `properties[]` (entity targets) | — | auto-extracted from crawl URL patterns |
|
||||
| `competitors[]` | — | from `references/competitor_sets.md` by vertical |
|
||||
| `market` / `language` | — | `South Korea` / `ko` (Naver-aware) |
|
||||
| `output_dir` | — | account workspace if exists, else `seo-workspace/prospecting/<date>-<prospect>/` (see §7) |
|
||||
| `vertical` | — | inferred (hotel/resort default rubric) |
|
||||
|
||||
Stage 0 also runs a **preflight tool check** (Firecrawl, DataForSEO, `GOOGLE_KG_API_KEY`, headless Chrome, `python-pptx`) and reports any missing capability with its fallback.
|
||||
|
||||
## 3. Pipeline (step-by-step; each stage presents results and WAITS for user confirmation)
|
||||
|
||||
| Stage | Actions | Primary tools | Output |
|
||||
|---|---|---|---|
|
||||
| 0 Scope | inputs, folders, preflight | — | `scope.md`, folders |
|
||||
| 1 Discovery | robots.txt, sitemap status, `firecrawl_map` inventory, scale estimate, URL hygiene (`/test`, params, dup `/sb`↔`/brand_loc`) | Firecrawl, WebFetch | `data/urls.json`, scan §1 |
|
||||
| 2 Technical/on-page | JSON-LD extraction, meta/title/H1 duplication, hreflang completeness, CWV (Lighthouse) | Firecrawl scrape, DataForSEO Lighthouse | `01_technical-onpage-scan.md`, `data/cwv_lighthouse.json` |
|
||||
| 3 KG/entity | KG API (ko) over master/parent/legacy/membership/sub-brand/property/competitor sets; live SERP panel verification | `kg_query.py` (Google KG API), DataForSEO SERP | `02_knowledge-graph-entity.md`, `data/kg_*.json`, `data/serp_panels_findings.md` |
|
||||
| 4 Brief | synthesize, severity ranking, competitive gap table | — | `03_presales-opportunity-brief.md` |
|
||||
| 5 Estimate | findings→rate card mapping → ranged 견적 | `estimate.py` + `rate_card.yaml` | `05_estimate_ko.md`, `05_estimate.xlsx` — **review gate** |
|
||||
| 6 Deliverables | short client PDF + branded PPTX deck | `render_pdf.sh` (Chrome), `build_deck.py` (python-pptx) | `client-brief.pdf`, `sales-deck.pptx` — **review before send** |
|
||||
| 7 Archive | push consolidated report (03 brief + estimate summary) to the OurDigital SEO Audit DB — **standard final stage** | `notion_writer.py` | Notion row in SEO Audit Log |
|
||||
|
||||
Each stage appends to the shared **`findings.json`** data contract (§5), the integration seam between analysis and the estimate/deck generators.
|
||||
|
||||
**Archive target (standard):** every run archives to the OurDigital SEO Audit database
|
||||
`2c8581e5-8a1e-8035-880b-e38cefc2f3ef`
|
||||
(https://www.notion.so/dintelligence/2c8581e58a1e8035880be38cefc2f3ef). Row title `<프로스펙트> SEO 사전진단 (Pre-sales) — <YYYY-MM-DD>`; set `Target URL`, `Audit Date`, `Account Code`. This is the system of record for prospect + client audits alike.
|
||||
|
||||
## 4. Component breakdown (units & interfaces)
|
||||
|
||||
- **`SKILL.md`** — orchestration: stage definitions, per-stage gating, Korean-first output rules, tool fallbacks, sandbox-disable notes for KG/Chrome/Notion network calls.
|
||||
- **`scripts/kg_query.py`** — IN: entity list (group,label,query) + lang + key (env). OUT: `kg_raw.json`, `kg_flat.json`, console summary (score/type/lodging-flag/own-entity). Generalized from the SHR script (entities parameterized, not hardcoded).
|
||||
- **`scripts/estimate.py`** — IN: `findings.json` + `rate_card.yaml` + rules. OUT: `05_estimate_ko.md` + `05_estimate.xlsx` (항목·상세·수량·단가 range·금액; one-time + monthly subtotals; `OD-YYYY-NNN`; disclaimer) **and `data/estimate.json`** (selected line items + totals, consumed by `build_deck.py`).
|
||||
- **`scripts/build_deck.py`** — IN: `findings.json` + `estimate.json` + `deck_theme`. OUT: `sales-deck.pptx` (9 slides, §6). python-pptx.
|
||||
- **`scripts/render_pdf.sh`** — IN: client-brief HTML. OUT: PDF via headless Chrome (Korean system fonts).
|
||||
- **`references/rate_card.yaml`** — single source of OurDigital service rates (see §5.1).
|
||||
- **`references/findings_to_service.md`** — finding-class → severity → service-line rubric (§5.2).
|
||||
- **`references/competitor_sets.md`** — default KR competitor benchmarks by vertical (hotel/resort seeded: 롯데/신라/조선/한화/켄싱턴).
|
||||
- **`templates/`** — `01/02/03` md, `client_brief.html`, `estimate_OD.md`, `deck_theme.py`.
|
||||
|
||||
## 5. Estimate logic
|
||||
|
||||
### 5.1 `rate_card.yaml` (from `ourdigital-backoffice`)
|
||||
```yaml
|
||||
quote_prefix: OD # OD-YYYY-NNN
|
||||
currency: KRW
|
||||
services:
|
||||
technical_audit: {label: "Technical Audit / 기술 SEO 진단", unit: one_time, min: 3000000, max: 5000000}
|
||||
technical_remediation: {label: "기술 개선 실행", unit: project, min: 3000000, max: 8000000}
|
||||
onpage_entity: {label: "On-Page / Entity Optimization", unit: monthly, min: 1500000, max: 3000000}
|
||||
schema_build: {label: "구조화 데이터 구축(1회)", unit: one_time, min: 2000000, max: 4000000}
|
||||
local_seo: {label: "Local SEO", unit: monthly, min: 1000000, max: 2000000}
|
||||
gtm_setup: {label: "GTM Setup", unit: project, min: 2000000, max: 4000000}
|
||||
ga4_impl: {label: "GA4 Implementation", unit: project, min: 1500000, max: 3000000}
|
||||
dashboard: {label: "Dashboard Development", unit: project, min: 3000000, max: 6000000}
|
||||
```
|
||||
*(Values mirror the backoffice rate card; treated as estimate ranges. Skill reads this file — no hardcoded prices.)*
|
||||
|
||||
### 5.2 `findings_to_service.md` rubric (finding class → service line)
|
||||
| Finding class (from `findings.json`) | Service line(s) | Scope driver |
|
||||
|---|---|---|
|
||||
| broken sitemap / low crawl-coverage / SPA rendering / CWV poor | `technical_audit` + `technical_remediation` | site size, # templates |
|
||||
| missing/weak schema, entity gaps, sub-brand/property entities, Wikipedia/sameAs | `schema_build` (one-time) + `onpage_entity` (retainer) | # sub-brands + # properties |
|
||||
| property local packs / GBP-URL mismatch | `local_seo` | # properties |
|
||||
| no GSC/GA4, measurement gaps | `ga4_impl` and/or `dashboard`, `gtm_setup` | — |
|
||||
| duplicate meta / title i18n / hreflang / content confusion | `onpage_entity` | # templates |
|
||||
|
||||
`estimate.py` selects line items per detected findings, scales `qty` by drivers (e.g., property count → local-SEO months/scope), sums one-time vs monthly, and renders the 견적. Always includes the disclaimer: *ranges; finalized after a precise diagnostic with GSC/GA4 access.*
|
||||
|
||||
## 6. PPTX deck spec (`build_deck.py`, 9 slides)
|
||||
1. Title — prospect + "검색 가시성 사전 진단" + date + OurDigital
|
||||
2. 한눈에 보기 — asset strength vs search-visibility gap
|
||||
3. Finding 1 — 크롤/색인 (sitemap, discoverable-URL count)
|
||||
4. Finding 2 — Core Web Vitals
|
||||
5. Finding 3 — 엔티티/브랜드 인식 (entity type, name split, legacy contamination)
|
||||
6. Finding 4 — 서브브랜드/프로퍼티 엔티티 + 경쟁 벤치마크 table
|
||||
7. 개선 로드맵 — Phase 0 (긴급 기술) / 1 (엔티티) / 2 (콘텐츠·로컬)
|
||||
8. 예상 견적 — rate-card line items + ranges + disclaimer
|
||||
9. 다음 단계 / CTA — 30분 미팅 · 정밀 진단 · 파일럿
|
||||
|
||||
Branding from `ourdigital-brand-guide` (colors/fonts/logo); fallback theme = navy `#11243d` / accent `#1b6fb3` (the SHR brief styling). Slides are content-populated from `findings.json` + `estimate.json`, leaving text editable.
|
||||
|
||||
## 7. Output routing
|
||||
Default: if `~/Workspaces/<slug>-workspace/` exists → `…/audits/<YYYY-MM-DD>-presales/`; else `~/Workspaces/seo-workspace/prospecting/<YYYY-MM-DD>-<prospect>/` (per global routing rule). Overridable at Stage 0. `data/` holds raw artifacts; audit md + deck/PDF at top level.
|
||||
|
||||
## 8. Dependencies & documented fallbacks
|
||||
| Capability | Tool | Fallback / gotcha |
|
||||
|---|---|---|
|
||||
| URL inventory | Firecrawl `map` | OurSEO `crawl_website` **caps ~60 pages** regardless of `max_pages`; broken sitemap limits discovery — report the discoverable count as a finding |
|
||||
| Page signals | Firecrawl `scrape` (json) | re-scrape if cache returns empty |
|
||||
| SERP panels / CWV | DataForSEO `serp_organic_live_advanced`, `on_page_lighthouse` | — |
|
||||
| Entity DB | Google KG Search API | needs `GOOGLE_KG_API_KEY` / `GOOGLE_API_KEY`; sandbox-disable for network |
|
||||
| Client PDF | headless Chrome `--print-to-pdf` | needs Korean system font (AppleSDGothicNeo present on macOS); sandbox-disable |
|
||||
| Deck | `python-pptx` | install if missing |
|
||||
| Notion archive | `notion_writer.py` → DB `2c8581e5-8a1e-8035-880b-e38cefc2f3ef` | use `--properties` with `Target URL`/`Audit Date`/`Account Code` only; `Audit ID` is a read-only **formula** (do not set); `Site`/`Found Date` from old docs are wrong property names |
|
||||
|
||||
## 9. Data contract — `findings.json` (analysis ↔ generators seam)
|
||||
```jsonc
|
||||
{
|
||||
"prospect": {"name": "", "domain": "", "aliases": [], "vertical": ""},
|
||||
"discovery": {"sitemap_status": 500, "robots_sitemap_declared": false,
|
||||
"discoverable_urls": 96, "estimated_pages": "thousands",
|
||||
"url_hygiene": ["test_page_exposed", "dup_path_scheme", "param_urls"]},
|
||||
"technical": {"cwv": {"lcp_ms":0,"cls":0,"ttfb_ms":0,"perf":0},
|
||||
"schema": {"org": "bare|complete|none", "hotel_on_property": true},
|
||||
"meta_dupe": true, "title_i18n_mismatch": true, "hreflang": "incomplete"},
|
||||
"entity": {"panel": "company|hotel|none", "name_split": true, "legacy_contamination": true,
|
||||
"subbrands_with_entity": 0, "properties_with_entity": 0,
|
||||
"competitor_benchmark": [{"name":"","score":0,"type":"","wikipedia":false}]},
|
||||
"findings": [{"id":"", "class":"", "severity":"critical|high|medium", "evidence":"", "recommended_services":[]}]
|
||||
}
|
||||
```
|
||||
Stages 1–4 populate it; Stages 5–6 consume it. This is the key isolation boundary: generators never re-crawl.
|
||||
|
||||
## 10. Validation
|
||||
- Dry-run on the SHR data (already collected) → estimate + deck must reproduce sensible output.
|
||||
- `kg_query.py` unit check: known entity (롯데호텔) returns LodgingBusiness + high score.
|
||||
- Deck opens in PowerPoint/Keynote; Korean renders; placeholders editable.
|
||||
- Estimate totals = sum of selected line items; disclaimer present.
|
||||
|
||||
## 11. Future (out of scope now)
|
||||
3-tier package view; auto Naver SERP module; multi-language decks; CRM hand-off.
|
||||
89
custom-skills/95-ourdigital-presales-seo/SKILL.md
Normal file
89
custom-skills/95-ourdigital-presales-seo/SKILL.md
Normal file
@@ -0,0 +1,89 @@
|
||||
---
|
||||
name: ourdigital-presales-seo
|
||||
description: Standardized pre-sales SEO + Knowledge Graph diagnostic for a prospect domain that produces a technical/on-page scan, KG/entity analysis, consolidated brief, a rate-card-based cost estimate (견적), and an editable PPTX sales deck. Use for pre-sales prospecting, sales briefing prep, "pre-sales SEO audit", "프리세일즈 진단", "견적 + 제안 슬라이드", or when preparing for a prospect meeting. Public-data only (no client GSC/GA4 access required). Korean-first output.
|
||||
---
|
||||
|
||||
# OurDigital Pre-sales SEO Workflow
|
||||
|
||||
Runs the full pre-sales diagnostic for **any prospect domain** as **step-by-step
|
||||
confirmed stages**, ending in a cost estimate and a sales-briefing deck. Public-data
|
||||
only — designed for prospects where you don't yet have GSC/GA4/GTM access.
|
||||
|
||||
Origin: standardizes the Sono Hotels & Resorts pre-sales diagnostic.
|
||||
|
||||
## Execution model — STEP BY STEP
|
||||
|
||||
Run one stage at a time. After each stage, **present the result and WAIT for the
|
||||
user's go-ahead** before the next. Create a TodoWrite/Task item per stage. Stages 5
|
||||
(estimate) and 6 (deliverables) are explicit review gates — never send without sign-off.
|
||||
|
||||
## Stage 0 — Scope & preflight
|
||||
Collect (ask only for what you can't infer):
|
||||
- `domain` (required); `brand_name` + `aliases` (앤/& / EN variants — infer from `<title>`/og first)
|
||||
- `sub_brands`, `properties` (auto-extract from crawl URL patterns in Stage 1 if not given)
|
||||
- `competitors` (else pick a set from `references/competitor_sets.md` by vertical)
|
||||
- `market`/`language` (default South Korea / ko), `vertical`
|
||||
- `output_dir` — default routing: if `~/Workspaces/<slug>-workspace/` exists →
|
||||
`…/audits/<YYYY-MM-DD>-presales/`, else `~/Workspaces/seo-workspace/prospecting/<YYYY-MM-DD>-<prospect>/`
|
||||
|
||||
Preflight tool check (report missing + fallback): Firecrawl, DataForSEO, `GOOGLE_KG_API_KEY`,
|
||||
headless Chrome, python-pptx. Create `data/` subfolder. Initialize `findings.json` (see `findings.schema.json`).
|
||||
|
||||
## Stage 1 — Discovery & crawl
|
||||
- `WebFetch` robots.txt + `/sitemap.xml` + `/sitemap_index.xml` (note status; 500/404 = finding).
|
||||
- `firecrawl_map` (limit high, includeSubdomains) → URL inventory; record `discoverable_urls`,
|
||||
derive URL architecture, `estimated_pages`, and hygiene flags (`/test`, params, duplicate path schemes).
|
||||
- Populate `findings.json.discovery`. → write scan §1.
|
||||
|
||||
## Stage 2 — Technical / on-page
|
||||
- `firecrawl_scrape` (json) homepage + 1-2 property pages: JSON-LD `@type`s, Organization completeness
|
||||
(sameAs/alternateName/address), meta/title/H1, hreflang set, robots/googlebot meta.
|
||||
- `mcp__dfs-mcp__on_page_lighthouse` homepage (+ a property) → CWV.
|
||||
- Populate `findings.json.technical`. → write `01_technical-onpage-scan.md`.
|
||||
- **Honesty rule:** never claim "noindexed" if the page ranks — flag conflicting directives as "verify".
|
||||
|
||||
## Stage 3 — Knowledge Graph / entity
|
||||
- `python scripts/kg_query.py --brand … --aliases … --parent … --legacy … --subbrands … --properties … --competitors … --out-dir <out>/data`
|
||||
(needs `GOOGLE_KG_API_KEY`; run with sandbox disabled — read-only API GET).
|
||||
- Live SERP panel verification: `mcp__dfs-mcp__serp_organic_live_advanced` (ko, South Korea) for brand,
|
||||
a sub-brand, a property, and one competitor → record actual panel type/title, local packs, reseller leakage.
|
||||
- Populate `findings.json.entity` (panel, name_split, legacy_contamination, sub-brand/property entity counts,
|
||||
competitor_benchmark[], wikipedia). → write `02_knowledge-graph-entity.md` + `data/serp_panels_findings.md`.
|
||||
|
||||
## Stage 4 — Consolidated brief
|
||||
- Synthesize, rank by severity, build the competitive-gap table. Populate `findings.json.findings[]`.
|
||||
→ write `03_presales-opportunity-brief.md`.
|
||||
|
||||
## Stage 5 — Estimate (견적) — REVIEW GATE
|
||||
- `python scripts/estimate.py --findings <out>/data/findings.json --rate-card references/rate_card.yaml --out-dir <out> --seq <N>`
|
||||
- Produces `05_estimate_ko.md`, `05_estimate.xlsx`, `data/estimate.json`. Present the ranged 견적; get sign-off.
|
||||
- Rules in `references/findings_to_service.md`; rates in `references/rate_card.yaml` (edit both together).
|
||||
|
||||
## Stage 6 — Deliverables — REVIEW GATE before send
|
||||
- **Client PDF**: author the short brief HTML from `templates/client_brief.html` (fill the content; keep the CSS),
|
||||
then `bash scripts/render_pdf.sh <brief>.html` → PDF. Verify Korean renders (Read the PDF).
|
||||
- **Sales deck**: `python scripts/build_deck.py --findings <out>/data/findings.json --estimate <out>/data/estimate.json --out <out>/sales-deck.pptx`
|
||||
- Sanitize the client-facing pieces: no internal pricing strategy beyond the 견적; tasteful competitor benchmark only.
|
||||
|
||||
## Stage 7 — Archive (standard)
|
||||
Push the consolidated report to the OurDigital SEO Audit DB:
|
||||
```
|
||||
python <notion-writer path>/notion_writer.py \
|
||||
--database 2c8581e5-8a1e-8035-880b-e38cefc2f3ef \
|
||||
--title "<프로스펙트> SEO 사전진단 (Pre-sales) — <YYYY-MM-DD>" \
|
||||
--properties '{"Target URL": "<domain>", "Audit Date": "<YYYY-MM-DD>", "Account Code": "<CODE>"}' \
|
||||
--file <out>/03_presales-opportunity-brief.md
|
||||
```
|
||||
Property names are exact: `Target URL`, `Audit Date`, `Account Code`. **Do NOT set `Audit ID`**
|
||||
(read-only formula). notion-writer path: `~/Project/our-claude-skills/custom-skills/32-notion-writer/code/scripts/notion_writer.py`.
|
||||
|
||||
## Tool gotchas (learned)
|
||||
- OurSEO `crawl_website` **caps ~60 pages** regardless of `max_pages` — use Firecrawl `map` for inventory; report the discoverable count as a finding.
|
||||
- Firecrawl `scrape` cache may return empty — re-scrape.
|
||||
- KG API + headless Chrome + Notion push need network → run those Bash calls with the sandbox disabled.
|
||||
- Korean PDF: headless Chrome uses system fonts (AppleSDGothicNeo on macOS). On Windows set deck font to Malgun Gothic in `build_deck.py`.
|
||||
|
||||
## Conventions
|
||||
- Korean-first for all client-facing output; keep technical terms in English (SEO, CWV, Schema, hreflang).
|
||||
- File names: `01_/02_/03_/05_` prefixes as above; raw artifacts in `data/`.
|
||||
- Outputs go to the workspace (Stage 0 routing), **never** into this skills repo.
|
||||
@@ -0,0 +1,99 @@
|
||||
{
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"title": "ourdigital-presales-seo findings contract",
|
||||
"description": "Shared artifact populated by analysis stages 1-4 and consumed by estimate.py (5) and build_deck.py (6). Generators must never re-crawl; they read this file.",
|
||||
"type": "object",
|
||||
"required": ["prospect", "discovery", "technical", "entity", "findings"],
|
||||
"properties": {
|
||||
"prospect": {
|
||||
"type": "object",
|
||||
"required": ["name", "domain"],
|
||||
"properties": {
|
||||
"name": {"type": "string"},
|
||||
"domain": {"type": "string"},
|
||||
"aliases": {"type": "array", "items": {"type": "string"}},
|
||||
"vertical": {"type": "string"},
|
||||
"audit_date": {"type": "string", "description": "YYYY-MM-DD"},
|
||||
"account_code": {"type": "string"}
|
||||
}
|
||||
},
|
||||
"discovery": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"sitemap_status": {"type": "integer"},
|
||||
"robots_sitemap_declared": {"type": "boolean"},
|
||||
"discoverable_urls": {"type": "integer"},
|
||||
"estimated_pages": {"type": "string"},
|
||||
"url_hygiene": {"type": "array", "items": {"type": "string"}}
|
||||
}
|
||||
},
|
||||
"technical": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"cwv": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"lcp_ms": {"type": "number"}, "cls": {"type": "number"},
|
||||
"ttfb_ms": {"type": "number"}, "perf": {"type": "number"}
|
||||
}
|
||||
},
|
||||
"schema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"org": {"type": "string", "enum": ["bare", "complete", "none"]},
|
||||
"hotel_on_property": {"type": "boolean"}
|
||||
}
|
||||
},
|
||||
"meta_dupe": {"type": "boolean"},
|
||||
"title_i18n_mismatch": {"type": "boolean"},
|
||||
"hreflang": {"type": "string", "enum": ["complete", "incomplete", "none"]}
|
||||
}
|
||||
},
|
||||
"entity": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"panel": {"type": "string", "enum": ["company", "hotel", "none"]},
|
||||
"name_split": {"type": "boolean"},
|
||||
"legacy_contamination": {"type": "boolean"},
|
||||
"subbrands_with_entity": {"type": "integer"},
|
||||
"subbrands_total": {"type": "integer"},
|
||||
"properties_with_entity": {"type": "integer"},
|
||||
"properties_total": {"type": "integer"},
|
||||
"wikipedia": {"type": "boolean"},
|
||||
"competitor_benchmark": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string"}, "score": {"type": "number"},
|
||||
"type": {"type": "string"}, "wikipedia": {"type": "boolean"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"measurement": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"gsc_access": {"type": "boolean"},
|
||||
"ga4_access": {"type": "boolean"},
|
||||
"tag_gaps": {"type": "boolean"}
|
||||
}
|
||||
},
|
||||
"findings": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"required": ["id", "class", "severity"],
|
||||
"properties": {
|
||||
"id": {"type": "string"},
|
||||
"class": {"type": "string", "description": "crawlability|cwv|schema_entity|subbrand_entity|local|measurement|onpage"},
|
||||
"severity": {"type": "string", "enum": ["critical", "high", "medium", "low"]},
|
||||
"title_ko": {"type": "string"},
|
||||
"evidence": {"type": "string"},
|
||||
"recommended_services": {"type": "array", "items": {"type": "string"}}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
# Default competitor benchmark sets (Korean market)
|
||||
|
||||
Used by Stage 3 (KG/entity) when the user doesn't supply competitors. Pick the set
|
||||
matching the prospect's vertical; pass as `--competitors` to `kg_query.py`.
|
||||
|
||||
## hotel_resort (호텔·리조트)
|
||||
- 롯데호텔 (Lotte Hotel) — strong KG, LodgingBusiness type, Korean Wikipedia
|
||||
- 신라호텔 / 호텔신라 (Shilla)
|
||||
- 조선호텔앤리조트 (Josun)
|
||||
- 한화리조트 / 한화호텔앤드리조트 (Hanwha)
|
||||
- 켄싱턴리조트 (Kensington)
|
||||
|
||||
## city_hotel (시티 호텔)
|
||||
- 롯데호텔, 신라호텔, 조선호텔, 글래드호텔, 나인트리
|
||||
|
||||
## condo_membership (콘도·회원권)
|
||||
- 한화리조트, 대명(소노), 한솔오크밸리, 금호리조트
|
||||
|
||||
## benchmark_signals
|
||||
For each competitor record in `findings.json.entity.competitor_benchmark`:
|
||||
`{name, score (KG result_score), type (@type), wikipedia (bool)}`.
|
||||
The benchmark table contrasts the prospect's entity strength/type/Wikipedia
|
||||
presence against these — the core competitive-gap visual in the deck.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Findings → service rubric
|
||||
|
||||
How `estimate.py` maps detected findings (from `findings.json`) to `rate_card.yaml`
|
||||
service lines. The script reads the structured signals below; this file is the
|
||||
human-readable source of truth for the rules.
|
||||
|
||||
| Trigger (signal in findings.json) | Service line(s) | Scope driver |
|
||||
|---|---|---|
|
||||
| `discovery.sitemap_status != 200` OR `discovery.robots_sitemap_declared == false` OR `discovery.discoverable_urls` low vs `estimated_pages` | `technical_audit` + `technical_remediation` | site size / # templates |
|
||||
| `technical.cwv.perf < 0.5` OR `cls > 0.1` OR `lcp_ms > 2500` OR `ttfb_ms > 600` | `technical_audit` (if not already) + `technical_remediation` | # templates |
|
||||
| `technical.schema.org == "bare"/"none"` OR `entity.panel != "hotel"` OR `entity.name_split` OR `entity.legacy_contamination` OR `entity.subbrands_with_entity == 0` | `schema_build` (one-time) + `onpage_entity` (retainer) | # sub-brands + # properties |
|
||||
| `entity.properties_with_entity == 0` OR `url_hygiene` contains GBP/local mismatch | `local_seo` | # properties |
|
||||
| `findings[].class` includes measurement gap / no GSC-GA4 | `ga4_impl` and/or `dashboard` (+ `gtm_setup` if tag gaps) | — |
|
||||
| `technical.meta_dupe` OR `technical.title_i18n_mismatch` OR `technical.hreflang == "incomplete"` | `onpage_entity` | # templates |
|
||||
|
||||
**Severity → priority** (for the brief/deck ordering, not pricing):
|
||||
- `critical`: crawl/index blocking, CWV failing, entity mistyped
|
||||
- `high`: entity/sub-brand gaps, duplicate URLs, meta dupes
|
||||
- `medium`: hreflang, H1, hygiene
|
||||
|
||||
**Quantity rules**
|
||||
- `monthly` line items use `rate_card.defaults.retainer_months` (default 6).
|
||||
- `local_seo` scope note scales with property count (`entity` / discovery counts).
|
||||
- One-time items counted once even if triggered by multiple findings.
|
||||
|
||||
Edit this file and `rate_card.yaml` together when rates or rules change.
|
||||
@@ -0,0 +1,63 @@
|
||||
# OurDigital service rate card — single source for estimate.py
|
||||
# Mirrors the ourdigital-backoffice quote ranges. Values are KRW, treated as
|
||||
# pre-sales estimate RANGES (finalize after a precise diagnostic with access).
|
||||
quote_prefix: OD # quote number format: OD-YYYY-NNN
|
||||
currency: KRW
|
||||
|
||||
services:
|
||||
technical_audit:
|
||||
label_ko: "Technical Audit / 기술 SEO 진단"
|
||||
unit: one_time # one_time | monthly | project
|
||||
min: 3000000
|
||||
max: 5000000
|
||||
technical_remediation:
|
||||
label_ko: "기술 개선 실행 (sitemap/CWV/SSR)"
|
||||
unit: project
|
||||
min: 3000000
|
||||
max: 8000000
|
||||
onpage_entity:
|
||||
label_ko: "On-Page / Entity Optimization (월 운영)"
|
||||
unit: monthly
|
||||
min: 1500000
|
||||
max: 3000000
|
||||
schema_build:
|
||||
label_ko: "구조화 데이터(Schema) 구축 (1회)"
|
||||
unit: one_time
|
||||
min: 2000000
|
||||
max: 4000000
|
||||
local_seo:
|
||||
label_ko: "Local SEO (프로퍼티 로컬 최적화)"
|
||||
unit: monthly
|
||||
min: 1000000
|
||||
max: 2000000
|
||||
gtm_setup:
|
||||
label_ko: "GTM Setup / 태그 관리 구축"
|
||||
unit: project
|
||||
min: 2000000
|
||||
max: 4000000
|
||||
ga4_impl:
|
||||
label_ko: "GA4 Implementation / 분석 환경 구축"
|
||||
unit: project
|
||||
min: 1500000
|
||||
max: 3000000
|
||||
dashboard:
|
||||
label_ko: "Dashboard Development / 대시보드 개발"
|
||||
unit: project
|
||||
min: 3000000
|
||||
max: 6000000
|
||||
|
||||
defaults:
|
||||
retainer_months: 6 # default contract length for monthly line items
|
||||
disclaimer_ko: "본 견적은 공개 데이터 기반 사전 추정 범위이며, Search Console/Analytics 권한 확보 후 정밀 진단을 통해 확정됩니다."
|
||||
|
||||
# Scope scaling — monthly line items scale (sub-linearly) by portfolio size.
|
||||
# driver: a count under findings.entity (properties_total | subbrands_total).
|
||||
# bands: ordered [max_count, multiplier]; first band whose max_count >= count wins.
|
||||
# A 25-property chain costs more to run than a single hotel, but not 25x.
|
||||
scaling:
|
||||
local_seo:
|
||||
driver: properties_total
|
||||
bands: [[1, 1.0], [5, 1.6], [15, 2.8], [30, 4.5], [999999, 6.5]]
|
||||
onpage_entity:
|
||||
driver: subbrands_total
|
||||
bands: [[1, 1.0], [3, 1.6], [6, 2.2], [999999, 3.2]]
|
||||
315
custom-skills/95-ourdigital-presales-seo/scripts/build_deck.py
Executable file
315
custom-skills/95-ourdigital-presales-seo/scripts/build_deck.py
Executable file
@@ -0,0 +1,315 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build an editable OurDigital-branded sales-briefing deck (PPTX) for pre-sales SEO.
|
||||
|
||||
Part of the ourdigital-presales-seo skill (Stage 6). Reads findings.json (+ optional
|
||||
estimate.json) and writes a 9-slide .pptx. Content is populated from data; text stays
|
||||
editable in PowerPoint/Keynote.
|
||||
|
||||
Usage:
|
||||
python build_deck.py --findings findings.json --estimate data/estimate.json \
|
||||
--out sales-deck.pptx
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from pptx import Presentation
|
||||
from pptx.dml.color import RGBColor
|
||||
from pptx.enum.shapes import MSO_SHAPE
|
||||
from pptx.enum.text import PP_ALIGN
|
||||
from pptx.util import Inches, Pt
|
||||
|
||||
NAVY = RGBColor(0x11, 0x24, 0x3D)
|
||||
ACCENT = RGBColor(0x1B, 0x6F, 0xB3)
|
||||
LIGHT = RGBColor(0xF3, 0xF7, 0xFB)
|
||||
GREY = RGBColor(0x6B, 0x77, 0x87)
|
||||
WHITE = RGBColor(0xFF, 0xFF, 0xFF)
|
||||
RED = RGBColor(0xC0, 0x39, 0x2B)
|
||||
FONT = "Apple SD Gothic Neo" # macOS Korean; edit in deck if on Windows (Malgun Gothic)
|
||||
|
||||
EMU_W, EMU_H = Inches(13.333), Inches(7.5)
|
||||
|
||||
|
||||
def _style(run, size, color=NAVY, bold=False):
|
||||
run.font.size = Pt(size)
|
||||
run.font.bold = bold
|
||||
run.font.color.rgb = color
|
||||
run.font.name = FONT
|
||||
|
||||
|
||||
def textbox(slide, l, t, w, h, lines, align=PP_ALIGN.LEFT):
|
||||
"""lines: list of (text, size, color, bold) or list of such lists (paragraphs)."""
|
||||
tb = slide.shapes.add_textbox(Inches(l), Inches(t), Inches(w), Inches(h))
|
||||
tf = tb.text_frame
|
||||
tf.word_wrap = True
|
||||
if lines and not isinstance(lines[0], list):
|
||||
lines = [lines]
|
||||
for idx, para in enumerate(lines):
|
||||
p = tf.paragraphs[0] if idx == 0 else tf.add_paragraph()
|
||||
p.alignment = align
|
||||
p.space_after = Pt(4)
|
||||
# para is a list of runs: each (text, size, color, bold)
|
||||
if para and isinstance(para[0], str):
|
||||
para = [tuple(para)]
|
||||
for (text, size, color, bold) in para:
|
||||
r = p.add_run()
|
||||
r.text = text
|
||||
_style(r, size, color, bold)
|
||||
return tb
|
||||
|
||||
|
||||
def bar(slide, l, t, w, h, color=ACCENT):
|
||||
shp = slide.shapes.add_shape(MSO_SHAPE.RECTANGLE, Inches(l), Inches(t), Inches(w), Inches(h))
|
||||
shp.fill.solid()
|
||||
shp.fill.fore_color.rgb = color
|
||||
shp.line.fill.background()
|
||||
shp.shadow.inherit = False
|
||||
return shp
|
||||
|
||||
|
||||
def blank(prs):
|
||||
return prs.slides.add_slide(prs.slide_layouts[6])
|
||||
|
||||
|
||||
def fill_bg(slide, color):
|
||||
slide.background.fill.solid()
|
||||
slide.background.fill.fore_color.rgb = color
|
||||
|
||||
|
||||
def header(slide, kicker, title):
|
||||
bar(slide, 0.6, 0.55, 0.12, 0.9)
|
||||
textbox(slide, 0.85, 0.5, 11.8, 1.1, [
|
||||
[(kicker, 11, ACCENT, True)],
|
||||
[(title, 24, NAVY, True)],
|
||||
])
|
||||
|
||||
|
||||
def card(slide, l, t, w, h, fill=LIGHT):
|
||||
shp = slide.shapes.add_shape(MSO_SHAPE.ROUNDED_RECTANGLE, Inches(l), Inches(t), Inches(w), Inches(h))
|
||||
shp.fill.solid()
|
||||
shp.fill.fore_color.rgb = fill
|
||||
shp.line.color.rgb = RGBColor(0xDC, 0xE7, 0xF1)
|
||||
shp.line.width = Pt(0.75)
|
||||
shp.shadow.inherit = False
|
||||
return shp
|
||||
|
||||
|
||||
def finding_slide(prs, kicker, title, headline, bullets, metric=None):
|
||||
s = blank(prs)
|
||||
header(s, kicker, title)
|
||||
if metric:
|
||||
card(s, 0.85, 1.95, 3.5, 4.6, NAVY)
|
||||
textbox(s, 1.0, 2.5, 3.2, 3.5, [
|
||||
[(metric[0], 40, WHITE, True)],
|
||||
[(metric[1], 13, RGBColor(0xCF, 0xDD, 0xEE), False)],
|
||||
])
|
||||
bx = 4.7
|
||||
else:
|
||||
bx = 0.85
|
||||
rows = [[(headline, 15, NAVY, True)]]
|
||||
for b in bullets:
|
||||
rows.append([("• ", 12, ACCENT, True), (b, 12, RGBColor(0x33, 0x3A, 0x45), False)])
|
||||
textbox(s, bx, 2.1, 13.0 - bx - 0.6, 4.4, rows)
|
||||
return s
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="Build pre-sales SEO sales deck (PPTX)")
|
||||
ap.add_argument("--findings", required=True)
|
||||
ap.add_argument("--estimate", default=None)
|
||||
ap.add_argument("--out", default="sales-deck.pptx")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.findings, encoding="utf-8") as fh:
|
||||
F = json.load(fh)
|
||||
EST = None
|
||||
if args.estimate:
|
||||
try:
|
||||
with open(args.estimate, encoding="utf-8") as fh:
|
||||
EST = json.load(fh)
|
||||
except FileNotFoundError:
|
||||
EST = None
|
||||
|
||||
p = F.get("prospect", {})
|
||||
name = p.get("name", "(프로스펙트)")
|
||||
date = p.get("audit_date", "")
|
||||
d = F.get("discovery", {})
|
||||
t = F.get("technical", {})
|
||||
e = F.get("entity", {})
|
||||
|
||||
prs = Presentation()
|
||||
prs.slide_width, prs.slide_height = EMU_W, EMU_H
|
||||
|
||||
# 1) Title
|
||||
s = blank(prs)
|
||||
fill_bg(s, NAVY)
|
||||
bar(s, 0.9, 2.7, 1.6, 0.14, ACCENT)
|
||||
textbox(s, 0.9, 2.9, 11.5, 3.0, [
|
||||
[("SEO PRE-SALES BRIEF", 13, RGBColor(0x7F, 0xA8, 0xCF), True)],
|
||||
[(f"{name}", 40, WHITE, True)],
|
||||
[("검색 가시성 사전 진단", 26, RGBColor(0xCF, 0xDD, 0xEE), True)],
|
||||
[(f"OurDigital · {date} · 공개 데이터 기준 사전 스냅샷", 12, RGBColor(0x9F, 0xB4, 0xCC), False)],
|
||||
])
|
||||
|
||||
# 2) Overview
|
||||
s = blank(prs)
|
||||
header(s, "OVERVIEW", "한눈에 보기")
|
||||
card(s, 0.85, 1.95, 11.6, 1.7)
|
||||
textbox(s, 1.1, 2.15, 11.1, 1.4, [
|
||||
[(f"{name}는 국내 최대급 호텔·리조트 자산을 보유하지만, ", 14, NAVY, False),
|
||||
("검색에서의 디지털 가시성은 그 규모에 미치지 못합니다.", 14, ACCENT, True)],
|
||||
])
|
||||
textbox(s, 0.95, 3.95, 11.6, 2.6, [
|
||||
[("핵심 병목 두 가지", 14, NAVY, True)],
|
||||
[("① 기술 — ", 13, RED, True), ('"보이지 않는 사이트": 색인·CWV 문제로 발견 가능 페이지가 극소수', 13, RGBColor(0x33, 0x3A, 0x45), False)],
|
||||
[("② 엔티티 — ", 13, RED, True), ('"잘못 인식된 브랜드": 회사 타입·표기 분열·레거시 잔존, 서브브랜드 부재', 13, RGBColor(0x33, 0x3A, 0x45), False)],
|
||||
])
|
||||
|
||||
# 3) Finding 1 — crawl/index
|
||||
finding_slide(
|
||||
prs, "FINDING 01", "크롤링 / 색인 — 사이트가 검색엔진에 보이지 않습니다",
|
||||
'사이트맵 오류와 크롤성 한계로 대부분의 페이지가 발견·색인되지 못합니다.',
|
||||
[
|
||||
f"sitemap 상태: HTTP {d.get('sitemap_status', 'N/A')}" + (" (정상)" if d.get('sitemap_status') == 200 else " — 오류/미동작"),
|
||||
"robots.txt 사이트맵 선언: " + ("있음" if d.get("robots_sitemap_declared") else "없음"),
|
||||
f"추정 전체 페이지: {d.get('estimated_pages', 'N/A')}",
|
||||
"위생 이슈: " + (", ".join(d.get("url_hygiene", [])) or "없음"),
|
||||
],
|
||||
metric=(str(d.get("discoverable_urls", "—")), "외부 발견 가능 URL (개)"),
|
||||
)
|
||||
|
||||
# 4) Finding 2 — CWV
|
||||
cwv = t.get("cwv", {})
|
||||
finding_slide(
|
||||
prs, "FINDING 02", "Core Web Vitals — 속도·안정성 취약",
|
||||
"구글 순위 요소이자 예약 전환·모바일 경험에 직접 영향을 줍니다.",
|
||||
[
|
||||
f"LCP {cwv.get('lcp_ms', 0)/1000:.1f}초 (기준 <2.5초)",
|
||||
f"TTFB {cwv.get('ttfb_ms', 0)/1000:.1f}초 (기준 <0.6초)",
|
||||
f"Performance score {cwv.get('perf', 0):.2f} (기준 ≥0.9)",
|
||||
],
|
||||
metric=(f"{cwv.get('cls', 0):.3f}", "CLS (화면 밀림) · 기준 <0.1"),
|
||||
)
|
||||
|
||||
# 5) Finding 3 — entity recognition
|
||||
panel_ko = {"company": '"회사"로만 인식 (호텔 아님)', "hotel": "호텔로 인식", "none": "지식패널 없음"}
|
||||
finding_slide(
|
||||
prs, "FINDING 03", "엔티티 인식 — 구글이 브랜드를 호텔로 보지 않습니다",
|
||||
"호텔 전용 검색 노출에 불리하고 리브랜딩 효과가 검색에 반영되지 못합니다.",
|
||||
[
|
||||
"지식패널 분류: " + panel_ko.get(e.get("panel"), "확인 필요"),
|
||||
"브랜드 표기 분열(앤 vs &): " + ("있음" if e.get("name_split") else "없음"),
|
||||
"레거시(구 브랜드명) 잔존: " + ("있음" if e.get("legacy_contamination") else "없음"),
|
||||
"Korean Wikipedia 등재: " + ("있음" if e.get("wikipedia") else "없음"),
|
||||
],
|
||||
)
|
||||
|
||||
# 6) Finding 4 — subbrand/competitor
|
||||
s = finding_slide(
|
||||
prs, "FINDING 04", "서브브랜드·프로퍼티 엔티티 + 경쟁 벤치마크",
|
||||
"다(多)브랜드 체계가 검색 자산으로 축적되지 못하고 있습니다.",
|
||||
[
|
||||
f"서브브랜드 엔티티: {e.get('subbrands_with_entity', 0)} / {e.get('subbrands_total', 0)}",
|
||||
f"프로퍼티 엔티티: {e.get('properties_with_entity', 0)} / {e.get('properties_total', 0)}",
|
||||
],
|
||||
)
|
||||
bench = e.get("competitor_benchmark", [])
|
||||
if bench:
|
||||
rows = min(len(bench) + 1, 7)
|
||||
tbl = s.shapes.add_table(rows, 4, Inches(7.0), Inches(2.4), Inches(5.5), Inches(0.4 * rows)).table
|
||||
for j, htxt in enumerate(["브랜드", "KG 강도", "타입", "위키"]):
|
||||
c = tbl.cell(0, j)
|
||||
c.text = htxt
|
||||
c.fill.solid()
|
||||
c.fill.fore_color.rgb = NAVY
|
||||
for para in c.text_frame.paragraphs:
|
||||
for r in para.runs:
|
||||
_style(r, 11, WHITE, True)
|
||||
for i, b in enumerate(bench[:rows - 1], 1):
|
||||
vals = [b.get("name", ""), str(int(b.get("score", 0))), b.get("type", ""), "○" if b.get("wikipedia") else "—"]
|
||||
for j, v in enumerate(vals):
|
||||
c = tbl.cell(i, j)
|
||||
c.text = v
|
||||
for para in c.text_frame.paragraphs:
|
||||
for r in para.runs:
|
||||
_style(r, 10, NAVY, False)
|
||||
|
||||
# 7) Roadmap
|
||||
s = blank(prs)
|
||||
header(s, "ROADMAP", "개선 로드맵")
|
||||
phases = [
|
||||
("Phase 0 · 긴급 기술 복구", "사이트맵 복구 · CWV(CLS/LCP/TTFB) · 중복 URL canonical", ACCENT),
|
||||
("Phase 1 · 엔티티 정합", "Organization/Hotel schema · 표기 통일 · 서브브랜드/프로퍼티 엔티티 · Wikipedia", NAVY),
|
||||
("Phase 2 · 콘텐츠·로컬·확장", "프로퍼티 로컬 SEO · 브랜드 체계 콘텐츠 · Naver · AI 검색 가시성", GREY),
|
||||
]
|
||||
for i, (h, body, col) in enumerate(phases):
|
||||
top = 2.1 + i * 1.55
|
||||
card(s, 0.85, top, 11.6, 1.35)
|
||||
bar(s, 0.85, top, 0.14, 1.35, col)
|
||||
textbox(s, 1.15, top + 0.18, 11.0, 1.1, [
|
||||
[(h, 15, col, True)],
|
||||
[(body, 12, RGBColor(0x33, 0x3A, 0x45), False)],
|
||||
])
|
||||
|
||||
# 8) Estimate
|
||||
s = blank(prs)
|
||||
header(s, "ESTIMATE", "예상 견적 (사전 추정 범위)")
|
||||
if EST:
|
||||
items = EST.get("line_items", [])
|
||||
rows = min(len(items) + 1, 9)
|
||||
tbl = s.shapes.add_table(rows, 3, Inches(0.85), Inches(2.0), Inches(11.6), Inches(0.45 * rows)).table
|
||||
tbl.columns[0].width = Inches(6.0)
|
||||
tbl.columns[1].width = Inches(2.0)
|
||||
tbl.columns[2].width = Inches(3.6)
|
||||
for j, htxt in enumerate(["항목", "단위", "금액(범위)"]):
|
||||
c = tbl.cell(0, j)
|
||||
c.text = htxt
|
||||
c.fill.solid()
|
||||
c.fill.fore_color.rgb = NAVY
|
||||
for para in c.text_frame.paragraphs:
|
||||
for r in para.runs:
|
||||
_style(r, 12, WHITE, True)
|
||||
unit_ko = {"one_time": "1회", "project": "프로젝트", "monthly": "월"}
|
||||
for i, it in enumerate(items[:rows - 1], 1):
|
||||
amt = f"{it['amount_min']:,}~{it['amount_max']:,}원"
|
||||
for j, v in enumerate([it["label"], unit_ko.get(it["unit"], it["unit"]), amt]):
|
||||
c = tbl.cell(i, j)
|
||||
c.text = v
|
||||
for para in c.text_frame.paragraphs:
|
||||
for r in para.runs:
|
||||
_style(r, 11, NAVY, False)
|
||||
tot = EST.get("totals", {})
|
||||
textbox(s, 0.85, 2.0 + 0.45 * rows + 0.2, 11.6, 1.2, [
|
||||
[("총계(범위): ", 14, NAVY, True),
|
||||
(f"{tot.get('grand_min', 0):,} ~ {tot.get('grand_max', 0):,}원", 14, RED, True)],
|
||||
[(EST.get("disclaimer", ""), 10, GREY, False)],
|
||||
])
|
||||
else:
|
||||
textbox(s, 0.85, 2.2, 11.6, 1.0, [[("견적 데이터(estimate.json) 미연결 — estimate.py 실행 후 재생성", 13, GREY, False)]])
|
||||
|
||||
# 9) Next steps
|
||||
s = blank(prs)
|
||||
fill_bg(s, NAVY)
|
||||
bar(s, 0.9, 1.0, 1.6, 0.14, ACCENT)
|
||||
textbox(s, 0.9, 1.2, 11.5, 1.2, [
|
||||
[("NEXT STEPS", 13, RGBColor(0x7F, 0xA8, 0xCF), True)],
|
||||
[("다음 단계", 30, WHITE, True)],
|
||||
])
|
||||
steps = [
|
||||
("1. 30분 미팅", "진단 결과 공유 및 우선순위 논의"),
|
||||
("2. 정밀 진단", "Search Console·Analytics 권한 확보 후 색인·트래픽·키워드 정량 분석"),
|
||||
("3. 단기 파일럿", "긴급 기술 복구(사이트맵·CWV)부터 빠른 가시 성과"),
|
||||
]
|
||||
for i, (h, body) in enumerate(steps):
|
||||
top = 2.8 + i * 1.2
|
||||
textbox(s, 1.1, top, 11.0, 1.1, [
|
||||
[(h, 17, WHITE, True)],
|
||||
[(body, 12, RGBColor(0xCF, 0xDD, 0xEE), False)],
|
||||
])
|
||||
textbox(s, 1.1, 6.7, 11.0, 0.5, [[("OurDigital · andrew.yim@ourdigital.org", 11, RGBColor(0x9F, 0xB4, 0xCC), False)]])
|
||||
|
||||
prs.save(args.out)
|
||||
print(f"Wrote {args.out} ({len(prs.slides)} slides)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
232
custom-skills/95-ourdigital-presales-seo/scripts/estimate.py
Executable file
232
custom-skills/95-ourdigital-presales-seo/scripts/estimate.py
Executable file
@@ -0,0 +1,232 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a ranged 견적 (estimate) from findings.json using the OurDigital rate card.
|
||||
|
||||
Part of the ourdigital-presales-seo skill (Stage 5). Maps detected findings to
|
||||
rate-card service lines (see references/findings_to_service.md) and emits:
|
||||
- 05_estimate_ko.md (Korean line-item quote)
|
||||
- data/estimate.json (consumed by build_deck.py)
|
||||
- 05_estimate.xlsx (spreadsheet quote)
|
||||
|
||||
Usage:
|
||||
python estimate.py --findings findings.json --rate-card ../references/rate_card.yaml \
|
||||
--out-dir ./audits/2026-05-27-presales --seq 1
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
|
||||
import yaml
|
||||
from openpyxl import Workbook
|
||||
from openpyxl.styles import Alignment, Font, PatternFill
|
||||
|
||||
|
||||
def won(n):
|
||||
return f"{n:,}원"
|
||||
|
||||
|
||||
def select_services(f):
|
||||
"""Return {service_key: [reasons]} based on findings signals."""
|
||||
chosen = {}
|
||||
d = f.get("discovery", {})
|
||||
t = f.get("technical", {})
|
||||
e = f.get("entity", {})
|
||||
m = f.get("measurement", {})
|
||||
|
||||
def need(key, reason):
|
||||
chosen.setdefault(key, [])
|
||||
if reason not in chosen[key]:
|
||||
chosen[key].append(reason)
|
||||
|
||||
# Crawlability / indexation
|
||||
if d.get("sitemap_status", 200) != 200 or d.get("robots_sitemap_declared", True) is False:
|
||||
need("technical_audit", "sitemap/robots 색인 이슈")
|
||||
need("technical_remediation", "sitemap 복구·크롤성 개선")
|
||||
# Core Web Vitals
|
||||
cwv = t.get("cwv", {})
|
||||
if (cwv.get("perf", 1) < 0.5 or cwv.get("cls", 0) > 0.1
|
||||
or cwv.get("lcp_ms", 0) > 2500 or cwv.get("ttfb_ms", 0) > 600):
|
||||
need("technical_audit", "Core Web Vitals 취약")
|
||||
need("technical_remediation", "CWV(CLS/LCP/TTFB) 개선")
|
||||
# Schema / entity
|
||||
if (t.get("schema", {}).get("org") in ("bare", "none") or e.get("panel") != "hotel"
|
||||
or e.get("name_split") or e.get("legacy_contamination")
|
||||
or (e.get("subbrands_with_entity", 0) == 0 and e.get("subbrands_total", 0) > 0)):
|
||||
need("schema_build", "Organization/Hotel schema·브랜드 표기 정합")
|
||||
need("onpage_entity", "엔티티·서브브랜드 최적화")
|
||||
# Local
|
||||
hygiene = " ".join(d.get("url_hygiene", [])).lower()
|
||||
if ((e.get("properties_with_entity", 0) == 0 and e.get("properties_total", 0) > 0)
|
||||
or "local" in hygiene or "gbp" in hygiene or "dup_path" in hygiene):
|
||||
need("local_seo", "프로퍼티 로컬·GBP 정합")
|
||||
# Measurement
|
||||
if m.get("gsc_access") is False or m.get("ga4_access") is False:
|
||||
need("ga4_impl", "측정 환경(GA4) 구축")
|
||||
need("dashboard", "리포팅 대시보드 구축")
|
||||
if m.get("tag_gaps"):
|
||||
need("gtm_setup", "태그 관리(GTM) 구축")
|
||||
# On-page hygiene
|
||||
if t.get("meta_dupe") or t.get("title_i18n_mismatch") or t.get("hreflang") == "incomplete":
|
||||
need("onpage_entity", "Meta/Title/hreflang 정리")
|
||||
return chosen
|
||||
|
||||
|
||||
DRIVER_LABEL = {"properties_total": "프로퍼티", "subbrands_total": "서브브랜드"}
|
||||
|
||||
|
||||
def scope_multiplier(rate, key, f):
|
||||
"""Sub-linear scope multiplier for a service, driven by portfolio size.
|
||||
|
||||
Returns (multiplier, driver, count). count is floored at 1 (unknown→base).
|
||||
"""
|
||||
rule = rate.get("scaling", {}).get(key)
|
||||
if not rule:
|
||||
return 1.0, None, None
|
||||
driver = rule["driver"]
|
||||
count = max(int(f.get("entity", {}).get(driver, 0) or 0), 1)
|
||||
for max_count, mult in rule["bands"]:
|
||||
if count <= max_count:
|
||||
return float(mult), driver, count
|
||||
return 1.0, driver, count
|
||||
|
||||
|
||||
def build_line_items(chosen, rate, f):
|
||||
months = rate["defaults"]["retainer_months"]
|
||||
order = {"one_time": 0, "project": 1, "monthly": 2}
|
||||
items = []
|
||||
for key, reasons in chosen.items():
|
||||
svc = rate["services"][key]
|
||||
unit = svc["unit"]
|
||||
qty = months if unit == "monthly" else 1
|
||||
mult, driver, count = scope_multiplier(rate, key, f)
|
||||
umin = int(round(svc["min"] * mult))
|
||||
umax = int(round(svc["max"] * mult))
|
||||
reason = "; ".join(reasons)
|
||||
scope_note = None
|
||||
if mult != 1.0:
|
||||
scope_note = f"{DRIVER_LABEL.get(driver, driver)} {count}개 기준 ×{mult:g}"
|
||||
reason = f"{reason} [{scope_note}]"
|
||||
items.append({
|
||||
"key": key, "label": svc["label_ko"], "unit": unit, "qty": qty,
|
||||
"unit_min": umin, "unit_max": umax,
|
||||
"amount_min": umin * qty, "amount_max": umax * qty,
|
||||
"reason": reason,
|
||||
"scope_multiplier": mult, "scope_driver": driver,
|
||||
"scope_count": count, "scope_note": scope_note,
|
||||
})
|
||||
items.sort(key=lambda x: order.get(x["unit"], 9))
|
||||
return items, months
|
||||
|
||||
|
||||
def totals(items):
|
||||
one = [i for i in items if i["unit"] != "monthly"]
|
||||
mon = [i for i in items if i["unit"] == "monthly"]
|
||||
return {
|
||||
"one_time_min": sum(i["amount_min"] for i in one),
|
||||
"one_time_max": sum(i["amount_max"] for i in one),
|
||||
"monthly_min": sum(i["amount_min"] for i in mon),
|
||||
"monthly_max": sum(i["amount_max"] for i in mon),
|
||||
"grand_min": sum(i["amount_min"] for i in items),
|
||||
"grand_max": sum(i["amount_max"] for i in items),
|
||||
}
|
||||
|
||||
|
||||
UNIT_KO = {"one_time": "1회", "project": "프로젝트", "monthly": "월"}
|
||||
|
||||
|
||||
def write_md(path, quote_no, date, prospect, items, tot, months, disclaimer):
|
||||
L = [f"# 견적서 (Pre-sales 추정) — {prospect}",
|
||||
"", f"- **견적번호**: {quote_no}", f"- **작성일**: {date}",
|
||||
f"- **대상**: {prospect}", f"- **공급자**: OurDigital (andrew.yim@ourdigital.org)",
|
||||
"", "## 견적 내역", "",
|
||||
"| 항목 | 근거 | 단위 | 수량 | 단가(범위) | 금액(범위) |",
|
||||
"|---|---|---|---:|---|---|"]
|
||||
for i in items:
|
||||
L.append(f"| {i['label']} | {i['reason']} | {UNIT_KO.get(i['unit'], i['unit'])} | {i['qty']} | "
|
||||
f"{won(i['unit_min'])}~{won(i['unit_max'])} | {won(i['amount_min'])}~{won(i['amount_max'])} |")
|
||||
L += ["", "## 합계 (범위)", "",
|
||||
f"- 일회성/프로젝트: **{won(tot['one_time_min'])} ~ {won(tot['one_time_max'])}**",
|
||||
f"- 월 운영({months}개월 기준): **{won(tot['monthly_min'])} ~ {won(tot['monthly_max'])}**",
|
||||
f"- 총계: **{won(tot['grand_min'])} ~ {won(tot['grand_max'])}**",
|
||||
"", f"> {disclaimer}"]
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(L) + "\n")
|
||||
|
||||
|
||||
def write_xlsx(path, quote_no, date, prospect, items, tot, months, disclaimer):
|
||||
wb = Workbook()
|
||||
ws = wb.active
|
||||
ws.title = "견적"
|
||||
hdr = PatternFill("solid", fgColor="11243D")
|
||||
hf = Font(color="FFFFFF", bold=True)
|
||||
ws.append([f"견적서 (Pre-sales 추정) — {prospect}"])
|
||||
ws.append([f"견적번호 {quote_no}", f"작성일 {date}", "공급자 OurDigital"])
|
||||
ws.append([])
|
||||
cols = ["항목", "근거", "단위", "수량", "단가 min", "단가 max", "금액 min", "금액 max"]
|
||||
ws.append(cols)
|
||||
for c in range(1, len(cols) + 1):
|
||||
cell = ws.cell(row=ws.max_row, column=c)
|
||||
cell.fill = hdr
|
||||
cell.font = hf
|
||||
for i in items:
|
||||
ws.append([i["label"], i["reason"], UNIT_KO.get(i["unit"], i["unit"]), i["qty"],
|
||||
i["unit_min"], i["unit_max"], i["amount_min"], i["amount_max"]])
|
||||
ws.append([])
|
||||
ws.append(["일회성/프로젝트 합계", "", "", "", "", "", tot["one_time_min"], tot["one_time_max"]])
|
||||
ws.append([f"월 운영 합계 ({months}개월)", "", "", "", "", "", tot["monthly_min"], tot["monthly_max"]])
|
||||
ws.append(["총계", "", "", "", "", "", tot["grand_min"], tot["grand_max"]])
|
||||
ws.cell(row=ws.max_row, column=1).font = Font(bold=True)
|
||||
ws.append([])
|
||||
ws.append([disclaimer])
|
||||
widths = [34, 30, 8, 6, 12, 12, 14, 14]
|
||||
for idx, w in enumerate(widths, 1):
|
||||
ws.column_dimensions[chr(64 + idx)].width = w
|
||||
wb.save(path)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="Generate ranged 견적 from findings.json")
|
||||
ap.add_argument("--findings", required=True)
|
||||
ap.add_argument("--rate-card", required=True)
|
||||
ap.add_argument("--out-dir", default=".")
|
||||
ap.add_argument("--seq", type=int, default=1, help="quote sequence number (NNN)")
|
||||
args = ap.parse_args()
|
||||
|
||||
with open(args.findings, encoding="utf-8") as fh:
|
||||
f = json.load(fh)
|
||||
with open(args.rate_card, encoding="utf-8") as fh:
|
||||
rate = yaml.safe_load(fh)
|
||||
|
||||
prospect = f.get("prospect", {}).get("name", "(prospect)")
|
||||
date = f.get("prospect", {}).get("audit_date") or datetime.date.today().isoformat()
|
||||
year = date[:4]
|
||||
quote_no = f"{rate.get('quote_prefix', 'OD')}-{year}-{args.seq:03d}"
|
||||
disclaimer = rate["defaults"]["disclaimer_ko"]
|
||||
|
||||
chosen = select_services(f)
|
||||
items, months = build_line_items(chosen, rate, f)
|
||||
tot = totals(items)
|
||||
|
||||
os.makedirs(args.out_dir, exist_ok=True)
|
||||
data_dir = os.path.join(args.out_dir, "data")
|
||||
os.makedirs(data_dir, exist_ok=True)
|
||||
|
||||
md_path = os.path.join(args.out_dir, "05_estimate_ko.md")
|
||||
xlsx_path = os.path.join(args.out_dir, "05_estimate.xlsx")
|
||||
json_path = os.path.join(data_dir, "estimate.json")
|
||||
|
||||
write_md(md_path, quote_no, date, prospect, items, tot, months, disclaimer)
|
||||
write_xlsx(xlsx_path, quote_no, date, prospect, items, tot, months, disclaimer)
|
||||
with open(json_path, "w", encoding="utf-8") as fh:
|
||||
json.dump({"quote_no": quote_no, "date": date, "prospect": prospect,
|
||||
"line_items": items, "totals": tot, "retainer_months": months,
|
||||
"disclaimer": disclaimer}, fh, ensure_ascii=False, indent=2)
|
||||
|
||||
print(f"견적 {quote_no}: {len(items)} line items | "
|
||||
f"one-time {won(tot['one_time_min'])}~{won(tot['one_time_max'])} | "
|
||||
f"monthly {won(tot['monthly_min'])}~{won(tot['monthly_max'])}/{months}mo")
|
||||
print(f"Wrote: {md_path}\n {xlsx_path}\n {json_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
141
custom-skills/95-ourdigital-presales-seo/scripts/kg_query.py
Executable file
141
custom-skills/95-ourdigital-presales-seo/scripts/kg_query.py
Executable file
@@ -0,0 +1,141 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Query Google Knowledge Graph (Korean) for a prospect's brand ecosystem + competitors.
|
||||
|
||||
Part of the ourdigital-presales-seo skill (Stage 3). Generalized from the
|
||||
Sono Hotels & Resorts pre-sales script: entities are built from CLI args, not
|
||||
hardcoded. Read-only GET to kgsearch.googleapis.com.
|
||||
|
||||
Key resolution: env GOOGLE_KG_API_KEY -> GOOGLE_API_KEY.
|
||||
|
||||
Example:
|
||||
python kg_query.py --brand "소노호텔앤리조트" \
|
||||
--aliases "소노호텔&리조트,SONO Hotels & Resorts" \
|
||||
--parent "소노인터내셔널" --legacy "대명소노그룹,대명리조트" \
|
||||
--subbrands "소노벨,소노캄,소노펠리체,쏠비치,소노문" \
|
||||
--properties "소노벨 비발디파크,소노캄 제주,쏠비치 양양" \
|
||||
--competitors "롯데호텔,신라호텔,조선호텔앤리조트,한화리조트,켄싱턴리조트" \
|
||||
--out-dir ./data
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
API_KEY = os.environ.get("GOOGLE_KG_API_KEY") or os.environ.get("GOOGLE_API_KEY")
|
||||
ENDPOINT = "https://kgsearch.googleapis.com/v1/entities:search"
|
||||
LODGING = {"Hotel", "LodgingBusiness", "Resort", "Place", "TouristAttraction"}
|
||||
|
||||
|
||||
def query_kg(q, lang, limit):
|
||||
params = {"query": q, "key": API_KEY, "languages": lang, "limit": limit, "indent": "true"}
|
||||
url = ENDPOINT + "?" + urllib.parse.urlencode(params)
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "OurDigital-SEO-Audit/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def flatten(label, group, data):
|
||||
rows = []
|
||||
for item in data.get("itemListElement", []):
|
||||
r = item.get("result", {})
|
||||
detail = r.get("detailedDescription", {}) or {}
|
||||
rows.append({
|
||||
"group": group, "input_label": label, "kg_id": r.get("@id"),
|
||||
"name": r.get("name"), "types": r.get("@type"), "description": r.get("description"),
|
||||
"result_score": item.get("resultScore"), "has_detailed_desc": bool(detail.get("articleBody")),
|
||||
"detailed_source": detail.get("url"), "image": (r.get("image") or {}).get("contentUrl"),
|
||||
"url": r.get("url"),
|
||||
})
|
||||
if not rows:
|
||||
rows.append({"group": group, "input_label": label, "kg_id": None, "name": None,
|
||||
"types": None, "description": "NO KG ENTITY FOUND", "result_score": 0,
|
||||
"has_detailed_desc": False, "detailed_source": None, "image": None, "url": None})
|
||||
return rows
|
||||
|
||||
|
||||
def build_entities(args):
|
||||
ents = []
|
||||
|
||||
def add(group, raw):
|
||||
for it in (raw or "").split(","):
|
||||
it = it.strip()
|
||||
if it:
|
||||
ents.append((group, it, it))
|
||||
|
||||
add("01_master", args.brand)
|
||||
add("01_master", args.aliases)
|
||||
add("02_corporate", args.parent)
|
||||
add("03_legacy", args.legacy)
|
||||
add("04_membership", args.membership)
|
||||
add("05_subbrand", args.subbrands)
|
||||
add("06_property", args.properties)
|
||||
add("09_competitor", args.competitors)
|
||||
return ents
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="KG entity examination for pre-sales SEO")
|
||||
ap.add_argument("--brand", required=True, help="Master brand (comma-separated allowed)")
|
||||
ap.add_argument("--aliases", default="", help="Brand name variants (& vs 앤, EN)")
|
||||
ap.add_argument("--parent", default="", help="Parent / corporate entity")
|
||||
ap.add_argument("--legacy", default="", help="Former / legacy names")
|
||||
ap.add_argument("--membership", default="", help="Membership / sales-rep units")
|
||||
ap.add_argument("--subbrands", default="", help="Sub-brands")
|
||||
ap.add_argument("--properties", default="", help="Flagship properties")
|
||||
ap.add_argument("--competitors", default="", help="Competitor benchmarks")
|
||||
ap.add_argument("--lang", default="ko")
|
||||
ap.add_argument("--limit", type=int, default=30)
|
||||
ap.add_argument("--out-dir", default=".")
|
||||
args = ap.parse_args()
|
||||
|
||||
if not API_KEY:
|
||||
sys.exit("ERROR: set GOOGLE_KG_API_KEY or GOOGLE_API_KEY in the environment.")
|
||||
|
||||
ents = build_entities(args)
|
||||
raw, flat = {}, []
|
||||
for group, label, q in ents:
|
||||
try:
|
||||
data = query_kg(q, args.lang, args.limit)
|
||||
raw[label] = data
|
||||
flat.extend(flatten(label, group, data))
|
||||
except Exception as e: # network/quota — record, continue
|
||||
raw[label] = {"error": str(e)}
|
||||
flat.append({"group": group, "input_label": label, "kg_id": None, "name": None,
|
||||
"types": None, "description": f"ERROR: {e}", "result_score": 0,
|
||||
"has_detailed_desc": False, "detailed_source": None, "image": None, "url": None})
|
||||
time.sleep(0.3)
|
||||
|
||||
os.makedirs(args.out_dir, exist_ok=True)
|
||||
with open(os.path.join(args.out_dir, "kg_korean_raw.json"), "w", encoding="utf-8") as f:
|
||||
json.dump(raw, f, ensure_ascii=False, indent=2)
|
||||
with open(os.path.join(args.out_dir, "kg_korean_flat.json"), "w", encoding="utf-8") as f:
|
||||
json.dump(flat, f, ensure_ascii=False, indent=2)
|
||||
|
||||
# Console summary: top match per input label + lodging-type flag
|
||||
by_label = {}
|
||||
for r in flat:
|
||||
e = by_label.setdefault(r["input_label"], {"group": r["group"], "n": 0, "lodging": False, "top": r})
|
||||
e["group"] = r["group"]
|
||||
if r.get("kg_id"):
|
||||
e["n"] += 1
|
||||
if set(r.get("types") or []) & LODGING:
|
||||
e["lodging"] = True
|
||||
if (r.get("result_score") or 0) > (e["top"].get("result_score") or 0):
|
||||
e["top"] = r
|
||||
|
||||
print(f"{'GROUP':<14}{'SCORE':>8} {'#':>3} {'LODG':<5} {'TOP TYPE':<22} {'DESC':<5} INPUT -> TOP KG NAME")
|
||||
print("-" * 120)
|
||||
for lbl, e in sorted(by_label.items(), key=lambda kv: kv[1]["group"]):
|
||||
top = e["top"]
|
||||
ttype = ",".join((top.get("types") or [])[:2]) or "-"
|
||||
print(f"{e['group']:<14}{top.get('result_score') or 0:>8.0f} {e['n']:>3} "
|
||||
f"{('YES' if e['lodging'] else '-'):<5} {ttype[:22]:<22} "
|
||||
f"{('yes' if top.get('has_detailed_desc') else 'no'):<5} {lbl[:40]} -> {top.get('name') or '(NONE)'}")
|
||||
print(f"\nWrote kg_korean_raw.json + kg_korean_flat.json to {args.out_dir}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
18
custom-skills/95-ourdigital-presales-seo/scripts/render_pdf.sh
Executable file
18
custom-skills/95-ourdigital-presales-seo/scripts/render_pdf.sh
Executable file
@@ -0,0 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
# Render an HTML brief to PDF via headless Chrome (uses system fonts → Korean OK).
|
||||
# Part of ourdigital-presales-seo (Stage 6).
|
||||
# Usage: render_pdf.sh <input.html> [output.pdf]
|
||||
set -euo pipefail
|
||||
HTML="${1:?usage: render_pdf.sh <input.html> [output.pdf]}"
|
||||
OUT="${2:-${HTML%.html}.pdf}"
|
||||
|
||||
CHROME="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
|
||||
if [ ! -x "$CHROME" ]; then CHROME="/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"; fi
|
||||
if [ ! -x "$CHROME" ]; then CHROME="/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"; fi
|
||||
if [ ! -x "$CHROME" ]; then echo "ERROR: no Chromium-based browser found for PDF rendering" >&2; exit 1; fi
|
||||
|
||||
DIR="$(cd "$(dirname "$HTML")" && pwd)"
|
||||
BASE="$(basename "$HTML")"
|
||||
"$CHROME" --headless --disable-gpu --no-pdf-header-footer \
|
||||
--print-to-pdf="$OUT" "file://$DIR/$BASE" 2>/dev/null
|
||||
echo "Wrote $OUT"
|
||||
@@ -0,0 +1,69 @@
|
||||
<!DOCTYPE html>
|
||||
<!--
|
||||
Client-facing pre-sales brief template (Korean). Stage 6.
|
||||
HOW TO USE: copy this file into the engagement output folder, then fill the
|
||||
{{TOKENS}} (and the four finding blocks) with the engagement's findings — keep
|
||||
the CSS as-is. Render to PDF with: bash scripts/render_pdf.sh <thisfile>.html
|
||||
Keep it ~1 page. Sanitize: no internal pricing strategy; tasteful competitor note only.
|
||||
-->
|
||||
<html lang="ko">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>{{PROSPECT}} 검색 가시성 사전 진단</title>
|
||||
<style>
|
||||
@page { size: A4; margin: 16mm 15mm 14mm 15mm; }
|
||||
* { box-sizing: border-box; }
|
||||
html, body { margin: 0; padding: 0; }
|
||||
body { font-family: "Apple SD Gothic Neo", "AppleGothic", "Noto Sans KR", sans-serif;
|
||||
color: #1f2733; font-size: 10.6pt; line-height: 1.6; -webkit-print-color-adjust: exact; }
|
||||
.accent { color: #1b6fb3; }
|
||||
header { border-bottom: 3px solid #1b6fb3; padding-bottom: 10px; margin-bottom: 14px; }
|
||||
header .kicker { font-size: 8.5pt; letter-spacing: .14em; color: #1b6fb3; font-weight: 700; text-transform: uppercase; }
|
||||
header h1 { font-size: 19pt; margin: 4px 0 2px; color: #11243d; }
|
||||
header .meta { font-size: 8.6pt; color: #6b7787; }
|
||||
.disclaimer { font-size: 8pt; color: #8a93a0; font-style: italic; margin-top: 4px; }
|
||||
h2 { font-size: 12.5pt; color: #11243d; margin: 18px 0 8px; padding-left: 9px; border-left: 4px solid #1b6fb3; }
|
||||
.lead { background: #f3f7fb; border: 1px solid #dce7f1; border-radius: 7px; padding: 11px 14px; font-size: 10.4pt; }
|
||||
.card { border: 1px solid #e2e8f0; border-radius: 7px; padding: 10px 13px; margin: 9px 0; page-break-inside: avoid; }
|
||||
.card .n { display: inline-block; min-width: 20px; height: 20px; line-height: 20px; text-align: center;
|
||||
background: #1b6fb3; color: #fff; border-radius: 50%; font-size: 9pt; font-weight: 700; margin-right: 7px; }
|
||||
.card h3 { display: inline; font-size: 11pt; color: #11243d; }
|
||||
.card ul { margin: 7px 0 6px; padding-left: 20px; }
|
||||
.why { font-size: 9.4pt; color: #334; background: #fbf6ec; border-left: 3px solid #e0a73c; padding: 5px 9px; border-radius: 3px; }
|
||||
.why b { color: #9a6a12; }
|
||||
.num { color: #c0392b; font-weight: 700; }
|
||||
.two { display: flex; gap: 14px; }
|
||||
.two > div { flex: 1; }
|
||||
.box { border: 1px solid #e2e8f0; border-radius: 7px; padding: 10px 13px; }
|
||||
ol.next { margin: 4px 0; padding-left: 18px; }
|
||||
footer { margin-top: 16px; border-top: 1px solid #e2e8f0; padding-top: 7px; font-size: 8.2pt; color: #8a93a0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<div class="kicker">SEO Pre-sales Brief · OurDigital</div>
|
||||
<h1>{{PROSPECT}} <span class="accent">검색 가시성 사전 진단</span></h1>
|
||||
<div class="meta">작성: OurDigital · {{DATE}} · 대상: {{DOMAIN}}</div>
|
||||
<div class="disclaimer">* 본 자료는 공개 데이터만으로 수행한 사전 스냅샷이며, Search Console 등 권한 확보 후 정밀 진단으로 보완됩니다.</div>
|
||||
</header>
|
||||
|
||||
<div class="lead">{{ONE_LINER — 자산은 최상급이나 검색 가시성이 규모에 미치지 못한다는 핵심 메시지}}</div>
|
||||
|
||||
<h2>핵심 발견</h2>
|
||||
<!-- Repeat this card for each headline finding (recommended 3-4). -->
|
||||
<div class="card">
|
||||
<span class="n">1</span><h3>{{FINDING_TITLE}}</h3>
|
||||
<ul><li>{{EVIDENCE_BULLET}} <span class="num">{{KEY_METRIC}}</span></li></ul>
|
||||
<div class="why"><b>왜 중요한가</b> — {{WHY_IT_MATTERS}}</div>
|
||||
</div>
|
||||
|
||||
<div class="two">
|
||||
<div class="box"><h2 style="margin-top:0;">기대 효과</h2>{{EXPECTED_IMPACT}}</div>
|
||||
<div class="box"><h2 style="margin-top:0;">다음 단계 제안</h2>
|
||||
<ol class="next"><li><b>30분 미팅</b></li><li><b>정밀 진단</b> (권한 확보 후)</li><li><b>단기 파일럿</b></li></ol>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<footer>OurDigital · andrew.yim@ourdigital.org · 공개 데이터 기준 사전 진단 ({{DATE}})</footer>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user