docs(skill): add 35-seo-signal-validation implementation plan
5 bite-sized tasks: SKILL.md measurement half (L1-L2) + decision half (L3-L4/verdict), gsc_signal_delta.py helper with TDD tests, marketplace registration + spec-layout reconcile, and a genesis-case smoke test that asserts the JHR "호텔" claim resolves to ARTIFACT. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KuT3W81t88QQFaxY2ruWv2
This commit is contained in:
729
custom-skills/35-seo-signal-validation/PLAN.md
Normal file
729
custom-skills/35-seo-signal-validation/PLAN.md
Normal file
@@ -0,0 +1,729 @@
|
|||||||
|
# SEO Signal Validation — Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Build the `35-seo-signal-validation` Claude Skill — a conductor that adjudicates whether a claimed SERP / Knowledge-Graph movement for a `(term, entity)` pair is real, misattributed, an artifact, or unprovable.
|
||||||
|
|
||||||
|
**Architecture:** A self-contained `SKILL.md` carries the decision procedure (entity classification → 4-layer evidence cascade → 4-way verdict). One stdlib Python helper (`gsc_signal_delta.py`) makes the L1/L4 GSC delta + mover-ranking deterministic (the part that was ad-hoc and overflowed context in the genesis case). The skill delegates measurement to existing skills (`20-seo-serp-analysis`, `21-seo-position-tracking`, `28-seo-knowledge-graph`) and is registered in the repo's marketplace manifest.
|
||||||
|
|
||||||
|
**Tech Stack:** Markdown skill (Claude Code format), Python 3 stdlib (`json`, `argparse`, `csv`), repo `.claude-plugin/marketplace.json`. No third-party deps. Code-only skill (no `desktop/` variant — matches precedent `95-ourdigital-presales-seo`, `96-ourdigital-estimate-engine`).
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- **Skill structure** = root `SKILL.md` (self-contained, ~180–220 lines, content NOT split into `references/`) + `code/` (CLAUDE.md + scripts). Code-only; **no `desktop/` variant**.
|
||||||
|
- **Register** the skill in `.claude-plugin/marketplace.json` under the `ourdigital-seo` plugin's `skills` array as `./custom-skills/35-seo-signal-validation`.
|
||||||
|
- **No new output directories** beyond the approved `custom-skills/35-seo-signal-validation/` folder (and its `code/scripts/fixtures/`).
|
||||||
|
- **Stateless, on-demand**: no cron/scheduler, no snapshot DB.
|
||||||
|
- **Notion writes via the notion-writer script only** — never Notion MCP write tools.
|
||||||
|
- **Never crawl/audit Marriott** for JHR — `sameAs` reference only.
|
||||||
|
- **Verify any Wikidata QID** against `Special:EntityData/{Q}.json` labels before trusting it (false-match guard: Q109455878 ≠ hotel, Q490787 ≠ Shinsegae Group).
|
||||||
|
- **Data-trust hierarchy**: 1st-party measured (GSC/GA4) > 3rd-party measured (backlinks, crawled rank) > 3rd-party modeled (estimated traffic).
|
||||||
|
- **Confidence cap**: third-party entities (no GSC/GA4 access) cannot reach `CONFIRMED` on traffic claims — at most `PARTIAL`, `ARTIFACT` only when live+entity reality clearly contradicts.
|
||||||
|
- **Verdict taxonomy**: `CONFIRMED | PARTIAL | ARTIFACT | INCONCLUSIVE`.
|
||||||
|
- **Branch**: all work commits to `feat/seo-signal-validation-skill` (already created; `DESIGN.md` already committed there).
|
||||||
|
- **Any client deliverable the skill emits** uses naming `{CODE}-{desc}-{class}-{YYYYMMDD}.{ext}`; KR client-facing content in Korean.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: `SKILL.md` — measurement half (frontmatter, classification, cascade L1–L2)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/SKILL.md`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: nothing (first task).
|
||||||
|
- Produces: the `SKILL.md` file with frontmatter `name: seo-signal-validation`; section anchors `## Step 0`, `## The validation loop` with layers `L1`/`L2`; references the helper script path `code/scripts/gsc_signal_delta.py` (implemented in Task 3).
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write `SKILL.md` frontmatter + measurement sections**
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
---
|
||||||
|
name: seo-signal-validation
|
||||||
|
description: |
|
||||||
|
Validate whether a claimed SERP / Knowledge-Graph movement for a (term, entity)
|
||||||
|
is real, misattributed, an artifact, or unprovable — before reporting impact.
|
||||||
|
Triggers: validate serp signal, is this ranking real, prove SEO impact,
|
||||||
|
SEMrush surge real, signal validation, real impact check,
|
||||||
|
신호 검증, 순위 변화 진짜, 오가닉 급증 검증, 임팩트 검증.
|
||||||
|
---
|
||||||
|
|
||||||
|
# SEO Signal Validation
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Given a `(term/intent, entity)` pair — and optionally a **claim** (a third-party
|
||||||
|
tool's reported movement) or a **baseline** (a prior state) — return an
|
||||||
|
evidence-backed verdict on whether SERP and Knowledge-Graph impact is real.
|
||||||
|
Built because modeled third-party signals (SEMrush/Ahrefs estimated organic
|
||||||
|
traffic, position snapshots) are easy to over-trust. This skill makes the
|
||||||
|
measured → live → entity → attribution cascade a single repeatable procedure
|
||||||
|
ending in a defensible verdict and a client-safe narrative.
|
||||||
|
|
||||||
|
## When to use (boundary)
|
||||||
|
|
||||||
|
This is the **conductor**, not an instrument. It sequences and synthesizes the
|
||||||
|
three measurement skills — it does not duplicate them.
|
||||||
|
|
||||||
|
| Use instead | When |
|
||||||
|
|---|---|
|
||||||
|
| `20-seo-serp-analysis` | You only need SERP composition / features |
|
||||||
|
| `21-seo-position-tracking` | You only need rank over time |
|
||||||
|
| `28-seo-knowledge-graph` | You only need an entity-presence audit |
|
||||||
|
| **this skill** | You must adjudicate whether a *claimed movement* is real across layers |
|
||||||
|
|
||||||
|
## Step 0 — Classify entity + pick mode
|
||||||
|
|
||||||
|
1. **Entity ownership** (gates which layers exist):
|
||||||
|
- **First-party** — a site/property you own or have GSC/GA4 access to (e.g. JHR
|
||||||
|
`sc-domain:josunhotel.com`, GA4 `258308769`) → **L1 measured available**.
|
||||||
|
- **Third-party** — a competitor brand or a person you do not control →
|
||||||
|
**L1 unavailable**; lean on L2 + L3 + clearly-tiered estimates; apply the
|
||||||
|
confidence cap (see Verdict). If unclear, ask once.
|
||||||
|
2. **Mode** (thin wrappers over the same cascade):
|
||||||
|
- `adjudicate(claim)` — a 3rd-party tool reports a move; confirm/refute.
|
||||||
|
- `prove(baseline)` — after our change; before/after from GSC/GA4 history.
|
||||||
|
- `snapshot()` — no claim; "where do we really stand."
|
||||||
|
|
||||||
|
## The validation loop (cost-ordered cascade, short-circuiting)
|
||||||
|
|
||||||
|
Run cheapest-first; stop early when a layer is already decisive.
|
||||||
|
|
||||||
|
### L1 — Measured (first-party ground truth) → via `21-seo-position-tracking`
|
||||||
|
|
||||||
|
- **GSC** `mcp__dda__gsc_fetch_performance`: the term at **query level** (exact)
|
||||||
|
AND **site-wide**, for **recent vs prior** windows. Pull clicks / impressions /
|
||||||
|
position / CTR. **Day-normalize** (compare windows differ in calendar-day count).
|
||||||
|
Note **~43% query-level anonymization** — the disclosed subset ≠ the whole.
|
||||||
|
- **GA4** `mcp__dda__ga4_run_report`: `Organic Search` sessions monthly trend
|
||||||
|
(dims `yearMonth` + `sessionDefaultChannelGroup`, metric `sessions`). GA4
|
||||||
|
includes Naver + all engines — use it to test whether a "surge" exceeds normal
|
||||||
|
month-to-month variance.
|
||||||
|
- **Compute deltas with the helper** (deterministic, avoids ad-hoc parsing):
|
||||||
|
save each GSC pull, then run
|
||||||
|
`python3 code/scripts/gsc_signal_delta.py --recent <recent.tsv> --prior <prior.tsv> --recent-days N --prior-days M --claim-term "<term>"`.
|
||||||
|
It returns day-normalized site totals, top gainers/decliners, and whether the
|
||||||
|
claimed term is among the real movers.
|
||||||
|
- **SHORT-CIRCUIT:** if the claimed keyword has trivial clicks and a real
|
||||||
|
position nowhere near the claim → **ARTIFACT**; stop unless the caller wants
|
||||||
|
the full picture.
|
||||||
|
|
||||||
|
### L2 — Live SERP (3rd-party measured, point-in-time) → via `20-seo-serp-analysis`
|
||||||
|
|
||||||
|
- **Geo-correct Google render** via `claude-in-chrome` (`navigate` → `read_page`):
|
||||||
|
force `gl`/`hl` + correct geo, `pws=0`; **decline precise-location prompts**.
|
||||||
|
Confirm whether the domain actually holds the claimed position; capture the
|
||||||
|
feature landscape (ads, local map-pack, PAA, knowledge panel) that explains why
|
||||||
|
a brand site can't own a head term.
|
||||||
|
- **Cheap rank spot-check**: `mcp__ourseo__check_serp(keyword, domain)`.
|
||||||
|
- **[KR market]** Naver SERP composition: `our research naver serp` (blog / cafe /
|
||||||
|
지식iN / Smart Store / brand zone) — Semrush/Ahrefs don't model Naver.
|
||||||
|
````
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify frontmatter parses and required anchors exist**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
python3 - <<'PY'
|
||||||
|
import sys, pathlib
|
||||||
|
p = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md")
|
||||||
|
t = p.read_text(encoding="utf-8")
|
||||||
|
assert t.startswith("---\n"), "missing frontmatter"
|
||||||
|
fm = t.split("---\n",2)[1]
|
||||||
|
assert "name: seo-signal-validation" in fm, "bad name"
|
||||||
|
assert "Triggers:" in fm, "missing triggers"
|
||||||
|
for anchor in ["## Step 0", "## The validation loop", "### L1", "### L2",
|
||||||
|
"gsc_signal_delta.py"]:
|
||||||
|
assert anchor in t, f"missing: {anchor}"
|
||||||
|
print("OK SKILL.md measurement half")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK SKILL.md measurement half`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
git add custom-skills/35-seo-signal-validation/SKILL.md
|
||||||
|
git commit -m "feat(skill): seo-signal-validation SKILL.md measurement half (L1-L2)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 2: `SKILL.md` — decision half (L3 KG, L4 synthesis, verdict, output)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `custom-skills/35-seo-signal-validation/SKILL.md` (append after L2)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: the `SKILL.md` from Task 1 (appends to it).
|
||||||
|
- Produces: sections `### L3`, `### L4`, `## Verdict`, `## Standing skepticism rules`, `## Output`, `## Non-goals` with the four verdict labels verbatim.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Append the decision sections to `SKILL.md`**
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
### L3 — Entity / Knowledge Graph → via `28-seo-knowledge-graph`
|
||||||
|
|
||||||
|
A real impact event should leave corroborating traces in the entity layer, not
|
||||||
|
just a rank number. Five checks:
|
||||||
|
|
||||||
|
1. **Google KG API** entity match + `resultScore` —
|
||||||
|
`mcp__ourseo__search_knowledge_graph(query)` (uses `GOOGLE_KG_API_KEY`).
|
||||||
|
2. **Wikidata** QID presence + key claims — **verify the QID against
|
||||||
|
`Special:EntityData/{Q}.json` labels before trusting it** (false-match guard:
|
||||||
|
Q109455878 = office tower ≠ hotel; Q490787 = Shinsegae Inc. ≠ Group).
|
||||||
|
3. **Knowledge Panel** presence/attributes on the live entity-name SERP (Chrome).
|
||||||
|
4. **sameAs** consistency on the entity's `Organization`/`Person` JSON-LD.
|
||||||
|
5. **[KR]** Naver 백과사전 / 지식iN presence.
|
||||||
|
|
||||||
|
`mcp__ourseo__monitor_brand` supplements with brand-mention / brand-SERP ownership.
|
||||||
|
|
||||||
|
### L4 — Attribution synthesis
|
||||||
|
|
||||||
|
Cross-check: does the **measured delta (L1)** corroborate the **live reality
|
||||||
|
(L2)**, and does the **entity layer (L3)** move consistently? The query-clicks
|
||||||
|
delta names the true drivers (brand/seasonal vs the claimed term).
|
||||||
|
|
||||||
|
## Verdict
|
||||||
|
|
||||||
|
| Verdict | Condition |
|
||||||
|
|---|---|
|
||||||
|
| **CONFIRMED** | Measured + live + (where relevant) entity all corroborate movement attributable to the term/intent |
|
||||||
|
| **PARTIAL** | Real movement, but misattributed, or only some layers agree |
|
||||||
|
| **ARTIFACT** | Modeling/snapshot artifact — measured + live reality don't support it |
|
||||||
|
| **INCONCLUSIVE** | Insufficient data (query anonymized, GSC lag, no entity baseline, third-party entity with no measured access) — name what's missing + how to resolve |
|
||||||
|
|
||||||
|
**Confidence cap:** third-party entities (no L1) cannot reach CONFIRMED on traffic
|
||||||
|
claims — at most PARTIAL; ARTIFACT only when live+entity clearly contradict.
|
||||||
|
|
||||||
|
Every verdict ships an **evidence ledger** (per layer: finding + data-trust tier +
|
||||||
|
corroborates/contradicts) and a **client-safe narrative** (the defensible story).
|
||||||
|
|
||||||
|
## Standing skepticism rules
|
||||||
|
|
||||||
|
- Estimated organic traffic = **smoke-detector, not scale** (Σ est-volume × position-CTR curve).
|
||||||
|
- **Head-term over-fire**: one high-volume keyword at an estimated high rank inflates the whole modeled number.
|
||||||
|
- **KR Naver blind spot**: Semrush models Google only; misses much of Korean organic.
|
||||||
|
- **Single-geo/device snapshot** diverges from GSC's national average.
|
||||||
|
- **Trust hierarchy**: 1st-party measured > 3rd-party measured > 3rd-party modeled.
|
||||||
|
|
||||||
|
## Output
|
||||||
|
|
||||||
|
- **Always**: inline report — verdict + evidence ledger + client-safe narrative +
|
||||||
|
"what would raise confidence."
|
||||||
|
- **Optional**: archive to Notion *Working with AI DB* (`data_source_id
|
||||||
|
f8f19ede-32bd-43ac-9f60-0651f6f40afe`) via the **notion-writer script** (never
|
||||||
|
Notion MCP write). Type=Memo/Research, Topic=SEO, Account Code as relevant.
|
||||||
|
- **Optional**: if a new generalizable gotcha emerges, append a memory entry to
|
||||||
|
the active workspace's memory dir.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
No cron/scheduler, no snapshot DB, no new directories. Does not replace the three
|
||||||
|
instrument skills. Returns INCONCLUSIVE rather than fabricating when data is thin.
|
||||||
|
**Never crawls/audits Marriott for JHR** (sameAs only).
|
||||||
|
````
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the four verdicts, confidence cap, and skepticism rules are present**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
python3 - <<'PY'
|
||||||
|
import pathlib
|
||||||
|
t = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md").read_text(encoding="utf-8")
|
||||||
|
for s in ["### L3", "### L4", "**CONFIRMED**", "**PARTIAL**", "**ARTIFACT**",
|
||||||
|
"**INCONCLUSIVE**", "Confidence cap", "smoke-detector, not scale",
|
||||||
|
"Special:EntityData", "## Output", "## Non-goals"]:
|
||||||
|
assert s in t, f"missing: {s}"
|
||||||
|
n = t.count("\n")
|
||||||
|
assert 150 <= n <= 320, f"SKILL.md length {n} lines outside expected band"
|
||||||
|
print(f"OK SKILL.md decision half ({n} lines)")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK SKILL.md decision half (… lines)`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
git add custom-skills/35-seo-signal-validation/SKILL.md
|
||||||
|
git commit -m "feat(skill): seo-signal-validation SKILL.md decision half (L3-L4, verdict, output)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 3: `gsc_signal_delta.py` helper + tests (TDD)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create test: `custom-skills/35-seo-signal-validation/code/scripts/test_gsc_signal_delta.py`
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/code/scripts/gsc_signal_delta.py`
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/code/scripts/requirements.txt`
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/code/CLAUDE.md`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: nothing at runtime.
|
||||||
|
- Produces: `compute_delta(recent: list[dict], prior: list[dict], recent_days: int, prior_days: int, claim_term: str|None=None, top_n: int=10) -> dict` and `load_gsc(path: str) -> list[dict]`; CLI `python3 gsc_signal_delta.py --recent --prior --recent-days --prior-days [--claim-term] [--top-n]`. Output dict keys: `site_totals`, `top_gainers`, `top_decliners`, `claim_term`, `verdict_hint`.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write the failing test**
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/scripts/test_gsc_signal_delta.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Tests for gsc_signal_delta. Run: `python3 test_gsc_signal_delta.py`
|
||||||
|
(also pytest-compatible). Stdlib only."""
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent))
|
||||||
|
from gsc_signal_delta import compute_delta # noqa: E402
|
||||||
|
|
||||||
|
# Genesis fixture: JHR "호텔" — flat head term, growth all brand (2026-06 case).
|
||||||
|
RECENT = [
|
||||||
|
{"query": "호텔", "clicks": 5, "impressions": 572, "position": 11.6},
|
||||||
|
{"query": "grand josun busan", "clicks": 250, "impressions": 4000, "position": 1.2},
|
||||||
|
{"query": "조선호텔", "clicks": 300, "impressions": 6000, "position": 1.1},
|
||||||
|
]
|
||||||
|
PRIOR = [
|
||||||
|
{"query": "호텔", "clicks": 9, "impressions": 371, "position": 18.1},
|
||||||
|
{"query": "grand josun busan", "clicks": 49, "impressions": 1500, "position": 3.4},
|
||||||
|
{"query": "조선호텔", "clicks": 150, "impressions": 5000, "position": 1.3},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_claim_term_flagged_artifact():
|
||||||
|
out = compute_delta(RECENT, PRIOR, 28, 30, claim_term="호텔")
|
||||||
|
ct = out["claim_term"]
|
||||||
|
assert ct["found"] is True
|
||||||
|
assert ct["in_top_movers"] is False
|
||||||
|
assert ct["click_share_pct"] < 1.0
|
||||||
|
assert "ARTIFACT" in out["verdict_hint"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_top_gainer_is_brand_term():
|
||||||
|
out = compute_delta(RECENT, PRIOR, 28, 30, claim_term="호텔")
|
||||||
|
assert out["top_gainers"][0]["query"] == "grand josun busan"
|
||||||
|
assert out["top_gainers"][0]["delta_clicks"] == 201
|
||||||
|
|
||||||
|
|
||||||
|
def test_day_normalization():
|
||||||
|
out = compute_delta(RECENT, PRIOR, 28, 30)
|
||||||
|
assert out["site_totals"]["recent"]["clicks_per_day"] == 19.82 # 555/28
|
||||||
|
assert out["site_totals"]["prior"]["clicks_per_day"] == 6.93 # 208/30
|
||||||
|
|
||||||
|
|
||||||
|
def test_positive_days_required():
|
||||||
|
try:
|
||||||
|
compute_delta(RECENT, PRIOR, 0, 30)
|
||||||
|
except ValueError:
|
||||||
|
return
|
||||||
|
raise AssertionError("expected ValueError for non-positive days")
|
||||||
|
|
||||||
|
|
||||||
|
def _run():
|
||||||
|
fns = [v for k, v in sorted(globals().items()) if k.startswith("test_")]
|
||||||
|
for fn in fns:
|
||||||
|
fn(); print(f"PASS {fn.__name__}")
|
||||||
|
print(f"\n{len(fns)} passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
_run()
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run test to verify it fails**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||||||
|
python3 test_gsc_signal_delta.py
|
||||||
|
```
|
||||||
|
Expected: FAIL — `ModuleNotFoundError: No module named 'gsc_signal_delta'`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Write the implementation**
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/scripts/gsc_signal_delta.py`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Day-normalized GSC query delta + mover ranking for signal validation.
|
||||||
|
|
||||||
|
Reads two GSC query exports (recent, prior) — JSON list or TSV with a header row
|
||||||
|
containing query / clicks / impressions / position — and reports day-normalized
|
||||||
|
site totals, top gainers/decliners, and whether a claimed term is a real mover.
|
||||||
|
This is the deterministic L1/L4 core of the 35-seo-signal-validation skill.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def _norm_row(r: dict) -> dict:
|
||||||
|
def num(*keys, default=0.0):
|
||||||
|
for k in keys:
|
||||||
|
if k in r and r[k] not in (None, ""):
|
||||||
|
try:
|
||||||
|
return float(str(r[k]).replace(",", ""))
|
||||||
|
except ValueError:
|
||||||
|
pass
|
||||||
|
return default
|
||||||
|
query = (r.get("query") or r.get("term") or "")
|
||||||
|
if isinstance(r.get("keys"), list) and r["keys"]:
|
||||||
|
query = str(r["keys"][0])
|
||||||
|
return {
|
||||||
|
"query": str(query).strip(),
|
||||||
|
"clicks": num("clicks"),
|
||||||
|
"impressions": num("impressions", "impr"),
|
||||||
|
"position": num("position", "pos", default=0.0),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def load_gsc(path: str) -> list[dict]:
|
||||||
|
"""Parse a GSC export (JSON list/{rows:[...]} or TSV-with-header)."""
|
||||||
|
text = Path(path).read_text(encoding="utf-8").strip()
|
||||||
|
if not text:
|
||||||
|
return []
|
||||||
|
if text[0] in "[{":
|
||||||
|
data = json.loads(text)
|
||||||
|
if isinstance(data, dict):
|
||||||
|
data = data.get("rows", [])
|
||||||
|
return [_norm_row(r) for r in data]
|
||||||
|
lines = text.splitlines()
|
||||||
|
header = [h.strip().lower() for h in lines[0].split("\t")]
|
||||||
|
rows = []
|
||||||
|
for line in lines[1:]:
|
||||||
|
if line.strip():
|
||||||
|
rows.append(_norm_row(dict(zip(header, line.split("\t")))))
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def _by_query(rows: list[dict]) -> dict:
|
||||||
|
return {r["query"]: r for r in rows if r["query"]}
|
||||||
|
|
||||||
|
|
||||||
|
def compute_delta(recent, prior, recent_days, prior_days,
|
||||||
|
claim_term=None, top_n=10) -> dict:
|
||||||
|
if recent_days <= 0 or prior_days <= 0:
|
||||||
|
raise ValueError("recent_days and prior_days must be positive")
|
||||||
|
r_by, p_by = _by_query(recent), _by_query(prior)
|
||||||
|
|
||||||
|
def totals(rows):
|
||||||
|
return {"clicks": sum(r["clicks"] for r in rows),
|
||||||
|
"impressions": sum(r["impressions"] for r in rows)}
|
||||||
|
rt, pt = totals(recent), totals(prior)
|
||||||
|
|
||||||
|
def per_day(total, days):
|
||||||
|
return round(total / days, 2)
|
||||||
|
|
||||||
|
def pct(new, old):
|
||||||
|
return round((new - old) / old * 100, 1) if old else None
|
||||||
|
|
||||||
|
r_cpd, p_cpd = per_day(rt["clicks"], recent_days), per_day(pt["clicks"], prior_days)
|
||||||
|
r_ipd, p_ipd = per_day(rt["impressions"], recent_days), per_day(pt["impressions"], prior_days)
|
||||||
|
|
||||||
|
deltas = []
|
||||||
|
for q in set(r_by) | set(p_by):
|
||||||
|
rc = r_by.get(q, {}).get("clicks", 0.0)
|
||||||
|
pc = p_by.get(q, {}).get("clicks", 0.0)
|
||||||
|
deltas.append({"query": q, "delta_clicks": rc - pc,
|
||||||
|
"recent_clicks": rc, "prior_clicks": pc})
|
||||||
|
deltas.sort(key=lambda d: d["delta_clicks"], reverse=True)
|
||||||
|
gainers = [d for d in deltas if d["delta_clicks"] > 0][:top_n]
|
||||||
|
decliners = sorted([d for d in deltas if d["delta_clicks"] < 0],
|
||||||
|
key=lambda d: d["delta_clicks"])[:top_n]
|
||||||
|
|
||||||
|
out = {
|
||||||
|
"site_totals": {
|
||||||
|
"recent": {**rt, "clicks_per_day": r_cpd,
|
||||||
|
"impressions_per_day": r_ipd, "days": recent_days},
|
||||||
|
"prior": {**pt, "clicks_per_day": p_cpd,
|
||||||
|
"impressions_per_day": p_ipd, "days": prior_days},
|
||||||
|
"clicks_per_day_pct": pct(r_cpd, p_cpd),
|
||||||
|
"impressions_per_day_pct": pct(r_ipd, p_ipd),
|
||||||
|
},
|
||||||
|
"top_gainers": gainers,
|
||||||
|
"top_decliners": decliners,
|
||||||
|
"claim_term": None,
|
||||||
|
"verdict_hint": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
if claim_term:
|
||||||
|
gainer_terms = {g["query"] for g in gainers}
|
||||||
|
rc, pc = r_by.get(claim_term, {}), p_by.get(claim_term, {})
|
||||||
|
in_movers = claim_term in gainer_terms
|
||||||
|
share = (rc.get("clicks", 0.0) / rt["clicks"] * 100) if rt["clicks"] else 0.0
|
||||||
|
out["claim_term"] = {
|
||||||
|
"term": claim_term, "found": bool(rc or pc),
|
||||||
|
"recent": {"clicks": rc.get("clicks", 0.0),
|
||||||
|
"impressions": rc.get("impressions", 0.0),
|
||||||
|
"position": rc.get("position")},
|
||||||
|
"prior": {"clicks": pc.get("clicks", 0.0),
|
||||||
|
"impressions": pc.get("impressions", 0.0),
|
||||||
|
"position": pc.get("position")},
|
||||||
|
"in_top_movers": in_movers,
|
||||||
|
"click_share_pct": round(share, 2),
|
||||||
|
}
|
||||||
|
if not in_movers and share < 1.0:
|
||||||
|
out["verdict_hint"] = (
|
||||||
|
f"'{claim_term}' contributes {share:.2f}% of recent clicks and is "
|
||||||
|
f"absent from top movers -> claimed impact likely ARTIFACT; real "
|
||||||
|
f"movement is elsewhere (see top_gainers).")
|
||||||
|
elif in_movers:
|
||||||
|
out["verdict_hint"] = (
|
||||||
|
f"'{claim_term}' is among top movers -> claim plausibly CONFIRMED/"
|
||||||
|
f"PARTIAL; corroborate with live SERP + entity layer.")
|
||||||
|
else:
|
||||||
|
out["verdict_hint"] = (
|
||||||
|
f"'{claim_term}' has non-trivial share ({share:.2f}%) but is not a "
|
||||||
|
f"top mover -> PARTIAL; inspect attribution.")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None):
|
||||||
|
ap = argparse.ArgumentParser(description="GSC signal delta for signal validation")
|
||||||
|
ap.add_argument("--recent", required=True)
|
||||||
|
ap.add_argument("--prior", required=True)
|
||||||
|
ap.add_argument("--recent-days", type=int, required=True)
|
||||||
|
ap.add_argument("--prior-days", type=int, required=True)
|
||||||
|
ap.add_argument("--claim-term", default=None)
|
||||||
|
ap.add_argument("--top-n", type=int, default=10)
|
||||||
|
a = ap.parse_args(argv)
|
||||||
|
out = compute_delta(load_gsc(a.recent), load_gsc(a.prior),
|
||||||
|
a.recent_days, a.prior_days, a.claim_term, a.top_n)
|
||||||
|
json.dump(out, sys.stdout, ensure_ascii=False, indent=2)
|
||||||
|
sys.stdout.write("\n")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run tests to verify they pass**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||||||
|
python3 test_gsc_signal_delta.py
|
||||||
|
```
|
||||||
|
Expected: `PASS test_claim_term_flagged_artifact` … `4 passed`
|
||||||
|
|
||||||
|
- [ ] **Step 5: Create `requirements.txt` and `code/CLAUDE.md`**
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/scripts/requirements.txt`:
|
||||||
|
```text
|
||||||
|
# gsc_signal_delta.py uses the Python 3 standard library only — no deps.
|
||||||
|
```
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/CLAUDE.md`:
|
||||||
|
```markdown
|
||||||
|
# seo-signal-validation — code environment notes
|
||||||
|
|
||||||
|
## Helper: scripts/gsc_signal_delta.py
|
||||||
|
Deterministic L1/L4 GSC delta. Feed it two saved GSC query exports (recent,
|
||||||
|
prior) as JSON or TSV (columns: query, clicks, impressions, position).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/gsc_signal_delta.py \
|
||||||
|
--recent recent.tsv --prior prior.tsv \
|
||||||
|
--recent-days 28 --prior-days 30 --claim-term "호텔"
|
||||||
|
```
|
||||||
|
Returns day-normalized site totals, top gainers/decliners, and a `verdict_hint`
|
||||||
|
(heuristic only — the final verdict is the skill's job, after L2/L3).
|
||||||
|
|
||||||
|
## Getting the exports
|
||||||
|
`mcp__dda__gsc_fetch_performance` (property pinned per workspace, e.g. JHR
|
||||||
|
`sc-domain:josunhotel.com`) → save the query-dimension rows to a file → run the
|
||||||
|
script. GSC anonymizes ~43% of query clicks; the disclosed subset ≠ the whole.
|
||||||
|
|
||||||
|
## Env / access
|
||||||
|
- `GOOGLE_KG_API_KEY` for `mcp__ourseo__search_knowledge_graph` (L3).
|
||||||
|
- GSC/GA4 only exist for first-party properties — third-party entities skip L1.
|
||||||
|
- Never crawl/audit Marriott for JHR (sameAs only).
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 6: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
git add custom-skills/35-seo-signal-validation/code
|
||||||
|
git commit -m "feat(skill): gsc_signal_delta helper + tests + code notes"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 4: Register in marketplace + reconcile DESIGN.md structure
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `.claude-plugin/marketplace.json` (add to `ourdigital-seo` → `skills`)
|
||||||
|
- Modify: `custom-skills/35-seo-signal-validation/DESIGN.md:§7` (replace `references/` layout with actual `code/` layout; mark Code-only)
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: the skill folder from Tasks 1–3.
|
||||||
|
- Produces: a registered, discoverable skill; a spec whose §7 matches the built structure.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Add the skill path to the manifest**
|
||||||
|
|
||||||
|
In `.claude-plugin/marketplace.json`, inside the `ourdigital-seo` plugin's
|
||||||
|
`skills` array, add (keep numeric order; insert after the `34-seo-reporting-dashboard` entry):
|
||||||
|
```json
|
||||||
|
"./custom-skills/35-seo-signal-validation",
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify the manifest still parses and contains the entry**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
python3 - <<'PY'
|
||||||
|
import json, pathlib
|
||||||
|
m = json.loads(pathlib.Path(".claude-plugin/marketplace.json").read_text())
|
||||||
|
seo = next(p for p in m["plugins"] if p["name"] == "ourdigital-seo")
|
||||||
|
assert "./custom-skills/35-seo-signal-validation" in seo["skills"], "not registered"
|
||||||
|
print("OK manifest valid + skill registered")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK manifest valid + skill registered`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Reconcile `DESIGN.md` §7 with the real structure**
|
||||||
|
|
||||||
|
In `custom-skills/35-seo-signal-validation/DESIGN.md`, replace the §7 "Repo
|
||||||
|
layout & conventions" code block (the `references/...` sketch) with:
|
||||||
|
```text
|
||||||
|
35-seo-signal-validation/
|
||||||
|
SKILL.md self-contained: classification, 4-layer cascade,
|
||||||
|
5 KG checks, 4-way verdict, skepticism rules, output
|
||||||
|
DESIGN.md PLAN.md spec + plan (live with the skill; no new top-level dir)
|
||||||
|
code/
|
||||||
|
CLAUDE.md code-environment notes (env, export→script flow)
|
||||||
|
scripts/
|
||||||
|
gsc_signal_delta.py deterministic L1/L4 GSC delta + mover ranking
|
||||||
|
test_gsc_signal_delta.py
|
||||||
|
requirements.txt (stdlib only)
|
||||||
|
```
|
||||||
|
And add one line under it: `Target environment: Claude Code only (no desktop/ variant — matches precedent 95/96). Registered in .claude-plugin/marketplace.json under ourdigital-seo.`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Verify the spec no longer references the old layout**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
python3 - <<'PY'
|
||||||
|
import pathlib
|
||||||
|
t = pathlib.Path("custom-skills/35-seo-signal-validation/DESIGN.md").read_text(encoding="utf-8")
|
||||||
|
assert "references/\n evidence-cascade.md" not in t, "old layout still present"
|
||||||
|
assert "gsc_signal_delta.py" in t and "marketplace.json" in t, "structure not reconciled"
|
||||||
|
print("OK DESIGN.md §7 reconciled")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK DESIGN.md §7 reconciled`
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
git add .claude-plugin/marketplace.json custom-skills/35-seo-signal-validation/DESIGN.md
|
||||||
|
git commit -m "feat(skill): register seo-signal-validation in marketplace; reconcile spec layout"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 5: Smoke test — genesis case end-to-end + consistency gate
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-recent.tsv`
|
||||||
|
- Create: `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-prior.tsv`
|
||||||
|
|
||||||
|
**Interfaces:**
|
||||||
|
- Consumes: `gsc_signal_delta.py` CLI (Task 3) and the full `SKILL.md` (Tasks 1–2).
|
||||||
|
- Produces: a reproducible CLI smoke run proving the genesis "호텔" case yields an ARTIFACT-leaning hint; a final spec↔skill consistency check.
|
||||||
|
|
||||||
|
- [ ] **Step 1: Create the genesis fixtures (TSV)**
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-recent.tsv`:
|
||||||
|
```text
|
||||||
|
query clicks impressions position
|
||||||
|
호텔 5 572 11.6
|
||||||
|
grand josun busan 250 4000 1.2
|
||||||
|
조선호텔 300 6000 1.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Create `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-prior.tsv`:
|
||||||
|
```text
|
||||||
|
query clicks impressions position
|
||||||
|
호텔 9 371 18.1
|
||||||
|
grand josun busan 49 1500 3.4
|
||||||
|
조선호텔 150 5000 1.3
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run the CLI end-to-end and verify the verdict hint**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||||||
|
python3 gsc_signal_delta.py --recent fixtures/jhr-hotel-recent.tsv \
|
||||||
|
--prior fixtures/jhr-hotel-prior.tsv --recent-days 28 --prior-days 30 \
|
||||||
|
--claim-term "호텔" | python3 - <<'PY'
|
||||||
|
import json, sys
|
||||||
|
o = json.load(sys.stdin)
|
||||||
|
assert o["claim_term"]["in_top_movers"] is False
|
||||||
|
assert o["claim_term"]["click_share_pct"] < 1.0
|
||||||
|
assert "ARTIFACT" in o["verdict_hint"]
|
||||||
|
assert o["top_gainers"][0]["query"] == "grand josun busan"
|
||||||
|
print("OK smoke: 호텔 → ARTIFACT-leaning; top mover = brand term")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK smoke: 호텔 → ARTIFACT-leaning; top mover = brand term`
|
||||||
|
|
||||||
|
- [ ] **Step 3: Consistency gate — every spec default maps to skill content**
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
python3 - <<'PY'
|
||||||
|
import pathlib
|
||||||
|
sk = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md").read_text(encoding="utf-8")
|
||||||
|
# Default 1: 5 KG checks Default 2: 4 verdicts Default 3: output triple
|
||||||
|
for s in ["Google KG API", "Wikidata", "Knowledge Panel", "sameAs", "지식iN"]:
|
||||||
|
assert s in sk, f"KG check missing: {s}"
|
||||||
|
for s in ["CONFIRMED", "PARTIAL", "ARTIFACT", "INCONCLUSIVE"]:
|
||||||
|
assert s in sk, f"verdict missing: {s}"
|
||||||
|
for s in ["notion-writer", "evidence ledger", "client-safe narrative"]:
|
||||||
|
assert s.lower() in sk.lower(), f"output element missing: {s}"
|
||||||
|
# Default 5: triggers (KR + EN)
|
||||||
|
assert "신호 검증" in sk and "validate serp signal" in sk, "triggers missing"
|
||||||
|
print("OK all five approved defaults present in SKILL.md")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
Expected: `OK all five approved defaults present in SKILL.md`
|
||||||
|
|
||||||
|
- [ ] **Step 4: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd ~/Project/our-claude-skills
|
||||||
|
git add custom-skills/35-seo-signal-validation/code/scripts/fixtures
|
||||||
|
git commit -m "test(skill): genesis 호텔 smoke fixtures + end-to-end ARTIFACT check"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Self-Review
|
||||||
|
|
||||||
|
**1. Spec coverage** (each DESIGN.md section → task):
|
||||||
|
- §1 Purpose, §2 Boundary → Task 1 (SKILL.md Purpose/boundary). ✓
|
||||||
|
- §3 Engine (entity classification, L1–L4, short-circuit) → Tasks 1 (L1–L2) + 2 (L3–L4); L1/L4 delta computation → Task 3 script. ✓
|
||||||
|
- §4 Modes → Task 1 (Step 0 mode dispatch). ✓
|
||||||
|
- §5 Verdict + skepticism + confidence cap → Task 2. ✓
|
||||||
|
- §6 Output → Task 2. ✓
|
||||||
|
- §7 Repo layout → corrected in Task 4 (was wrong in spec); registration in Task 4. ✓
|
||||||
|
- §8 Non-goals → Task 2. ✓
|
||||||
|
- §9 Future options → intentionally not implemented (YAGNI). ✓
|
||||||
|
- Genesis verification → Task 5 smoke. ✓
|
||||||
|
|
||||||
|
**2. Placeholder scan:** No TBD/TODO; all code blocks complete; every test shows real assertions; commands show expected output. ✓
|
||||||
|
|
||||||
|
**3. Type consistency:** `compute_delta(recent, prior, recent_days, prior_days, claim_term=None, top_n=10)` and `load_gsc(path)` are referenced identically in Task 3 (definition + test) and Task 5 (CLI). Output keys (`site_totals`, `top_gainers`, `claim_term.in_top_movers`, `claim_term.click_share_pct`, `verdict_hint`) match across the implementation, the tests, and both smoke checks. Day-normalization fixtures (555/28=19.82, 208/30=6.93; gainer delta 201) are arithmetically consistent. ✓
|
||||||
|
|
||||||
|
**No gaps found.**
|
||||||
Reference in New Issue
Block a user