730 lines
31 KiB
Markdown
730 lines
31 KiB
Markdown
# SEO Signal Validation — Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Build the `35-seo-signal-validation` Claude Skill — a conductor that adjudicates whether a claimed SERP / Knowledge-Graph movement for a `(term, entity)` pair is real, misattributed, an artifact, or unprovable.
|
||
|
||
**Architecture:** A self-contained `SKILL.md` carries the decision procedure (entity classification → 4-layer evidence cascade → 4-way verdict). One stdlib Python helper (`gsc_signal_delta.py`) makes the L1/L4 GSC delta + mover-ranking deterministic (the part that was ad-hoc and overflowed context in the genesis case). The skill delegates measurement to existing skills (`20-seo-serp-analysis`, `21-seo-position-tracking`, `28-seo-knowledge-graph`) and is registered in the repo's marketplace manifest.
|
||
|
||
**Tech Stack:** Markdown skill (Claude Code format), Python 3 stdlib (`json`, `argparse`, `csv`), repo `.claude-plugin/marketplace.json`. No third-party deps. Code-only skill (no `desktop/` variant — matches precedent `95-ourdigital-presales-seo`, `96-ourdigital-estimate-engine`).
|
||
|
||
## Global Constraints
|
||
|
||
- **Skill structure** = root `SKILL.md` (self-contained, ~180–220 lines, content NOT split into `references/`) + `code/` (CLAUDE.md + scripts). Code-only; **no `desktop/` variant**.
|
||
- **Register** the skill in `.claude-plugin/marketplace.json` under the `ourdigital-seo` plugin's `skills` array as `./custom-skills/35-seo-signal-validation`.
|
||
- **No new output directories** beyond the approved `custom-skills/35-seo-signal-validation/` folder (and its `code/scripts/fixtures/`).
|
||
- **Stateless, on-demand**: no cron/scheduler, no snapshot DB.
|
||
- **Notion writes via the notion-writer script only** — never Notion MCP write tools.
|
||
- **Never crawl/audit Marriott** for JHR — `sameAs` reference only.
|
||
- **Verify any Wikidata QID** against `Special:EntityData/{Q}.json` labels before trusting it (false-match guard: Q109455878 ≠ hotel, Q490787 ≠ Shinsegae Group).
|
||
- **Data-trust hierarchy**: 1st-party measured (GSC/GA4) > 3rd-party measured (backlinks, crawled rank) > 3rd-party modeled (estimated traffic).
|
||
- **Confidence cap**: third-party entities (no GSC/GA4 access) cannot reach `CONFIRMED` on traffic claims — at most `PARTIAL`, `ARTIFACT` only when live+entity reality clearly contradicts.
|
||
- **Verdict taxonomy**: `CONFIRMED | PARTIAL | ARTIFACT | INCONCLUSIVE`.
|
||
- **Branch**: all work commits to `feat/seo-signal-validation-skill` (already created; `DESIGN.md` already committed there).
|
||
- **Any client deliverable the skill emits** uses naming `{CODE}-{desc}-{class}-{YYYYMMDD}.{ext}`; KR client-facing content in Korean.
|
||
|
||
---
|
||
|
||
### Task 1: `SKILL.md` — measurement half (frontmatter, classification, cascade L1–L2)
|
||
|
||
**Files:**
|
||
- Create: `custom-skills/35-seo-signal-validation/SKILL.md`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing (first task).
|
||
- Produces: the `SKILL.md` file with frontmatter `name: seo-signal-validation`; section anchors `## Step 0`, `## The validation loop` with layers `L1`/`L2`; references the helper script path `code/scripts/gsc_signal_delta.py` (implemented in Task 3).
|
||
|
||
- [ ] **Step 1: Write `SKILL.md` frontmatter + measurement sections**
|
||
|
||
````markdown
|
||
---
|
||
name: seo-signal-validation
|
||
description: |
|
||
Validate whether a claimed SERP / Knowledge-Graph movement for a (term, entity)
|
||
is real, misattributed, an artifact, or unprovable — before reporting impact.
|
||
Triggers: validate serp signal, is this ranking real, prove SEO impact,
|
||
SEMrush surge real, signal validation, real impact check,
|
||
신호 검증, 순위 변화 진짜, 오가닉 급증 검증, 임팩트 검증.
|
||
---
|
||
|
||
# SEO Signal Validation
|
||
|
||
## Purpose
|
||
|
||
Given a `(term/intent, entity)` pair — and optionally a **claim** (a third-party
|
||
tool's reported movement) or a **baseline** (a prior state) — return an
|
||
evidence-backed verdict on whether SERP and Knowledge-Graph impact is real.
|
||
Built because modeled third-party signals (SEMrush/Ahrefs estimated organic
|
||
traffic, position snapshots) are easy to over-trust. This skill makes the
|
||
measured → live → entity → attribution cascade a single repeatable procedure
|
||
ending in a defensible verdict and a client-safe narrative.
|
||
|
||
## When to use (boundary)
|
||
|
||
This is the **conductor**, not an instrument. It sequences and synthesizes the
|
||
three measurement skills — it does not duplicate them.
|
||
|
||
| Use instead | When |
|
||
|---|---|
|
||
| `20-seo-serp-analysis` | You only need SERP composition / features |
|
||
| `21-seo-position-tracking` | You only need rank over time |
|
||
| `28-seo-knowledge-graph` | You only need an entity-presence audit |
|
||
| **this skill** | You must adjudicate whether a *claimed movement* is real across layers |
|
||
|
||
## Step 0 — Classify entity + pick mode
|
||
|
||
1. **Entity ownership** (gates which layers exist):
|
||
- **First-party** — a site/property you own or have GSC/GA4 access to (e.g. JHR
|
||
`sc-domain:josunhotel.com`, GA4 `258308769`) → **L1 measured available**.
|
||
- **Third-party** — a competitor brand or a person you do not control →
|
||
**L1 unavailable**; lean on L2 + L3 + clearly-tiered estimates; apply the
|
||
confidence cap (see Verdict). If unclear, ask once.
|
||
2. **Mode** (thin wrappers over the same cascade):
|
||
- `adjudicate(claim)` — a 3rd-party tool reports a move; confirm/refute.
|
||
- `prove(baseline)` — after our change; before/after from GSC/GA4 history.
|
||
- `snapshot()` — no claim; "where do we really stand."
|
||
|
||
## The validation loop (cost-ordered cascade, short-circuiting)
|
||
|
||
Run cheapest-first; stop early when a layer is already decisive.
|
||
|
||
### L1 — Measured (first-party ground truth) → via `21-seo-position-tracking`
|
||
|
||
- **GSC** `mcp__dda__gsc_fetch_performance`: the term at **query level** (exact)
|
||
AND **site-wide**, for **recent vs prior** windows. Pull clicks / impressions /
|
||
position / CTR. **Day-normalize** (compare windows differ in calendar-day count).
|
||
Note **~43% query-level anonymization** — the disclosed subset ≠ the whole.
|
||
- **GA4** `mcp__dda__ga4_run_report`: `Organic Search` sessions monthly trend
|
||
(dims `yearMonth` + `sessionDefaultChannelGroup`, metric `sessions`). GA4
|
||
includes Naver + all engines — use it to test whether a "surge" exceeds normal
|
||
month-to-month variance.
|
||
- **Compute deltas with the helper** (deterministic, avoids ad-hoc parsing):
|
||
save each GSC pull, then run
|
||
`python3 code/scripts/gsc_signal_delta.py --recent <recent.tsv> --prior <prior.tsv> --recent-days N --prior-days M --claim-term "<term>"`.
|
||
It returns day-normalized site totals, top gainers/decliners, and whether the
|
||
claimed term is among the real movers.
|
||
- **SHORT-CIRCUIT:** if the claimed keyword has trivial clicks and a real
|
||
position nowhere near the claim → **ARTIFACT**; stop unless the caller wants
|
||
the full picture.
|
||
|
||
### L2 — Live SERP (3rd-party measured, point-in-time) → via `20-seo-serp-analysis`
|
||
|
||
- **Geo-correct Google render** via `claude-in-chrome` (`navigate` → `read_page`):
|
||
force `gl`/`hl` + correct geo, `pws=0`; **decline precise-location prompts**.
|
||
Confirm whether the domain actually holds the claimed position; capture the
|
||
feature landscape (ads, local map-pack, PAA, knowledge panel) that explains why
|
||
a brand site can't own a head term.
|
||
- **Cheap rank spot-check**: `mcp__ourseo__check_serp(keyword, domain)`.
|
||
- **[KR market]** Naver SERP composition: `our research naver serp` (blog / cafe /
|
||
지식iN / Smart Store / brand zone) — Semrush/Ahrefs don't model Naver.
|
||
````
|
||
|
||
- [ ] **Step 2: Verify frontmatter parses and required anchors exist**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
python3 - <<'PY'
|
||
import sys, pathlib
|
||
p = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md")
|
||
t = p.read_text(encoding="utf-8")
|
||
assert t.startswith("---\n"), "missing frontmatter"
|
||
fm = t.split("---\n",2)[1]
|
||
assert "name: seo-signal-validation" in fm, "bad name"
|
||
assert "Triggers:" in fm, "missing triggers"
|
||
for anchor in ["## Step 0", "## The validation loop", "### L1", "### L2",
|
||
"gsc_signal_delta.py"]:
|
||
assert anchor in t, f"missing: {anchor}"
|
||
print("OK SKILL.md measurement half")
|
||
PY
|
||
```
|
||
Expected: `OK SKILL.md measurement half`
|
||
|
||
- [ ] **Step 3: Commit**
|
||
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
git add custom-skills/35-seo-signal-validation/SKILL.md
|
||
git commit -m "feat(skill): seo-signal-validation SKILL.md measurement half (L1-L2)"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: `SKILL.md` — decision half (L3 KG, L4 synthesis, verdict, output)
|
||
|
||
**Files:**
|
||
- Modify: `custom-skills/35-seo-signal-validation/SKILL.md` (append after L2)
|
||
|
||
**Interfaces:**
|
||
- Consumes: the `SKILL.md` from Task 1 (appends to it).
|
||
- Produces: sections `### L3`, `### L4`, `## Verdict`, `## Standing skepticism rules`, `## Output`, `## Non-goals` with the four verdict labels verbatim.
|
||
|
||
- [ ] **Step 1: Append the decision sections to `SKILL.md`**
|
||
|
||
````markdown
|
||
### L3 — Entity / Knowledge Graph → via `28-seo-knowledge-graph`
|
||
|
||
A real impact event should leave corroborating traces in the entity layer, not
|
||
just a rank number. Five checks:
|
||
|
||
1. **Google KG API** entity match + `resultScore` —
|
||
`mcp__ourseo__search_knowledge_graph(query)` (uses `GOOGLE_KG_API_KEY`).
|
||
2. **Wikidata** QID presence + key claims — **verify the QID against
|
||
`Special:EntityData/{Q}.json` labels before trusting it** (false-match guard:
|
||
Q109455878 = office tower ≠ hotel; Q490787 = Shinsegae Inc. ≠ Group).
|
||
3. **Knowledge Panel** presence/attributes on the live entity-name SERP (Chrome).
|
||
4. **sameAs** consistency on the entity's `Organization`/`Person` JSON-LD.
|
||
5. **[KR]** Naver 백과사전 / 지식iN presence.
|
||
|
||
`mcp__ourseo__monitor_brand` supplements with brand-mention / brand-SERP ownership.
|
||
|
||
### L4 — Attribution synthesis
|
||
|
||
Cross-check: does the **measured delta (L1)** corroborate the **live reality
|
||
(L2)**, and does the **entity layer (L3)** move consistently? The query-clicks
|
||
delta names the true drivers (brand/seasonal vs the claimed term).
|
||
|
||
## Verdict
|
||
|
||
| Verdict | Condition |
|
||
|---|---|
|
||
| **CONFIRMED** | Measured + live + (where relevant) entity all corroborate movement attributable to the term/intent |
|
||
| **PARTIAL** | Real movement, but misattributed, or only some layers agree |
|
||
| **ARTIFACT** | Modeling/snapshot artifact — measured + live reality don't support it |
|
||
| **INCONCLUSIVE** | Insufficient data (query anonymized, GSC lag, no entity baseline, third-party entity with no measured access) — name what's missing + how to resolve |
|
||
|
||
**Confidence cap:** third-party entities (no L1) cannot reach CONFIRMED on traffic
|
||
claims — at most PARTIAL; ARTIFACT only when live+entity clearly contradict.
|
||
|
||
Every verdict ships an **evidence ledger** (per layer: finding + data-trust tier +
|
||
corroborates/contradicts) and a **client-safe narrative** (the defensible story).
|
||
|
||
## Standing skepticism rules
|
||
|
||
- Estimated organic traffic = **smoke-detector, not scale** (Σ est-volume × position-CTR curve).
|
||
- **Head-term over-fire**: one high-volume keyword at an estimated high rank inflates the whole modeled number.
|
||
- **KR Naver blind spot**: Semrush models Google only; misses much of Korean organic.
|
||
- **Single-geo/device snapshot** diverges from GSC's national average.
|
||
- **Trust hierarchy**: 1st-party measured > 3rd-party measured > 3rd-party modeled.
|
||
|
||
## Output
|
||
|
||
- **Always**: inline report — verdict + evidence ledger + client-safe narrative +
|
||
"what would raise confidence."
|
||
- **Optional**: archive to Notion *Working with AI DB* (`data_source_id
|
||
f8f19ede-32bd-43ac-9f60-0651f6f40afe`) via the **notion-writer script** (never
|
||
Notion MCP write). Type=Memo/Research, Topic=SEO, Account Code as relevant.
|
||
- **Optional**: if a new generalizable gotcha emerges, append a memory entry to
|
||
the active workspace's memory dir.
|
||
|
||
## Non-goals
|
||
|
||
No cron/scheduler, no snapshot DB, no new directories. Does not replace the three
|
||
instrument skills. Returns INCONCLUSIVE rather than fabricating when data is thin.
|
||
**Never crawls/audits Marriott for JHR** (sameAs only).
|
||
````
|
||
|
||
- [ ] **Step 2: Verify the four verdicts, confidence cap, and skepticism rules are present**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
python3 - <<'PY'
|
||
import pathlib
|
||
t = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md").read_text(encoding="utf-8")
|
||
for s in ["### L3", "### L4", "**CONFIRMED**", "**PARTIAL**", "**ARTIFACT**",
|
||
"**INCONCLUSIVE**", "Confidence cap", "smoke-detector, not scale",
|
||
"Special:EntityData", "## Output", "## Non-goals"]:
|
||
assert s in t, f"missing: {s}"
|
||
n = t.count("\n")
|
||
assert 130 <= n <= 320, f"SKILL.md length {n} lines outside expected band"
|
||
print(f"OK SKILL.md decision half ({n} lines)")
|
||
PY
|
||
```
|
||
Expected: `OK SKILL.md decision half (… lines)`
|
||
|
||
- [ ] **Step 3: Commit**
|
||
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
git add custom-skills/35-seo-signal-validation/SKILL.md
|
||
git commit -m "feat(skill): seo-signal-validation SKILL.md decision half (L3-L4, verdict, output)"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: `gsc_signal_delta.py` helper + tests (TDD)
|
||
|
||
**Files:**
|
||
- Create test: `custom-skills/35-seo-signal-validation/code/scripts/test_gsc_signal_delta.py`
|
||
- Create: `custom-skills/35-seo-signal-validation/code/scripts/gsc_signal_delta.py`
|
||
- Create: `custom-skills/35-seo-signal-validation/code/scripts/requirements.txt`
|
||
- Create: `custom-skills/35-seo-signal-validation/code/CLAUDE.md`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing at runtime.
|
||
- Produces: `compute_delta(recent: list[dict], prior: list[dict], recent_days: int, prior_days: int, claim_term: str|None=None, top_n: int=10) -> dict` and `load_gsc(path: str) -> list[dict]`; CLI `python3 gsc_signal_delta.py --recent --prior --recent-days --prior-days [--claim-term] [--top-n]`. Output dict keys: `site_totals`, `top_gainers`, `top_decliners`, `claim_term`, `verdict_hint`.
|
||
|
||
- [ ] **Step 1: Write the failing test**
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/scripts/test_gsc_signal_delta.py`:
|
||
|
||
```python
|
||
#!/usr/bin/env python3
|
||
"""Tests for gsc_signal_delta. Run: `python3 test_gsc_signal_delta.py`
|
||
(also pytest-compatible). Stdlib only."""
|
||
import sys
|
||
from pathlib import Path
|
||
sys.path.insert(0, str(Path(__file__).parent))
|
||
from gsc_signal_delta import compute_delta # noqa: E402
|
||
|
||
# Genesis fixture: JHR "호텔" — flat head term, growth all brand (2026-06 case).
|
||
RECENT = [
|
||
{"query": "호텔", "clicks": 5, "impressions": 572, "position": 11.6},
|
||
{"query": "grand josun busan", "clicks": 250, "impressions": 4000, "position": 1.2},
|
||
{"query": "조선호텔", "clicks": 300, "impressions": 6000, "position": 1.1},
|
||
]
|
||
PRIOR = [
|
||
{"query": "호텔", "clicks": 9, "impressions": 371, "position": 18.1},
|
||
{"query": "grand josun busan", "clicks": 49, "impressions": 1500, "position": 3.4},
|
||
{"query": "조선호텔", "clicks": 150, "impressions": 5000, "position": 1.3},
|
||
]
|
||
|
||
|
||
def test_claim_term_flagged_artifact():
|
||
out = compute_delta(RECENT, PRIOR, 28, 30, claim_term="호텔")
|
||
ct = out["claim_term"]
|
||
assert ct["found"] is True
|
||
assert ct["in_top_movers"] is False
|
||
assert ct["click_share_pct"] < 1.0
|
||
assert "ARTIFACT" in out["verdict_hint"]
|
||
|
||
|
||
def test_top_gainer_is_brand_term():
|
||
out = compute_delta(RECENT, PRIOR, 28, 30, claim_term="호텔")
|
||
assert out["top_gainers"][0]["query"] == "grand josun busan"
|
||
assert out["top_gainers"][0]["delta_clicks"] == 201
|
||
|
||
|
||
def test_day_normalization():
|
||
out = compute_delta(RECENT, PRIOR, 28, 30)
|
||
assert out["site_totals"]["recent"]["clicks_per_day"] == 19.82 # 555/28
|
||
assert out["site_totals"]["prior"]["clicks_per_day"] == 6.93 # 208/30
|
||
|
||
|
||
def test_positive_days_required():
|
||
try:
|
||
compute_delta(RECENT, PRIOR, 0, 30)
|
||
except ValueError:
|
||
return
|
||
raise AssertionError("expected ValueError for non-positive days")
|
||
|
||
|
||
def _run():
|
||
fns = [v for k, v in sorted(globals().items()) if k.startswith("test_")]
|
||
for fn in fns:
|
||
fn(); print(f"PASS {fn.__name__}")
|
||
print(f"\n{len(fns)} passed")
|
||
|
||
|
||
if __name__ == "__main__":
|
||
_run()
|
||
```
|
||
|
||
- [ ] **Step 2: Run test to verify it fails**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||
python3 test_gsc_signal_delta.py
|
||
```
|
||
Expected: FAIL — `ModuleNotFoundError: No module named 'gsc_signal_delta'`
|
||
|
||
- [ ] **Step 3: Write the implementation**
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/scripts/gsc_signal_delta.py`:
|
||
|
||
```python
|
||
#!/usr/bin/env python3
|
||
"""Day-normalized GSC query delta + mover ranking for signal validation.
|
||
|
||
Reads two GSC query exports (recent, prior) — JSON list or TSV with a header row
|
||
containing query / clicks / impressions / position — and reports day-normalized
|
||
site totals, top gainers/decliners, and whether a claimed term is a real mover.
|
||
This is the deterministic L1/L4 core of the 35-seo-signal-validation skill.
|
||
"""
|
||
from __future__ import annotations
|
||
import argparse
|
||
import json
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
|
||
def _norm_row(r: dict) -> dict:
|
||
def num(*keys, default=0.0):
|
||
for k in keys:
|
||
if k in r and r[k] not in (None, ""):
|
||
try:
|
||
return float(str(r[k]).replace(",", ""))
|
||
except ValueError:
|
||
pass
|
||
return default
|
||
query = (r.get("query") or r.get("term") or "")
|
||
if isinstance(r.get("keys"), list) and r["keys"]:
|
||
query = str(r["keys"][0])
|
||
return {
|
||
"query": str(query).strip(),
|
||
"clicks": num("clicks"),
|
||
"impressions": num("impressions", "impr"),
|
||
"position": num("position", "pos", default=0.0),
|
||
}
|
||
|
||
|
||
def load_gsc(path: str) -> list[dict]:
|
||
"""Parse a GSC export (JSON list/{rows:[...]} or TSV-with-header)."""
|
||
text = Path(path).read_text(encoding="utf-8").strip()
|
||
if not text:
|
||
return []
|
||
if text[0] in "[{":
|
||
data = json.loads(text)
|
||
if isinstance(data, dict):
|
||
data = data.get("rows", [])
|
||
return [_norm_row(r) for r in data]
|
||
lines = text.splitlines()
|
||
header = [h.strip().lower() for h in lines[0].split("\t")]
|
||
rows = []
|
||
for line in lines[1:]:
|
||
if line.strip():
|
||
rows.append(_norm_row(dict(zip(header, line.split("\t")))))
|
||
return rows
|
||
|
||
|
||
def _by_query(rows: list[dict]) -> dict:
|
||
return {r["query"]: r for r in rows if r["query"]}
|
||
|
||
|
||
def compute_delta(recent, prior, recent_days, prior_days,
|
||
claim_term=None, top_n=10) -> dict:
|
||
if recent_days <= 0 or prior_days <= 0:
|
||
raise ValueError("recent_days and prior_days must be positive")
|
||
r_by, p_by = _by_query(recent), _by_query(prior)
|
||
|
||
def totals(rows):
|
||
return {"clicks": sum(r["clicks"] for r in rows),
|
||
"impressions": sum(r["impressions"] for r in rows)}
|
||
rt, pt = totals(recent), totals(prior)
|
||
|
||
def per_day(total, days):
|
||
return round(total / days, 2)
|
||
|
||
def pct(new, old):
|
||
return round((new - old) / old * 100, 1) if old else None
|
||
|
||
r_cpd, p_cpd = per_day(rt["clicks"], recent_days), per_day(pt["clicks"], prior_days)
|
||
r_ipd, p_ipd = per_day(rt["impressions"], recent_days), per_day(pt["impressions"], prior_days)
|
||
|
||
deltas = []
|
||
for q in set(r_by) | set(p_by):
|
||
rc = r_by.get(q, {}).get("clicks", 0.0)
|
||
pc = p_by.get(q, {}).get("clicks", 0.0)
|
||
deltas.append({"query": q, "delta_clicks": rc - pc,
|
||
"recent_clicks": rc, "prior_clicks": pc})
|
||
deltas.sort(key=lambda d: d["delta_clicks"], reverse=True)
|
||
gainers = [d for d in deltas if d["delta_clicks"] > 0][:top_n]
|
||
decliners = sorted([d for d in deltas if d["delta_clicks"] < 0],
|
||
key=lambda d: d["delta_clicks"])[:top_n]
|
||
|
||
out = {
|
||
"site_totals": {
|
||
"recent": {**rt, "clicks_per_day": r_cpd,
|
||
"impressions_per_day": r_ipd, "days": recent_days},
|
||
"prior": {**pt, "clicks_per_day": p_cpd,
|
||
"impressions_per_day": p_ipd, "days": prior_days},
|
||
"clicks_per_day_pct": pct(r_cpd, p_cpd),
|
||
"impressions_per_day_pct": pct(r_ipd, p_ipd),
|
||
},
|
||
"top_gainers": gainers,
|
||
"top_decliners": decliners,
|
||
"claim_term": None,
|
||
"verdict_hint": None,
|
||
}
|
||
|
||
if claim_term:
|
||
gainer_terms = {g["query"] for g in gainers}
|
||
rc, pc = r_by.get(claim_term, {}), p_by.get(claim_term, {})
|
||
in_movers = claim_term in gainer_terms
|
||
share = (rc.get("clicks", 0.0) / rt["clicks"] * 100) if rt["clicks"] else 0.0
|
||
out["claim_term"] = {
|
||
"term": claim_term, "found": bool(rc or pc),
|
||
"recent": {"clicks": rc.get("clicks", 0.0),
|
||
"impressions": rc.get("impressions", 0.0),
|
||
"position": rc.get("position")},
|
||
"prior": {"clicks": pc.get("clicks", 0.0),
|
||
"impressions": pc.get("impressions", 0.0),
|
||
"position": pc.get("position")},
|
||
"in_top_movers": in_movers,
|
||
"click_share_pct": round(share, 2),
|
||
}
|
||
if not in_movers and share < 1.0:
|
||
out["verdict_hint"] = (
|
||
f"'{claim_term}' contributes {share:.2f}% of recent clicks and is "
|
||
f"absent from top movers -> claimed impact likely ARTIFACT; real "
|
||
f"movement is elsewhere (see top_gainers).")
|
||
elif in_movers:
|
||
out["verdict_hint"] = (
|
||
f"'{claim_term}' is among top movers -> claim plausibly CONFIRMED/"
|
||
f"PARTIAL; corroborate with live SERP + entity layer.")
|
||
else:
|
||
out["verdict_hint"] = (
|
||
f"'{claim_term}' has non-trivial share ({share:.2f}%) but is not a "
|
||
f"top mover -> PARTIAL; inspect attribution.")
|
||
return out
|
||
|
||
|
||
def main(argv=None):
|
||
ap = argparse.ArgumentParser(description="GSC signal delta for signal validation")
|
||
ap.add_argument("--recent", required=True)
|
||
ap.add_argument("--prior", required=True)
|
||
ap.add_argument("--recent-days", type=int, required=True)
|
||
ap.add_argument("--prior-days", type=int, required=True)
|
||
ap.add_argument("--claim-term", default=None)
|
||
ap.add_argument("--top-n", type=int, default=10)
|
||
a = ap.parse_args(argv)
|
||
out = compute_delta(load_gsc(a.recent), load_gsc(a.prior),
|
||
a.recent_days, a.prior_days, a.claim_term, a.top_n)
|
||
json.dump(out, sys.stdout, ensure_ascii=False, indent=2)
|
||
sys.stdout.write("\n")
|
||
return 0
|
||
|
||
|
||
if __name__ == "__main__":
|
||
raise SystemExit(main())
|
||
```
|
||
|
||
- [ ] **Step 4: Run tests to verify they pass**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||
python3 test_gsc_signal_delta.py
|
||
```
|
||
Expected: `PASS test_claim_term_flagged_artifact` … `4 passed`
|
||
|
||
- [ ] **Step 5: Create `requirements.txt` and `code/CLAUDE.md`**
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/scripts/requirements.txt`:
|
||
```text
|
||
# gsc_signal_delta.py uses the Python 3 standard library only — no deps.
|
||
```
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/CLAUDE.md`:
|
||
```markdown
|
||
# seo-signal-validation — code environment notes
|
||
|
||
## Helper: scripts/gsc_signal_delta.py
|
||
Deterministic L1/L4 GSC delta. Feed it two saved GSC query exports (recent,
|
||
prior) as JSON or TSV (columns: query, clicks, impressions, position).
|
||
|
||
```bash
|
||
python3 scripts/gsc_signal_delta.py \
|
||
--recent recent.tsv --prior prior.tsv \
|
||
--recent-days 28 --prior-days 30 --claim-term "호텔"
|
||
```
|
||
Returns day-normalized site totals, top gainers/decliners, and a `verdict_hint`
|
||
(heuristic only — the final verdict is the skill's job, after L2/L3).
|
||
|
||
## Getting the exports
|
||
`mcp__dda__gsc_fetch_performance` (property pinned per workspace, e.g. JHR
|
||
`sc-domain:josunhotel.com`) → save the query-dimension rows to a file → run the
|
||
script. GSC anonymizes ~43% of query clicks; the disclosed subset ≠ the whole.
|
||
|
||
## Env / access
|
||
- `GOOGLE_KG_API_KEY` for `mcp__ourseo__search_knowledge_graph` (L3).
|
||
- GSC/GA4 only exist for first-party properties — third-party entities skip L1.
|
||
- Never crawl/audit Marriott for JHR (sameAs only).
|
||
```
|
||
|
||
- [ ] **Step 6: Commit**
|
||
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
git add custom-skills/35-seo-signal-validation/code
|
||
git commit -m "feat(skill): gsc_signal_delta helper + tests + code notes"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: Register in marketplace + reconcile DESIGN.md structure
|
||
|
||
**Files:**
|
||
- Modify: `.claude-plugin/marketplace.json` (add to `ourdigital-seo` → `skills`)
|
||
- Modify: `custom-skills/35-seo-signal-validation/DESIGN.md:§7` (replace `references/` layout with actual `code/` layout; mark Code-only)
|
||
|
||
**Interfaces:**
|
||
- Consumes: the skill folder from Tasks 1–3.
|
||
- Produces: a registered, discoverable skill; a spec whose §7 matches the built structure.
|
||
|
||
- [ ] **Step 1: Add the skill path to the manifest**
|
||
|
||
In `.claude-plugin/marketplace.json`, inside the `ourdigital-seo` plugin's
|
||
`skills` array, add (keep numeric order; insert after the `34-seo-reporting-dashboard` entry):
|
||
```json
|
||
"./custom-skills/35-seo-signal-validation",
|
||
```
|
||
|
||
- [ ] **Step 2: Verify the manifest still parses and contains the entry**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
python3 - <<'PY'
|
||
import json, pathlib
|
||
m = json.loads(pathlib.Path(".claude-plugin/marketplace.json").read_text())
|
||
seo = next(p for p in m["plugins"] if p["name"] == "ourdigital-seo")
|
||
assert "./custom-skills/35-seo-signal-validation" in seo["skills"], "not registered"
|
||
print("OK manifest valid + skill registered")
|
||
PY
|
||
```
|
||
Expected: `OK manifest valid + skill registered`
|
||
|
||
- [ ] **Step 3: Reconcile `DESIGN.md` §7 with the real structure**
|
||
|
||
In `custom-skills/35-seo-signal-validation/DESIGN.md`, replace the §7 "Repo
|
||
layout & conventions" code block (the `references/...` sketch) with:
|
||
```text
|
||
35-seo-signal-validation/
|
||
SKILL.md self-contained: classification, 4-layer cascade,
|
||
5 KG checks, 4-way verdict, skepticism rules, output
|
||
DESIGN.md PLAN.md spec + plan (live with the skill; no new top-level dir)
|
||
code/
|
||
CLAUDE.md code-environment notes (env, export→script flow)
|
||
scripts/
|
||
gsc_signal_delta.py deterministic L1/L4 GSC delta + mover ranking
|
||
test_gsc_signal_delta.py
|
||
requirements.txt (stdlib only)
|
||
```
|
||
And add one line under it: `Target environment: Claude Code only (no desktop/ variant — matches precedent 95/96). Registered in .claude-plugin/marketplace.json under ourdigital-seo.`
|
||
|
||
- [ ] **Step 4: Verify the spec no longer references the old layout**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
python3 - <<'PY'
|
||
import pathlib
|
||
t = pathlib.Path("custom-skills/35-seo-signal-validation/DESIGN.md").read_text(encoding="utf-8")
|
||
assert "references/\n evidence-cascade.md" not in t, "old layout still present"
|
||
assert "gsc_signal_delta.py" in t and "marketplace.json" in t, "structure not reconciled"
|
||
print("OK DESIGN.md §7 reconciled")
|
||
PY
|
||
```
|
||
Expected: `OK DESIGN.md §7 reconciled`
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
git add .claude-plugin/marketplace.json custom-skills/35-seo-signal-validation/DESIGN.md
|
||
git commit -m "feat(skill): register seo-signal-validation in marketplace; reconcile spec layout"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: Smoke test — genesis case end-to-end + consistency gate
|
||
|
||
**Files:**
|
||
- Create: `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-recent.tsv`
|
||
- Create: `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-prior.tsv`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `gsc_signal_delta.py` CLI (Task 3) and the full `SKILL.md` (Tasks 1–2).
|
||
- Produces: a reproducible CLI smoke run proving the genesis "호텔" case yields an ARTIFACT-leaning hint; a final spec↔skill consistency check.
|
||
|
||
- [ ] **Step 1: Create the genesis fixtures (TSV)**
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-recent.tsv`:
|
||
```text
|
||
query clicks impressions position
|
||
호텔 5 572 11.6
|
||
grand josun busan 250 4000 1.2
|
||
조선호텔 300 6000 1.1
|
||
```
|
||
|
||
Create `custom-skills/35-seo-signal-validation/code/scripts/fixtures/jhr-hotel-prior.tsv`:
|
||
```text
|
||
query clicks impressions position
|
||
호텔 9 371 18.1
|
||
grand josun busan 49 1500 3.4
|
||
조선호텔 150 5000 1.3
|
||
```
|
||
|
||
- [ ] **Step 2: Run the CLI end-to-end and verify the verdict hint**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills/custom-skills/35-seo-signal-validation/code/scripts
|
||
python3 gsc_signal_delta.py --recent fixtures/jhr-hotel-recent.tsv \
|
||
--prior fixtures/jhr-hotel-prior.tsv --recent-days 28 --prior-days 30 \
|
||
--claim-term "호텔" | python3 - <<'PY'
|
||
import json, sys
|
||
o = json.load(sys.stdin)
|
||
assert o["claim_term"]["in_top_movers"] is False
|
||
assert o["claim_term"]["click_share_pct"] < 1.0
|
||
assert "ARTIFACT" in o["verdict_hint"]
|
||
assert o["top_gainers"][0]["query"] == "grand josun busan"
|
||
print("OK smoke: 호텔 → ARTIFACT-leaning; top mover = brand term")
|
||
PY
|
||
```
|
||
Expected: `OK smoke: 호텔 → ARTIFACT-leaning; top mover = brand term`
|
||
|
||
- [ ] **Step 3: Consistency gate — every spec default maps to skill content**
|
||
|
||
Run:
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
python3 - <<'PY'
|
||
import pathlib
|
||
sk = pathlib.Path("custom-skills/35-seo-signal-validation/SKILL.md").read_text(encoding="utf-8")
|
||
# Default 1: 5 KG checks Default 2: 4 verdicts Default 3: output triple
|
||
for s in ["Google KG API", "Wikidata", "Knowledge Panel", "sameAs", "지식iN"]:
|
||
assert s in sk, f"KG check missing: {s}"
|
||
for s in ["CONFIRMED", "PARTIAL", "ARTIFACT", "INCONCLUSIVE"]:
|
||
assert s in sk, f"verdict missing: {s}"
|
||
for s in ["notion-writer", "evidence ledger", "client-safe narrative"]:
|
||
assert s.lower() in sk.lower(), f"output element missing: {s}"
|
||
# Default 5: triggers (KR + EN)
|
||
assert "신호 검증" in sk and "validate serp signal" in sk, "triggers missing"
|
||
print("OK all five approved defaults present in SKILL.md")
|
||
PY
|
||
```
|
||
Expected: `OK all five approved defaults present in SKILL.md`
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
cd ~/Project/our-claude-skills
|
||
git add custom-skills/35-seo-signal-validation/code/scripts/fixtures
|
||
git commit -m "test(skill): genesis 호텔 smoke fixtures + end-to-end ARTIFACT check"
|
||
```
|
||
|
||
---
|
||
|
||
## Self-Review
|
||
|
||
**1. Spec coverage** (each DESIGN.md section → task):
|
||
- §1 Purpose, §2 Boundary → Task 1 (SKILL.md Purpose/boundary). ✓
|
||
- §3 Engine (entity classification, L1–L4, short-circuit) → Tasks 1 (L1–L2) + 2 (L3–L4); L1/L4 delta computation → Task 3 script. ✓
|
||
- §4 Modes → Task 1 (Step 0 mode dispatch). ✓
|
||
- §5 Verdict + skepticism + confidence cap → Task 2. ✓
|
||
- §6 Output → Task 2. ✓
|
||
- §7 Repo layout → corrected in Task 4 (was wrong in spec); registration in Task 4. ✓
|
||
- §8 Non-goals → Task 2. ✓
|
||
- §9 Future options → intentionally not implemented (YAGNI). ✓
|
||
- Genesis verification → Task 5 smoke. ✓
|
||
|
||
**2. Placeholder scan:** No TBD/TODO; all code blocks complete; every test shows real assertions; commands show expected output. ✓
|
||
|
||
**3. Type consistency:** `compute_delta(recent, prior, recent_days, prior_days, claim_term=None, top_n=10)` and `load_gsc(path)` are referenced identically in Task 3 (definition + test) and Task 5 (CLI). Output keys (`site_totals`, `top_gainers`, `claim_term.in_top_movers`, `claim_term.click_share_pct`, `verdict_hint`) match across the implementation, the tests, and both smoke checks. Day-normalization fixtures (555/28=19.82, 208/30=6.93; gainer delta 201) are arithmetically consistent. ✓
|
||
|
||
**No gaps found.**
|