142 lines
7.2 KiB
Markdown
142 lines
7.2 KiB
Markdown
---
|
||
name: seo-signal-validation
|
||
description: |
|
||
Validate whether a claimed SERP / Knowledge-Graph movement for a (term, entity)
|
||
is real, misattributed, an artifact, or unprovable — before reporting impact.
|
||
Triggers: validate serp signal, is this ranking real, prove SEO impact,
|
||
SEMrush surge real, signal validation, real impact check,
|
||
신호 검증, 순위 변화 진짜, 오가닉 급증 검증, 임팩트 검증.
|
||
---
|
||
|
||
# SEO Signal Validation
|
||
|
||
## Purpose
|
||
|
||
Given a `(term/intent, entity)` pair — and optionally a **claim** (a third-party
|
||
tool's reported movement) or a **baseline** (a prior state) — return an
|
||
evidence-backed verdict on whether SERP and Knowledge-Graph impact is real.
|
||
Built because modeled third-party signals (SEMrush/Ahrefs estimated organic
|
||
traffic, position snapshots) are easy to over-trust. This skill makes the
|
||
measured → live → entity → attribution cascade a single repeatable procedure
|
||
ending in a defensible verdict and a client-safe narrative.
|
||
|
||
## When to use (boundary)
|
||
|
||
This is the **conductor**, not an instrument. It sequences and synthesizes the
|
||
three measurement skills — it does not duplicate them.
|
||
|
||
| Use instead | When |
|
||
|---|---|
|
||
| `20-seo-serp-analysis` | You only need SERP composition / features |
|
||
| `21-seo-position-tracking` | You only need rank over time |
|
||
| `28-seo-knowledge-graph` | You only need an entity-presence audit |
|
||
| **this skill** | You must adjudicate whether a *claimed movement* is real across layers |
|
||
|
||
## Step 0 — Classify entity + pick mode
|
||
|
||
1. **Entity ownership** (gates which layers exist):
|
||
- **First-party** — a site/property you own or have GSC/GA4 access to (e.g. JHR
|
||
`sc-domain:josunhotel.com`, GA4 `258308769`) → **L1 measured available**.
|
||
- **Third-party** — a competitor brand or a person you do not control →
|
||
**L1 unavailable**; lean on L2 + L3 + clearly-tiered estimates; apply the
|
||
confidence cap (see Verdict). If unclear, ask once.
|
||
2. **Mode** (thin wrappers over the same cascade):
|
||
- `adjudicate(claim)` — a 3rd-party tool reports a move; confirm/refute.
|
||
- `prove(baseline)` — after our change; before/after from GSC/GA4 history.
|
||
- `snapshot()` — no claim; "where do we really stand."
|
||
|
||
## The validation loop (cost-ordered cascade, short-circuiting)
|
||
|
||
Run cheapest-first; stop early when a layer is already decisive.
|
||
|
||
### L1 — Measured (first-party ground truth) → via `21-seo-position-tracking`
|
||
|
||
- **GSC** `mcp__dda__gsc_fetch_performance`: the term at **query level** (exact)
|
||
AND **site-wide**, for **recent vs prior** windows. Pull clicks / impressions /
|
||
position / CTR. **Day-normalize** (compare windows differ in calendar-day count).
|
||
Note **~43% query-level anonymization** — the disclosed subset ≠ the whole.
|
||
- **GA4** `mcp__dda__ga4_run_report`: `Organic Search` sessions monthly trend
|
||
(dims `yearMonth` + `sessionDefaultChannelGroup`, metric `sessions`). GA4
|
||
includes Naver + all engines — use it to test whether a "surge" exceeds normal
|
||
month-to-month variance.
|
||
- **Compute deltas with the helper** (deterministic, avoids ad-hoc parsing):
|
||
save each GSC pull, then run
|
||
`python3 code/scripts/gsc_signal_delta.py --recent <recent.tsv> --prior <prior.tsv> --recent-days N --prior-days M --claim-term "<term>"`.
|
||
It returns day-normalized site totals, top gainers/decliners, and whether the
|
||
claimed term is among the real movers.
|
||
- **SHORT-CIRCUIT:** if the claimed keyword has trivial clicks and a real
|
||
position nowhere near the claim → **ARTIFACT**; stop unless the caller wants
|
||
the full picture.
|
||
|
||
### L2 — Live SERP (3rd-party measured, point-in-time) → via `20-seo-serp-analysis`
|
||
|
||
- **Geo-correct Google render** via `claude-in-chrome` (`navigate` → `read_page`):
|
||
force `gl`/`hl` + correct geo, `pws=0`; **decline precise-location prompts**.
|
||
Confirm whether the domain actually holds the claimed position; capture the
|
||
feature landscape (ads, local map-pack, PAA, knowledge panel) that explains why
|
||
a brand site can't own a head term.
|
||
- **Cheap rank spot-check**: `mcp__ourseo__check_serp(keyword, domain)`.
|
||
- **[KR market]** Naver SERP composition: `our research naver serp` (blog / cafe /
|
||
지식iN / Smart Store / brand zone) — Semrush/Ahrefs don't model Naver.
|
||
|
||
### L3 — Entity / Knowledge Graph → via `28-seo-knowledge-graph`
|
||
|
||
A real impact event should leave corroborating traces in the entity layer, not
|
||
just a rank number. Five checks:
|
||
|
||
1. **Google KG API** entity match + `resultScore` —
|
||
`mcp__ourseo__search_knowledge_graph(query)` (uses `GOOGLE_KG_API_KEY`).
|
||
2. **Wikidata** QID presence + key claims — **verify the QID against
|
||
`Special:EntityData/{Q}.json` labels before trusting it** (false-match guard:
|
||
Q109455878 = office tower ≠ hotel; Q490787 = Shinsegae Inc. ≠ Group).
|
||
3. **Knowledge Panel** presence/attributes on the live entity-name SERP (Chrome).
|
||
4. **sameAs** consistency on the entity's `Organization`/`Person` JSON-LD.
|
||
5. **[KR]** Naver 백과사전 / 지식iN presence.
|
||
|
||
`mcp__ourseo__monitor_brand` supplements with brand-mention / brand-SERP ownership.
|
||
|
||
### L4 — Attribution synthesis
|
||
|
||
Cross-check: does the **measured delta (L1)** corroborate the **live reality
|
||
(L2)**, and does the **entity layer (L3)** move consistently? The query-clicks
|
||
delta names the true drivers (brand/seasonal vs the claimed term).
|
||
|
||
## Verdict
|
||
|
||
| Verdict | Condition |
|
||
|---|---|
|
||
| **CONFIRMED** | Measured + live + (where relevant) entity all corroborate movement attributable to the term/intent |
|
||
| **PARTIAL** | Real movement, but misattributed, or only some layers agree |
|
||
| **ARTIFACT** | Modeling/snapshot artifact — measured + live reality don't support it |
|
||
| **INCONCLUSIVE** | Insufficient data (query anonymized, GSC lag, no entity baseline, third-party entity with no measured access) — name what's missing + how to resolve |
|
||
|
||
**Confidence cap:** third-party entities (no L1) cannot reach CONFIRMED on traffic
|
||
claims — at most PARTIAL; ARTIFACT only when live+entity clearly contradict.
|
||
|
||
Every verdict ships an **evidence ledger** (per layer: finding + data-trust tier +
|
||
corroborates/contradicts) and a **client-safe narrative** (the defensible story).
|
||
|
||
## Standing skepticism rules
|
||
|
||
- Estimated organic traffic = **smoke-detector, not scale** (Σ est-volume × position-CTR curve).
|
||
- **Head-term over-fire**: one high-volume keyword at an estimated high rank inflates the whole modeled number.
|
||
- **KR Naver blind spot**: Semrush models Google only; misses much of Korean organic.
|
||
- **Single-geo/device snapshot** diverges from GSC's national average.
|
||
- **Trust hierarchy**: 1st-party measured > 3rd-party measured > 3rd-party modeled.
|
||
|
||
## Output
|
||
|
||
- **Always**: inline report — verdict + evidence ledger + client-safe narrative +
|
||
"what would raise confidence."
|
||
- **Optional**: archive to Notion *Working with AI DB* (`data_source_id
|
||
f8f19ede-32bd-43ac-9f60-0651f6f40afe`) via the **notion-writer script** (never
|
||
Notion MCP write). Type=Memo/Research, Topic=SEO, Account Code as relevant.
|
||
- **Optional**: if a new generalizable gotcha emerges, append a memory entry to
|
||
the active workspace's memory dir.
|
||
|
||
## Non-goals
|
||
|
||
No cron/scheduler, no snapshot DB, no new directories. Does not replace the three
|
||
instrument skills. Returns INCONCLUSIVE rather than fabricating when data is thin.
|
||
**Never crawls/audits Marriott for JHR** (sameAs only).
|