feat(ga-agent): Add GA Agent project with decomposed architecture
Create workspace for building Google Analytics Claude Skill with: - 5 independent components (MCP setup, skill, dimension explorer, Slack reporter, realtime watcher) - Comprehensive project plan and documentation - Step-by-step setup guides for each component Components: 1. MCP Setup - GA4 + BigQuery MCP server installation 2. GA Agent Skill - Core Claude Skill for interactive analysis 3. Dimension Explorer - Validate dims/metrics with explanations 4. Slack Reporter - Automated reports to Slack (P2) 5. Realtime Watcher - Real-time monitoring (deferred) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
26
ga-agent-project/.gitignore
vendored
Normal file
26
ga-agent-project/.gitignore
vendored
Normal file
@@ -0,0 +1,26 @@
|
||||
# Credentials - NEVER commit these
|
||||
config/*.json
|
||||
config/*.key
|
||||
*.credentials.json
|
||||
service-account*.json
|
||||
|
||||
# Environment
|
||||
.env
|
||||
.env.local
|
||||
*.env
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# Node
|
||||
node_modules/
|
||||
|
||||
# IDE
|
||||
.idea/
|
||||
.vscode/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
143
ga-agent-project/01-mcp-setup/README.md
Normal file
143
ga-agent-project/01-mcp-setup/README.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Component 1: MCP Setup
|
||||
|
||||
**Type:** Infrastructure
|
||||
**Priority:** P0
|
||||
**Status:** Not Started
|
||||
|
||||
## Goal
|
||||
|
||||
Install and configure GA4 + BigQuery MCP servers for Claude Code.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Google Cloud account
|
||||
- GA4 property access
|
||||
- `gcloud` CLI installed
|
||||
|
||||
## Setup Steps
|
||||
|
||||
### Step 1: Google Cloud Project
|
||||
|
||||
```bash
|
||||
# Authenticate
|
||||
gcloud auth login
|
||||
|
||||
# Set project (create if needed)
|
||||
gcloud config set project YOUR_PROJECT_ID
|
||||
|
||||
# Enable APIs
|
||||
gcloud services enable \
|
||||
analyticsdata.googleapis.com \
|
||||
analyticsadmin.googleapis.com \
|
||||
bigquery.googleapis.com
|
||||
```
|
||||
|
||||
### Step 2: Service Account
|
||||
|
||||
```bash
|
||||
# Create service account
|
||||
gcloud iam service-accounts create ga-mcp-agent \
|
||||
--display-name="GA MCP Agent"
|
||||
|
||||
# Get email
|
||||
SA_EMAIL="ga-mcp-agent@YOUR_PROJECT_ID.iam.gserviceaccount.com"
|
||||
|
||||
# Grant BigQuery roles
|
||||
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
|
||||
--member="serviceAccount:$SA_EMAIL" \
|
||||
--role="roles/bigquery.dataViewer"
|
||||
|
||||
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
|
||||
--member="serviceAccount:$SA_EMAIL" \
|
||||
--role="roles/bigquery.jobUser"
|
||||
|
||||
# Download key
|
||||
gcloud iam service-accounts keys create \
|
||||
../config/service-account.json \
|
||||
--iam-account=$SA_EMAIL
|
||||
```
|
||||
|
||||
### Step 3: GA4 Property Access
|
||||
|
||||
1. Go to [GA4 Admin](https://analytics.google.com/)
|
||||
2. Property → Property Access Management
|
||||
3. Add user: `ga-mcp-agent@YOUR_PROJECT_ID.iam.gserviceaccount.com`
|
||||
4. Role: Viewer
|
||||
|
||||
### Step 4: Install GA4 MCP Server
|
||||
|
||||
```bash
|
||||
# Clone official server
|
||||
git clone https://github.com/googleanalytics/google-analytics-mcp.git
|
||||
cd google-analytics-mcp
|
||||
|
||||
# Setup Python environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -e .
|
||||
|
||||
# Test
|
||||
export GOOGLE_APPLICATION_CREDENTIALS="../config/service-account.json"
|
||||
python -m google_analytics_mcp --help
|
||||
```
|
||||
|
||||
### Step 5: Install BigQuery MCP Server
|
||||
|
||||
```bash
|
||||
# Test with npx (no install needed)
|
||||
npx -y @ergut/mcp-bigquery-server \
|
||||
--project-id YOUR_PROJECT_ID \
|
||||
--location us-central1 \
|
||||
--key-file ../config/service-account.json
|
||||
```
|
||||
|
||||
### Step 6: Configure Claude Code
|
||||
|
||||
Add to `~/.claude/mcp_servers.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"google-analytics": {
|
||||
"command": "python",
|
||||
"args": ["-m", "google_analytics_mcp"],
|
||||
"cwd": "/path/to/google-analytics-mcp",
|
||||
"env": {
|
||||
"GOOGLE_APPLICATION_CREDENTIALS": "/path/to/service-account.json"
|
||||
}
|
||||
},
|
||||
"bigquery": {
|
||||
"command": "npx",
|
||||
"args": [
|
||||
"-y",
|
||||
"@ergut/mcp-bigquery-server",
|
||||
"--project-id", "YOUR_PROJECT_ID",
|
||||
"--location", "us-central1",
|
||||
"--key-file", "/path/to/service-account.json"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 7: Verify
|
||||
|
||||
```bash
|
||||
# Restart Claude Code, then:
|
||||
mcp-cli servers
|
||||
# Should show: google-analytics, bigquery
|
||||
|
||||
mcp-cli tools google-analytics
|
||||
mcp-cli tools bigquery
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] GCP project configured
|
||||
- [ ] APIs enabled
|
||||
- [ ] Service account created
|
||||
- [ ] GA4 access granted
|
||||
- [ ] GA4 MCP installed
|
||||
- [ ] BigQuery MCP installed
|
||||
- [ ] Claude Code configured
|
||||
- [ ] Connection verified
|
||||
117
ga-agent-project/02-ga-agent-skill/README.md
Normal file
117
ga-agent-project/02-ga-agent-skill/README.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Component 2: GA Agent Skill
|
||||
|
||||
**Type:** Claude Skill
|
||||
**Priority:** P0
|
||||
**Status:** Not Started
|
||||
**Final Location:** `ourdigital-custom-skills/15-ourdigital-ga-agent/`
|
||||
|
||||
## Goal
|
||||
|
||||
Interactive GA4 analysis and reporting skill for Claude Code.
|
||||
|
||||
## Features
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| Traffic Analysis | Users, sessions, pageviews with trends |
|
||||
| Period Comparison | WoW, MoM, YoY comparisons |
|
||||
| Top Content | Pages, sources, campaigns |
|
||||
| Report Generation | HTML reports |
|
||||
| BigQuery Queries | Complex analysis on exported data |
|
||||
|
||||
## Triggers
|
||||
|
||||
**English:**
|
||||
- "Analyze GA4 traffic"
|
||||
- "Compare last week vs this week"
|
||||
- "Generate traffic report"
|
||||
- "Top landing pages"
|
||||
- "Query BigQuery for GA data"
|
||||
|
||||
**Korean:**
|
||||
- "GA4 트래픽 분석"
|
||||
- "지난주 대비 비교"
|
||||
- "트래픽 리포트 생성"
|
||||
- "인기 랜딩 페이지"
|
||||
- "BigQuery GA 데이터 조회"
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
15-ourdigital-ga-agent/
|
||||
├── SKILL.md # Skill definition
|
||||
├── scripts/
|
||||
│ ├── analyze_traffic.py # Traffic analysis
|
||||
│ ├── compare_periods.py # Period comparisons
|
||||
│ ├── top_content.py # Top pages/sources
|
||||
│ └── generate_report.py # HTML report generation
|
||||
├── templates/
|
||||
│ └── report.html # Report template
|
||||
├── references/
|
||||
│ └── ga4-api-reference.md # Quick API reference
|
||||
└── examples/
|
||||
└── sample-queries.md # Example usage
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
Requires Component 1 (MCP Setup) to be complete.
|
||||
|
||||
## Scripts
|
||||
|
||||
### analyze_traffic.py
|
||||
|
||||
Fetches traffic metrics for a date range:
|
||||
- Active users
|
||||
- Sessions
|
||||
- Pageviews
|
||||
- Bounce rate
|
||||
- Session duration
|
||||
|
||||
### compare_periods.py
|
||||
|
||||
Compares metrics between two periods:
|
||||
- Current vs previous period
|
||||
- Percentage changes
|
||||
- Trend indicators
|
||||
|
||||
### top_content.py
|
||||
|
||||
Lists top performing content:
|
||||
- Landing pages
|
||||
- Traffic sources
|
||||
- Campaigns
|
||||
- Countries/cities
|
||||
|
||||
### generate_report.py
|
||||
|
||||
Generates HTML report with:
|
||||
- Summary metrics
|
||||
- Charts (via Plotly)
|
||||
- Top content tables
|
||||
- Period comparison
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Work in this directory
|
||||
cd 02-ga-agent-skill
|
||||
|
||||
# Create skill structure
|
||||
mkdir -p skill/{scripts,templates,references,examples}
|
||||
|
||||
# When complete, move to final location
|
||||
mv skill ../ourdigital-custom-skills/15-ourdigital-ga-agent
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] SKILL.md created
|
||||
- [ ] analyze_traffic.py
|
||||
- [ ] compare_periods.py
|
||||
- [ ] top_content.py
|
||||
- [ ] generate_report.py
|
||||
- [ ] report.html template
|
||||
- [ ] Examples documented
|
||||
- [ ] Tested with Claude Code
|
||||
- [ ] Moved to ourdigital-custom-skills/
|
||||
154
ga-agent-project/03-dimension-explorer/README.md
Normal file
154
ga-agent-project/03-dimension-explorer/README.md
Normal file
@@ -0,0 +1,154 @@
|
||||
# Component 3: Dimension Explorer
|
||||
|
||||
**Type:** Utility (MCP Server / CLI / Reference)
|
||||
**Priority:** P1
|
||||
**Status:** Not Started
|
||||
|
||||
## Goal
|
||||
|
||||
Validate GA4 dimensions and metrics with detailed explanations.
|
||||
|
||||
## Features
|
||||
|
||||
- List all available dimensions/metrics
|
||||
- Validate if a dimension/metric exists
|
||||
- Get description, data type, category
|
||||
- Fuzzy search for typos
|
||||
- Compatibility checking
|
||||
|
||||
## Implementation Options
|
||||
|
||||
| Option | Approach | Effort |
|
||||
|--------|----------|--------|
|
||||
| A | Reference JSON in skill | Low |
|
||||
| B | CLI tool | Low |
|
||||
| C | MCP Server | Medium |
|
||||
|
||||
**Recommendation:** Start with A, upgrade to C later.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
03-dimension-explorer/
|
||||
├── README.md
|
||||
├── fetch_metadata.py # Fetch from GA4 Admin API
|
||||
├── data/
|
||||
│ ├── dimensions.json # All dimensions
|
||||
│ └── metrics.json # All metrics
|
||||
├── explorer.py # CLI tool (optional)
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
## Data Format
|
||||
|
||||
### dimensions.json
|
||||
|
||||
```json
|
||||
{
|
||||
"dimensions": [
|
||||
{
|
||||
"apiName": "sessionSource",
|
||||
"uiName": "Session source",
|
||||
"description": "The source that initiated a session",
|
||||
"category": "Traffic source",
|
||||
"deprecatedApiNames": []
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### metrics.json
|
||||
|
||||
```json
|
||||
{
|
||||
"metrics": [
|
||||
{
|
||||
"apiName": "activeUsers",
|
||||
"uiName": "Active users",
|
||||
"description": "Number of distinct users who visited",
|
||||
"category": "User",
|
||||
"type": "TYPE_INTEGER",
|
||||
"expression": ""
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## fetch_metadata.py
|
||||
|
||||
```python
|
||||
from google.analytics.admin import AnalyticsAdminServiceClient
|
||||
|
||||
def fetch_metadata(property_id: str):
|
||||
"""Fetch all dimensions and metrics for a property."""
|
||||
client = AnalyticsAdminServiceClient()
|
||||
|
||||
# Get metadata
|
||||
metadata = client.get_metadata(
|
||||
name=f"properties/{property_id}/metadata"
|
||||
)
|
||||
|
||||
dimensions = [
|
||||
{
|
||||
"apiName": d.api_name,
|
||||
"uiName": d.ui_name,
|
||||
"description": d.description,
|
||||
"category": d.category,
|
||||
}
|
||||
for d in metadata.dimensions
|
||||
]
|
||||
|
||||
metrics = [
|
||||
{
|
||||
"apiName": m.api_name,
|
||||
"uiName": m.ui_name,
|
||||
"description": m.description,
|
||||
"category": m.category,
|
||||
"type": m.type_.name,
|
||||
}
|
||||
for m in metadata.metrics
|
||||
]
|
||||
|
||||
return {"dimensions": dimensions, "metrics": metrics}
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### As Reference (Option A)
|
||||
|
||||
Include `data/dimensions.json` and `data/metrics.json` in the GA Agent skill's `references/` folder.
|
||||
|
||||
### As CLI (Option B)
|
||||
|
||||
```bash
|
||||
# Validate a dimension
|
||||
python explorer.py validate --dimension sessionSource
|
||||
|
||||
# Search for metrics
|
||||
python explorer.py search --query "user"
|
||||
|
||||
# List by category
|
||||
python explorer.py list --category "Traffic source"
|
||||
```
|
||||
|
||||
### As MCP Server (Option C)
|
||||
|
||||
```bash
|
||||
# Run server
|
||||
python server.py
|
||||
|
||||
# Claude can use tools like:
|
||||
# - validate_dimension
|
||||
# - validate_metric
|
||||
# - search_metadata
|
||||
# - list_by_category
|
||||
```
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] fetch_metadata.py created
|
||||
- [ ] Metadata fetched and saved
|
||||
- [ ] dimensions.json generated
|
||||
- [ ] metrics.json generated
|
||||
- [ ] explorer.py (optional)
|
||||
- [ ] Integrated with GA Agent skill
|
||||
160
ga-agent-project/04-slack-reporter/README.md
Normal file
160
ga-agent-project/04-slack-reporter/README.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# Component 4: Slack Reporter
|
||||
|
||||
**Type:** Standalone Service
|
||||
**Priority:** P2
|
||||
**Status:** Not Started
|
||||
|
||||
## Goal
|
||||
|
||||
Automated GA4 reports delivered to Slack channels.
|
||||
|
||||
## Features
|
||||
|
||||
| Report | Schedule | Content |
|
||||
|--------|----------|---------|
|
||||
| Daily Summary | 9:00 AM | Users, sessions, top 5 pages |
|
||||
| Weekly Digest | Monday 9 AM | WoW comparison, trends |
|
||||
| Anomaly Alert | Real-time | Traffic ±30% from baseline |
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
04-slack-reporter/
|
||||
├── README.md
|
||||
├── config.yaml # Configuration
|
||||
├── reporter.py # Main service
|
||||
├── queries/
|
||||
│ ├── daily_summary.py
|
||||
│ ├── weekly_digest.py
|
||||
│ └── anomaly_check.py
|
||||
├── templates/
|
||||
│ └── slack_blocks.py # Slack Block Kit
|
||||
├── requirements.txt
|
||||
├── Dockerfile
|
||||
└── docker-compose.yml
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### config.yaml
|
||||
|
||||
```yaml
|
||||
slack:
|
||||
bot_token: ${SLACK_BOT_TOKEN}
|
||||
default_channel: "#analytics-reports"
|
||||
|
||||
ga4:
|
||||
property_id: "123456789"
|
||||
credentials_path: "/path/to/credentials.json"
|
||||
|
||||
reports:
|
||||
daily_summary:
|
||||
enabled: true
|
||||
schedule: "0 9 * * *" # 9 AM daily
|
||||
channel: "#analytics-reports"
|
||||
|
||||
weekly_digest:
|
||||
enabled: true
|
||||
schedule: "0 9 * * 1" # 9 AM Monday
|
||||
channel: "#analytics-reports"
|
||||
|
||||
anomaly_alert:
|
||||
enabled: true
|
||||
check_interval: 3600 # Check every hour
|
||||
threshold: 0.3 # 30% deviation
|
||||
channel: "#analytics-alerts"
|
||||
```
|
||||
|
||||
## Slack App Setup
|
||||
|
||||
1. Go to [api.slack.com/apps](https://api.slack.com/apps)
|
||||
2. Create New App → From scratch
|
||||
3. Add OAuth scopes:
|
||||
- `chat:write`
|
||||
- `files:write`
|
||||
- `channels:read`
|
||||
4. Install to workspace
|
||||
5. Copy Bot Token (`xoxb-...`)
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
# requirements.txt
|
||||
google-analytics-data>=0.18.0
|
||||
google-auth>=2.23.0
|
||||
slack-sdk>=3.23.0
|
||||
apscheduler>=3.10.0
|
||||
pandas>=2.0.0
|
||||
plotly>=5.18.0
|
||||
kaleido>=0.2.1
|
||||
pyyaml>=6.0
|
||||
```
|
||||
|
||||
## Slack Message Format
|
||||
|
||||
Using Block Kit for rich formatting:
|
||||
|
||||
```python
|
||||
def daily_summary_blocks(data: dict) -> list:
|
||||
return [
|
||||
{
|
||||
"type": "header",
|
||||
"text": {"type": "plain_text", "text": "📊 Daily GA4 Summary"}
|
||||
},
|
||||
{
|
||||
"type": "section",
|
||||
"fields": [
|
||||
{"type": "mrkdwn", "text": f"*Users:* {data['users']:,}"},
|
||||
{"type": "mrkdwn", "text": f"*Sessions:* {data['sessions']:,}"},
|
||||
{"type": "mrkdwn", "text": f"*Pageviews:* {data['pageviews']:,}"},
|
||||
{"type": "mrkdwn", "text": f"*Bounce Rate:* {data['bounce_rate']:.1%}"},
|
||||
]
|
||||
},
|
||||
{"type": "divider"},
|
||||
{
|
||||
"type": "section",
|
||||
"text": {"type": "mrkdwn", "text": "*Top Pages:*\n" + data['top_pages']}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Set environment variables
|
||||
export SLACK_BOT_TOKEN=xoxb-...
|
||||
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/creds.json
|
||||
|
||||
# Run
|
||||
python reporter.py
|
||||
```
|
||||
|
||||
### Docker
|
||||
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### Cloud Options
|
||||
|
||||
- Google Cloud Run (scheduled via Cloud Scheduler)
|
||||
- AWS Lambda + EventBridge
|
||||
- Railway / Render
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Slack App created
|
||||
- [ ] config.yaml template
|
||||
- [ ] daily_summary.py
|
||||
- [ ] weekly_digest.py
|
||||
- [ ] anomaly_check.py
|
||||
- [ ] slack_blocks.py templates
|
||||
- [ ] reporter.py scheduler
|
||||
- [ ] Dockerfile
|
||||
- [ ] Tested locally
|
||||
- [ ] Deployed
|
||||
98
ga-agent-project/05-realtime-watcher/README.md
Normal file
98
ga-agent-project/05-realtime-watcher/README.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# Component 5: Realtime Watcher
|
||||
|
||||
**Type:** Standalone Service
|
||||
**Priority:** P3
|
||||
**Status:** Deferred
|
||||
|
||||
## Goal
|
||||
|
||||
Real-time GA4 monitoring with periodic snapshots to Slack.
|
||||
|
||||
## Status
|
||||
|
||||
**Deferred** — Complete components 1-4 first.
|
||||
|
||||
## Original Concept
|
||||
|
||||
- Screenshot GA4 real-time dashboard every 5 minutes
|
||||
- Send screenshots to Slack channel
|
||||
- Trigger via Slack command or user request
|
||||
|
||||
## Challenges
|
||||
|
||||
| Challenge | Issue |
|
||||
|-----------|-------|
|
||||
| Browser auth | GA4 requires Google login |
|
||||
| Maintenance | Screenshots break when UI changes |
|
||||
| Complexity | Headless browser + auth + scheduling |
|
||||
| Value | Screenshots may not be best UX |
|
||||
|
||||
## Simplified Approach (Recommended)
|
||||
|
||||
Instead of screenshots, use the GA4 Real-time API:
|
||||
|
||||
1. Fetch real-time data via API
|
||||
2. Generate chart image with Plotly
|
||||
3. Send image to Slack
|
||||
|
||||
### Benefits
|
||||
|
||||
- No browser automation
|
||||
- More reliable
|
||||
- Cleaner output
|
||||
- Programmatic data access
|
||||
|
||||
## Structure (Future)
|
||||
|
||||
```
|
||||
05-realtime-watcher/
|
||||
├── README.md
|
||||
├── realtime_api.py # GA4 Real-time API client
|
||||
├── chart_generator.py # Generate chart images
|
||||
├── slack_sender.py # Upload to Slack
|
||||
├── watcher.py # Main service
|
||||
├── config.yaml
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
## GA4 Real-time API
|
||||
|
||||
```python
|
||||
from google.analytics.data_v1beta import BetaAnalyticsDataClient
|
||||
from google.analytics.data_v1beta.types import RunRealtimeReportRequest
|
||||
|
||||
def get_realtime_users(property_id: str):
|
||||
client = BetaAnalyticsDataClient()
|
||||
|
||||
request = RunRealtimeReportRequest(
|
||||
property=f"properties/{property_id}",
|
||||
dimensions=[{"name": "unifiedScreenName"}],
|
||||
metrics=[{"name": "activeUsers"}]
|
||||
)
|
||||
|
||||
response = client.run_realtime_report(request)
|
||||
return response
|
||||
```
|
||||
|
||||
## Trigger Options
|
||||
|
||||
1. **Slack Command:** `/ga-realtime start`
|
||||
2. **Scheduled:** During campaign launches
|
||||
3. **API Endpoint:** Webhook trigger
|
||||
|
||||
## Implementation (When Ready)
|
||||
|
||||
1. Build real-time API client
|
||||
2. Create chart generator
|
||||
3. Add Slack integration
|
||||
4. Implement start/stop controls
|
||||
5. Add session timeout (1 hour default)
|
||||
|
||||
## Checklist (Future)
|
||||
|
||||
- [ ] Real-time API client
|
||||
- [ ] Chart generation
|
||||
- [ ] Slack integration
|
||||
- [ ] Trigger mechanism
|
||||
- [ ] Session management
|
||||
- [ ] Deployment
|
||||
90
ga-agent-project/README.md
Normal file
90
ga-agent-project/README.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# GA Agent Project
|
||||
|
||||
Build workspace for Google Analytics tools and the `15-ourdigital-ga-agent` Claude Skill.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Infrastructure │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
||||
│ │ GA4 MCP │ │ BigQuery MCP │ │ Dimension Explorer│ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 15-ourdigital-ga-agent (Claude Skill) │
|
||||
│ • Interactive analysis • Reports • Period comparisons │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Standalone Services │
|
||||
│ ┌────────────────────┐ ┌────────────────────────────┐ │
|
||||
│ │ Slack Reporter │ │ Realtime Watcher (deferred)│ │
|
||||
│ └────────────────────┘ └────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
| # | Component | Type | Priority | Status |
|
||||
|---|-----------|------|----------|--------|
|
||||
| 1 | [MCP Setup](01-mcp-setup/) | Infrastructure | P0 | Pending |
|
||||
| 2 | [GA Agent Skill](02-ga-agent-skill/) | Claude Skill | P0 | Pending |
|
||||
| 3 | [Dimension Explorer](03-dimension-explorer/) | Utility | P1 | Pending |
|
||||
| 4 | [Slack Reporter](04-slack-reporter/) | Service | P2 | Pending |
|
||||
| 5 | [Realtime Watcher](05-realtime-watcher/) | Service | P3 | Deferred |
|
||||
|
||||
## Build Order
|
||||
|
||||
```
|
||||
Phase 1: Foundation
|
||||
├── [1] MCP Setup ←── START HERE
|
||||
└── [2] GA Agent Skill
|
||||
|
||||
Phase 2: Enhancements
|
||||
├── [3] Dimension Explorer
|
||||
└── [4] Slack Reporter
|
||||
|
||||
Phase 3: Advanced
|
||||
└── [5] Realtime Watcher (deferred)
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
ga-agent-project/
|
||||
├── README.md
|
||||
├── .gitignore
|
||||
├── config/ # Credentials (gitignored)
|
||||
├── docs/
|
||||
│ ├── PROJECT-PLAN.md # Full implementation plan
|
||||
│ ├── 01-mcp-servers-overview.md
|
||||
│ ├── 02-setup-guide.md
|
||||
│ └── 03-visualization-setup.md
|
||||
├── 01-mcp-setup/ # MCP server installation
|
||||
├── 02-ga-agent-skill/ # Core Claude Skill
|
||||
├── 03-dimension-explorer/ # Dimension/metric validator
|
||||
├── 04-slack-reporter/ # Automated Slack reports
|
||||
└── 05-realtime-watcher/ # Real-time monitoring (deferred)
|
||||
```
|
||||
|
||||
## Quick Resume
|
||||
|
||||
```bash
|
||||
cd /Users/ourdigital/Projects/claude-skills-factory/ga-agent-project
|
||||
|
||||
# Read the full plan
|
||||
cat docs/PROJECT-PLAN.md
|
||||
|
||||
# Start with Component 1
|
||||
cat 01-mcp-setup/README.md
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Google Cloud account with billing enabled
|
||||
- GA4 property access (Admin or Viewer)
|
||||
- Python 3.10+
|
||||
- Node.js 18+ (for BigQuery MCP)
|
||||
- Slack workspace (for Component 4)
|
||||
94
ga-agent-project/docs/01-mcp-servers-overview.md
Normal file
94
ga-agent-project/docs/01-mcp-servers-overview.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# MCP Servers Overview for GA Agent
|
||||
|
||||
## Available MCP Servers
|
||||
|
||||
### Google Analytics MCP Servers
|
||||
|
||||
| Server | Language | Source | Status |
|
||||
|--------|----------|--------|--------|
|
||||
| **google-analytics-mcp** (Official) | Python | [googleanalytics/google-analytics-mcp](https://github.com/googleanalytics/google-analytics-mcp) | Recommended |
|
||||
| mcp-server-google-analytics | TypeScript | [ruchernchong/mcp-server-google-analytics](https://github.com/ruchernchong/mcp-server-google-analytics) | Community |
|
||||
|
||||
**Official Google GA MCP Features:**
|
||||
- Real-time reporting
|
||||
- Custom/standard dimensions/metrics
|
||||
- Natural language queries (e.g., "top products by revenue")
|
||||
- `order_by` support
|
||||
- OAuth + Service Account auth
|
||||
|
||||
### BigQuery MCP Servers
|
||||
|
||||
| Server | Language | npm | Status |
|
||||
|--------|----------|-----|--------|
|
||||
| **@ergut/mcp-bigquery-server** | Node.js | `npx -y @ergut/mcp-bigquery-server` | Recommended |
|
||||
| mcp-server-bigquery | Python | - | Alternative |
|
||||
| Google MCP Toolbox | Python | - | Official (multi-DB) |
|
||||
|
||||
**ergut/mcp-bigquery-server Features:**
|
||||
- Read-only secure access
|
||||
- Schema discovery
|
||||
- Natural language to SQL
|
||||
- 1GB query limit
|
||||
|
||||
## Recommended Stack
|
||||
|
||||
For our GA Agent, we recommend:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ Claude Code │
|
||||
│ │ │
|
||||
│ ┌───────────┴───────────┐ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Google Analytics│ │ BigQuery │ │
|
||||
│ │ MCP Server │ │ MCP Server │ │
|
||||
│ └────────┬────────┘ └────────┬────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ GA4 Data API │ │ BigQuery API │ │
|
||||
│ │ GA4 Admin API │ │ (GA4 Export) │ │
|
||||
│ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Why Both?
|
||||
|
||||
1. **GA4 MCP** - Direct API access for:
|
||||
- Real-time data
|
||||
- Quick metrics queries
|
||||
- Account/property management
|
||||
|
||||
2. **BigQuery MCP** - For advanced analysis:
|
||||
- Historical data (GA4 → BigQuery export)
|
||||
- Complex SQL queries
|
||||
- Cross-dataset joins
|
||||
- Large-scale analysis
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Google Cloud Setup
|
||||
|
||||
1. Create a Google Cloud Project (or use existing)
|
||||
2. Enable these APIs:
|
||||
- Google Analytics Data API
|
||||
- Google Analytics Admin API
|
||||
- BigQuery API
|
||||
|
||||
3. Create Service Account:
|
||||
- Go to IAM & Admin → Service Accounts
|
||||
- Create new service account
|
||||
- Grant roles:
|
||||
- `Analytics Viewer` (or Admin for write ops)
|
||||
- `BigQuery Data Viewer`
|
||||
- `BigQuery Job User`
|
||||
- Download JSON key file
|
||||
|
||||
4. Grant GA4 Access:
|
||||
- In GA4 Admin → Property Access Management
|
||||
- Add service account email with Viewer role
|
||||
|
||||
## Next Steps
|
||||
|
||||
See `02-setup-guide.md` for installation instructions.
|
||||
203
ga-agent-project/docs/02-setup-guide.md
Normal file
203
ga-agent-project/docs/02-setup-guide.md
Normal file
@@ -0,0 +1,203 @@
|
||||
# MCP Server Setup Guide
|
||||
|
||||
## Step 1: Google Cloud Prerequisites
|
||||
|
||||
### 1.1 Create/Select Project
|
||||
|
||||
```bash
|
||||
# List existing projects
|
||||
gcloud projects list
|
||||
|
||||
# Create new project (optional)
|
||||
gcloud projects create ga-agent-project --name="GA Agent Project"
|
||||
|
||||
# Set active project
|
||||
gcloud config set project YOUR_PROJECT_ID
|
||||
```
|
||||
|
||||
### 1.2 Enable Required APIs
|
||||
|
||||
```bash
|
||||
# Enable all required APIs
|
||||
gcloud services enable \
|
||||
analyticsdata.googleapis.com \
|
||||
analyticsadmin.googleapis.com \
|
||||
bigquery.googleapis.com
|
||||
```
|
||||
|
||||
### 1.3 Create Service Account
|
||||
|
||||
```bash
|
||||
# Create service account
|
||||
gcloud iam service-accounts create ga-agent-sa \
|
||||
--display-name="GA Agent Service Account"
|
||||
|
||||
# Get the email
|
||||
SA_EMAIL="ga-agent-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com"
|
||||
|
||||
# Grant BigQuery roles
|
||||
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
|
||||
--member="serviceAccount:$SA_EMAIL" \
|
||||
--role="roles/bigquery.dataViewer"
|
||||
|
||||
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
|
||||
--member="serviceAccount:$SA_EMAIL" \
|
||||
--role="roles/bigquery.jobUser"
|
||||
|
||||
# Create and download key
|
||||
gcloud iam service-accounts keys create \
|
||||
~/ga-agent-credentials.json \
|
||||
--iam-account=$SA_EMAIL
|
||||
|
||||
# Move to secure location
|
||||
mv ~/ga-agent-credentials.json /path/to/secure/location/
|
||||
```
|
||||
|
||||
### 1.4 Grant GA4 Property Access
|
||||
|
||||
1. Go to [Google Analytics Admin](https://analytics.google.com/analytics/web/)
|
||||
2. Select your property
|
||||
3. Admin → Property Access Management
|
||||
4. Click "+" → Add users
|
||||
5. Enter service account email: `ga-agent-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com`
|
||||
6. Select role: **Viewer** (or Analyst for more access)
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Install Google Analytics MCP Server
|
||||
|
||||
### Option A: Official Google GA MCP (Python)
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/googleanalytics/google-analytics-mcp.git
|
||||
cd google-analytics-mcp
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
|
||||
# Install dependencies
|
||||
pip install -e .
|
||||
|
||||
# Set credentials
|
||||
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/ga-agent-credentials.json"
|
||||
|
||||
# Test the server
|
||||
python -m google_analytics_mcp
|
||||
```
|
||||
|
||||
### Option B: TypeScript Community Server
|
||||
|
||||
```bash
|
||||
# Install globally
|
||||
npm install -g @anthropic/mcp-server-google-analytics
|
||||
|
||||
# Or run with npx
|
||||
npx @anthropic/mcp-server-google-analytics
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Install BigQuery MCP Server
|
||||
|
||||
```bash
|
||||
# Using npx (recommended - no install needed)
|
||||
npx -y @ergut/mcp-bigquery-server \
|
||||
--project-id YOUR_PROJECT_ID \
|
||||
--location us-central1 \
|
||||
--key-file /path/to/ga-agent-credentials.json
|
||||
|
||||
# Or install globally
|
||||
npm install -g @ergut/mcp-bigquery-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Configure Claude Code
|
||||
|
||||
Add to your Claude Code MCP configuration (`~/.claude/mcp_servers.json` or project `.mcp.json`):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"google-analytics": {
|
||||
"command": "python",
|
||||
"args": ["-m", "google_analytics_mcp"],
|
||||
"env": {
|
||||
"GOOGLE_APPLICATION_CREDENTIALS": "/path/to/ga-agent-credentials.json"
|
||||
}
|
||||
},
|
||||
"bigquery": {
|
||||
"command": "npx",
|
||||
"args": [
|
||||
"-y",
|
||||
"@ergut/mcp-bigquery-server",
|
||||
"--project-id", "YOUR_PROJECT_ID",
|
||||
"--location", "us-central1",
|
||||
"--key-file", "/path/to/ga-agent-credentials.json"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Verify Installation
|
||||
|
||||
After restarting Claude Code:
|
||||
|
||||
```bash
|
||||
# Check servers are connected
|
||||
mcp-cli servers
|
||||
|
||||
# List available tools
|
||||
mcp-cli tools google-analytics
|
||||
mcp-cli tools bigquery
|
||||
```
|
||||
|
||||
Expected output should show tools like:
|
||||
- `google-analytics/run_report`
|
||||
- `google-analytics/run_realtime_report`
|
||||
- `bigquery/execute-query`
|
||||
- `bigquery/list-tables`
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Authentication Errors
|
||||
|
||||
```bash
|
||||
# Verify credentials
|
||||
gcloud auth application-default print-access-token
|
||||
|
||||
# Check service account permissions
|
||||
gcloud projects get-iam-policy YOUR_PROJECT_ID \
|
||||
--filter="bindings.members:ga-agent-sa"
|
||||
```
|
||||
|
||||
### GA4 Access Issues
|
||||
|
||||
- Ensure service account email is added to GA4 property
|
||||
- Wait 5-10 minutes after adding access
|
||||
- Check property ID is correct (numeric, not "UA-" format)
|
||||
|
||||
### BigQuery Connection Issues
|
||||
|
||||
```bash
|
||||
# Test BigQuery access directly
|
||||
bq ls YOUR_PROJECT_ID:analytics_*
|
||||
|
||||
# Check dataset exists
|
||||
bq show YOUR_PROJECT_ID:analytics_PROPERTY_ID
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Set up GA4 → BigQuery export (if not already)
|
||||
2. Create visualization tools (see `03-visualization-setup.md`)
|
||||
3. Build the Claude Skill
|
||||
286
ga-agent-project/docs/03-visualization-setup.md
Normal file
286
ga-agent-project/docs/03-visualization-setup.md
Normal file
@@ -0,0 +1,286 @@
|
||||
# Visualization Tools Setup
|
||||
|
||||
## Overview
|
||||
|
||||
For lightweight dashboards displaying GA4/BigQuery insights, we recommend:
|
||||
|
||||
| Tool | Best For | Complexity |
|
||||
|------|----------|------------|
|
||||
| **Streamlit** | Quick Python dashboards | Low |
|
||||
| **Plotly Dash** | Interactive charts | Medium |
|
||||
| **HTML + Chart.js** | Portable, no server | Low |
|
||||
|
||||
## Option 1: Streamlit Dashboard (Recommended)
|
||||
|
||||
### Install Dependencies
|
||||
|
||||
```bash
|
||||
cd /path/to/ga-agent-project/visualization
|
||||
|
||||
# Create virtual environment
|
||||
python -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
# Install packages
|
||||
pip install streamlit pandas plotly google-cloud-bigquery google-analytics-data
|
||||
```
|
||||
|
||||
### Basic Dashboard Template
|
||||
|
||||
Create `visualization/streamlit_dashboard.py`:
|
||||
|
||||
```python
|
||||
import streamlit as st
|
||||
import pandas as pd
|
||||
import plotly.express as px
|
||||
from google.cloud import bigquery
|
||||
from google.analytics.data_v1beta import BetaAnalyticsDataClient
|
||||
from google.analytics.data_v1beta.types import RunReportRequest
|
||||
|
||||
# Page config
|
||||
st.set_page_config(
|
||||
page_title="GA4 Analytics Dashboard",
|
||||
page_icon="📊",
|
||||
layout="wide"
|
||||
)
|
||||
|
||||
st.title("📊 GA4 Analytics Dashboard")
|
||||
|
||||
# Sidebar for configuration
|
||||
with st.sidebar:
|
||||
st.header("Settings")
|
||||
property_id = st.text_input("GA4 Property ID", "YOUR_PROPERTY_ID")
|
||||
date_range = st.selectbox(
|
||||
"Date Range",
|
||||
["Last 7 days", "Last 30 days", "Last 90 days"]
|
||||
)
|
||||
|
||||
# Date mapping
|
||||
date_map = {
|
||||
"Last 7 days": "7daysAgo",
|
||||
"Last 30 days": "30daysAgo",
|
||||
"Last 90 days": "90daysAgo"
|
||||
}
|
||||
|
||||
@st.cache_data(ttl=3600)
|
||||
def fetch_ga4_data(property_id: str, start_date: str):
|
||||
"""Fetch data from GA4 API"""
|
||||
client = BetaAnalyticsDataClient()
|
||||
|
||||
request = RunReportRequest(
|
||||
property=f"properties/{property_id}",
|
||||
dimensions=[{"name": "date"}],
|
||||
metrics=[
|
||||
{"name": "activeUsers"},
|
||||
{"name": "sessions"},
|
||||
{"name": "screenPageViews"}
|
||||
],
|
||||
date_ranges=[{"start_date": start_date, "end_date": "today"}]
|
||||
)
|
||||
|
||||
response = client.run_report(request)
|
||||
|
||||
data = []
|
||||
for row in response.rows:
|
||||
data.append({
|
||||
"date": row.dimension_values[0].value,
|
||||
"users": int(row.metric_values[0].value),
|
||||
"sessions": int(row.metric_values[1].value),
|
||||
"pageviews": int(row.metric_values[2].value)
|
||||
})
|
||||
|
||||
return pd.DataFrame(data)
|
||||
|
||||
# Fetch and display data
|
||||
try:
|
||||
df = fetch_ga4_data(property_id, date_map[date_range])
|
||||
|
||||
# Metrics row
|
||||
col1, col2, col3 = st.columns(3)
|
||||
with col1:
|
||||
st.metric("Total Users", f"{df['users'].sum():,}")
|
||||
with col2:
|
||||
st.metric("Total Sessions", f"{df['sessions'].sum():,}")
|
||||
with col3:
|
||||
st.metric("Total Pageviews", f"{df['pageviews'].sum():,}")
|
||||
|
||||
# Charts
|
||||
st.subheader("Traffic Over Time")
|
||||
fig = px.line(df, x="date", y=["users", "sessions"],
|
||||
title="Users & Sessions")
|
||||
st.plotly_chart(fig, use_container_width=True)
|
||||
|
||||
# Raw data
|
||||
with st.expander("View Raw Data"):
|
||||
st.dataframe(df)
|
||||
|
||||
except Exception as e:
|
||||
st.error(f"Error fetching data: {e}")
|
||||
st.info("Ensure GOOGLE_APPLICATION_CREDENTIALS is set")
|
||||
```
|
||||
|
||||
### Run Dashboard
|
||||
|
||||
```bash
|
||||
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/credentials.json"
|
||||
streamlit run visualization/streamlit_dashboard.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option 2: Static HTML Dashboard
|
||||
|
||||
For portable reports without a server:
|
||||
|
||||
Create `visualization/templates/report.html`:
|
||||
|
||||
```html
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>GA4 Report</title>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
|
||||
<style>
|
||||
body { font-family: -apple-system, sans-serif; margin: 40px; }
|
||||
.metrics { display: flex; gap: 20px; margin-bottom: 40px; }
|
||||
.metric-card {
|
||||
background: #f5f5f5;
|
||||
padding: 20px;
|
||||
border-radius: 8px;
|
||||
flex: 1;
|
||||
}
|
||||
.metric-value { font-size: 32px; font-weight: bold; }
|
||||
.metric-label { color: #666; }
|
||||
.chart-container { max-width: 800px; margin: 40px 0; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>📊 GA4 Analytics Report</h1>
|
||||
<p>Generated: <span id="date"></span></p>
|
||||
|
||||
<div class="metrics">
|
||||
<div class="metric-card">
|
||||
<div class="metric-value" id="users">--</div>
|
||||
<div class="metric-label">Active Users</div>
|
||||
</div>
|
||||
<div class="metric-card">
|
||||
<div class="metric-value" id="sessions">--</div>
|
||||
<div class="metric-label">Sessions</div>
|
||||
</div>
|
||||
<div class="metric-card">
|
||||
<div class="metric-value" id="pageviews">--</div>
|
||||
<div class="metric-label">Page Views</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="chart-container">
|
||||
<canvas id="trafficChart"></canvas>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Data will be injected by Python script
|
||||
const reportData = {{ DATA_JSON }};
|
||||
|
||||
document.getElementById('date').textContent = new Date().toLocaleDateString();
|
||||
document.getElementById('users').textContent = reportData.totals.users.toLocaleString();
|
||||
document.getElementById('sessions').textContent = reportData.totals.sessions.toLocaleString();
|
||||
document.getElementById('pageviews').textContent = reportData.totals.pageviews.toLocaleString();
|
||||
|
||||
new Chart(document.getElementById('trafficChart'), {
|
||||
type: 'line',
|
||||
data: {
|
||||
labels: reportData.dates,
|
||||
datasets: [{
|
||||
label: 'Users',
|
||||
data: reportData.users,
|
||||
borderColor: '#4285f4',
|
||||
tension: 0.1
|
||||
}, {
|
||||
label: 'Sessions',
|
||||
data: reportData.sessions,
|
||||
borderColor: '#34a853',
|
||||
tension: 0.1
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
responsive: true,
|
||||
plugins: {
|
||||
title: { display: true, text: 'Traffic Over Time' }
|
||||
}
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option 3: Python Chart Generation
|
||||
|
||||
For generating standalone chart images:
|
||||
|
||||
```python
|
||||
# visualization/scripts/generate_charts.py
|
||||
import pandas as pd
|
||||
import plotly.express as px
|
||||
import plotly.io as pio
|
||||
|
||||
def generate_traffic_chart(df: pd.DataFrame, output_path: str):
|
||||
"""Generate traffic chart as HTML or PNG"""
|
||||
fig = px.line(
|
||||
df,
|
||||
x="date",
|
||||
y=["users", "sessions"],
|
||||
title="Traffic Overview",
|
||||
template="plotly_white"
|
||||
)
|
||||
|
||||
fig.update_layout(
|
||||
xaxis_title="Date",
|
||||
yaxis_title="Count",
|
||||
legend_title="Metric"
|
||||
)
|
||||
|
||||
# Save as interactive HTML
|
||||
fig.write_html(f"{output_path}/traffic_chart.html")
|
||||
|
||||
# Save as static image (requires kaleido)
|
||||
# pip install kaleido
|
||||
fig.write_image(f"{output_path}/traffic_chart.png", scale=2)
|
||||
|
||||
return fig
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration with Claude Skill
|
||||
|
||||
The Claude Skill will use these visualization tools via Python scripts:
|
||||
|
||||
```
|
||||
15-ourdigital-ga-agent/
|
||||
├── SKILL.md
|
||||
├── scripts/
|
||||
│ ├── fetch_ga4_data.py # Get data from GA4/BigQuery
|
||||
│ ├── generate_report.py # Create visualizations
|
||||
│ └── streamlit_app.py # Launch dashboard
|
||||
├── templates/
|
||||
│ └── report.html # Static report template
|
||||
└── assets/
|
||||
└── styles.css # Dashboard styling
|
||||
```
|
||||
|
||||
## Requirements File
|
||||
|
||||
Create `visualization/requirements.txt`:
|
||||
|
||||
```
|
||||
streamlit>=1.28.0
|
||||
pandas>=2.0.0
|
||||
plotly>=5.18.0
|
||||
google-cloud-bigquery>=3.12.0
|
||||
google-analytics-data>=0.18.0
|
||||
kaleido>=0.2.1
|
||||
```
|
||||
319
ga-agent-project/docs/PROJECT-PLAN.md
Normal file
319
ga-agent-project/docs/PROJECT-PLAN.md
Normal file
@@ -0,0 +1,319 @@
|
||||
# GA Agent Project Plan (Revised)
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Infrastructure │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
||||
│ │ GA4 MCP │ │ BigQuery MCP │ │ Dimension Explorer│ │
|
||||
│ │ (install) │ │ (install) │ │ (build - small) │ │
|
||||
│ └──────────────┘ └──────────────┘ └──────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Claude Skill │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ 15-ourdigital-ga-agent │ │
|
||||
│ │ • Interactive analysis │ │
|
||||
│ │ • Report generation │ │
|
||||
│ │ • Period comparisons │ │
|
||||
│ └──────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Standalone Services (Later) │
|
||||
│ ┌────────────────────┐ ┌────────────────────────────┐ │
|
||||
│ │ ga4-slack-reporter │ │ ga4-realtime-watcher │ │
|
||||
│ │ (Python service) │ │ (defer or API-based) │ │
|
||||
│ └────────────────────┘ └────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Components
|
||||
|
||||
| # | Component | Type | Priority | Effort |
|
||||
|---|-----------|------|----------|--------|
|
||||
| 1 | MCP Setup | Infrastructure | P0 | Low |
|
||||
| 2 | ga-agent-skill | Claude Skill | P0 | Medium |
|
||||
| 3 | dimension-explorer | MCP Server / CLI | P1 | Low |
|
||||
| 4 | slack-reporter | Standalone Service | P2 | Medium |
|
||||
| 5 | realtime-watcher | Standalone Service | P3 | High (defer) |
|
||||
|
||||
---
|
||||
|
||||
## Component 1: MCP Setup
|
||||
|
||||
**Location:** `01-mcp-setup/`
|
||||
|
||||
**Goal:** Install and configure existing MCP servers
|
||||
|
||||
### Tasks
|
||||
|
||||
- [ ] Google Cloud project setup
|
||||
- [ ] Enable Analytics Data API
|
||||
- [ ] Enable Analytics Admin API
|
||||
- [ ] Enable BigQuery API
|
||||
- [ ] Service account creation
|
||||
- [ ] Create service account
|
||||
- [ ] Grant Analytics Viewer role
|
||||
- [ ] Grant BigQuery Data Viewer role
|
||||
- [ ] Download JSON key
|
||||
- [ ] GA4 property access
|
||||
- [ ] Add service account to GA4 property
|
||||
- [ ] Install GA4 MCP server
|
||||
- [ ] Clone `googleanalytics/google-analytics-mcp`
|
||||
- [ ] Configure credentials
|
||||
- [ ] Test connection
|
||||
- [ ] Install BigQuery MCP server
|
||||
- [ ] Configure `@ergut/mcp-bigquery-server`
|
||||
- [ ] Verify GA4 export dataset access
|
||||
- [ ] Add to Claude Code config
|
||||
- [ ] Update `~/.claude/mcp_servers.json`
|
||||
- [ ] Verify with `mcp-cli servers`
|
||||
|
||||
### Deliverables
|
||||
|
||||
- `01-mcp-setup/setup-guide.md` - Step-by-step instructions
|
||||
- `01-mcp-setup/mcp-config.example.json` - Example MCP configuration
|
||||
- Working MCP connections verified
|
||||
|
||||
---
|
||||
|
||||
## Component 2: GA Agent Skill (Core)
|
||||
|
||||
**Location:** `02-ga-agent-skill/` → Final: `ourdigital-custom-skills/15-ourdigital-ga-agent/`
|
||||
|
||||
**Goal:** Interactive GA4 analysis and reporting skill
|
||||
|
||||
### Features
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| Traffic Analysis | Users, sessions, pageviews with trends |
|
||||
| Period Comparison | WoW, MoM, YoY comparisons |
|
||||
| Top Content | Pages, sources, campaigns |
|
||||
| Report Generation | HTML/PDF reports |
|
||||
| BigQuery Queries | Complex analysis on exported data |
|
||||
|
||||
### Triggers (EN/KR)
|
||||
|
||||
- "Analyze GA4 traffic" / "GA4 트래픽 분석"
|
||||
- "Compare last week vs this week" / "지난주 대비 비교"
|
||||
- "Generate traffic report" / "트래픽 리포트 생성"
|
||||
- "Top landing pages" / "인기 랜딩 페이지"
|
||||
- "Query BigQuery for GA data" / "BigQuery GA 데이터 조회"
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
15-ourdigital-ga-agent/
|
||||
├── SKILL.md
|
||||
├── scripts/
|
||||
│ ├── analyze_traffic.py
|
||||
│ ├── compare_periods.py
|
||||
│ ├── top_content.py
|
||||
│ └── generate_report.py
|
||||
├── templates/
|
||||
│ └── report.html
|
||||
├── references/
|
||||
│ └── ga4-api-reference.md
|
||||
└── examples/
|
||||
└── sample-queries.md
|
||||
```
|
||||
|
||||
### Tasks
|
||||
|
||||
- [ ] Create SKILL.md with triggers
|
||||
- [ ] Build analysis scripts
|
||||
- [ ] analyze_traffic.py
|
||||
- [ ] compare_periods.py
|
||||
- [ ] top_content.py
|
||||
- [ ] Create report template
|
||||
- [ ] Add examples
|
||||
- [ ] Test with Claude Code
|
||||
- [ ] Move to `ourdigital-custom-skills/15-ourdigital-ga-agent/`
|
||||
|
||||
---
|
||||
|
||||
## Component 3: Dimension Explorer
|
||||
|
||||
**Location:** `03-dimension-explorer/`
|
||||
|
||||
**Goal:** Validate GA4 dimensions/metrics with explanations
|
||||
|
||||
### Options
|
||||
|
||||
| Option | Pros | Cons |
|
||||
|--------|------|------|
|
||||
| **A. MCP Server** | Claude can use directly | More setup |
|
||||
| **B. CLI Tool** | Simple, standalone | Manual invocation |
|
||||
| **C. Reference JSON** | No code needed | Static, needs refresh |
|
||||
|
||||
**Recommendation:** Start with C (Reference JSON), upgrade to A (MCP Server) later
|
||||
|
||||
### Features
|
||||
|
||||
- List all available dimensions/metrics
|
||||
- Validate if a dimension/metric exists
|
||||
- Get description, data type, category
|
||||
- Fuzzy search for typos
|
||||
- Compatibility checking
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
03-dimension-explorer/
|
||||
├── README.md
|
||||
├── fetch_metadata.py # Script to refresh metadata
|
||||
├── data/
|
||||
│ ├── dimensions.json # All dimensions with descriptions
|
||||
│ └── metrics.json # All metrics with descriptions
|
||||
└── explorer.py # CLI tool (optional)
|
||||
```
|
||||
|
||||
### Tasks
|
||||
|
||||
- [ ] Fetch metadata from GA4 Admin API
|
||||
- [ ] Structure as searchable JSON
|
||||
- [ ] Create CLI explorer (optional)
|
||||
- [ ] Document usage
|
||||
|
||||
---
|
||||
|
||||
## Component 4: Slack Reporter
|
||||
|
||||
**Location:** `04-slack-reporter/`
|
||||
|
||||
**Goal:** Automated GA4 reports to Slack
|
||||
|
||||
### Features
|
||||
|
||||
| Report | Schedule | Content |
|
||||
|--------|----------|---------|
|
||||
| Daily Summary | 9:00 AM | Users, sessions, top pages |
|
||||
| Weekly Digest | Monday 9 AM | WoW comparison, trends |
|
||||
| Anomaly Alert | Real-time | Traffic ±30% from baseline |
|
||||
|
||||
### Structure
|
||||
|
||||
```
|
||||
04-slack-reporter/
|
||||
├── README.md
|
||||
├── config.yaml # Schedules, channels, properties
|
||||
├── reporter.py # Main service
|
||||
├── queries/
|
||||
│ ├── daily_summary.py
|
||||
│ ├── weekly_digest.py
|
||||
│ └── anomaly_check.py
|
||||
├── templates/
|
||||
│ └── slack_blocks.py # Slack Block Kit templates
|
||||
├── requirements.txt
|
||||
└── Dockerfile # For deployment
|
||||
```
|
||||
|
||||
### Tasks
|
||||
|
||||
- [ ] Create Slack App
|
||||
- [ ] Build query functions
|
||||
- [ ] Create Slack message templates
|
||||
- [ ] Implement scheduler
|
||||
- [ ] Add Docker deployment
|
||||
- [ ] Document setup
|
||||
|
||||
---
|
||||
|
||||
## Component 5: Realtime Watcher (Deferred)
|
||||
|
||||
**Location:** `05-realtime-watcher/`
|
||||
|
||||
**Goal:** Real-time monitoring snapshots to Slack
|
||||
|
||||
**Status:** Deferred — revisit after components 1-4 complete
|
||||
|
||||
### Simplified Approach (API-based)
|
||||
|
||||
Instead of screenshots:
|
||||
1. Fetch real-time data via GA4 Real-time API
|
||||
2. Generate chart image with Plotly/Matplotlib
|
||||
3. Send to Slack
|
||||
|
||||
### Structure (Future)
|
||||
|
||||
```
|
||||
05-realtime-watcher/
|
||||
├── README.md
|
||||
├── realtime_api.py # Fetch real-time data
|
||||
├── chart_generator.py # Generate chart images
|
||||
├── watcher.py # Main service
|
||||
└── config.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Build Order
|
||||
|
||||
```
|
||||
Phase 1: Foundation
|
||||
├── [1] MCP Setup ←── START HERE
|
||||
└── [2] GA Agent Skill (core)
|
||||
|
||||
Phase 2: Enhancements
|
||||
├── [3] Dimension Explorer
|
||||
└── [4] Slack Reporter
|
||||
|
||||
Phase 3: Advanced (Deferred)
|
||||
└── [5] Realtime Watcher
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Setup
|
||||
|
||||
### Required Credentials
|
||||
|
||||
```bash
|
||||
# Google Cloud
|
||||
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
|
||||
GA4_PROPERTY_ID=123456789
|
||||
BIGQUERY_PROJECT_ID=your-project
|
||||
|
||||
# Slack (for Component 4)
|
||||
SLACK_BOT_TOKEN=xoxb-...
|
||||
SLACK_CHANNEL_ID=C0123456789
|
||||
```
|
||||
|
||||
### Python Dependencies
|
||||
|
||||
```
|
||||
# Core (Components 1-3)
|
||||
google-analytics-data>=0.18.0
|
||||
google-cloud-bigquery>=3.12.0
|
||||
google-auth>=2.23.0
|
||||
pandas>=2.0.0
|
||||
|
||||
# Visualization
|
||||
plotly>=5.18.0
|
||||
jinja2>=3.1.0
|
||||
|
||||
# Slack Reporter (Component 4)
|
||||
slack-sdk>=3.23.0
|
||||
apscheduler>=3.10.0
|
||||
pyyaml>=6.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Navigate to project
|
||||
cd /Users/ourdigital/Projects/claude-skills-factory/ga-agent-project
|
||||
|
||||
# Start with MCP setup
|
||||
cat 01-mcp-setup/setup-guide.md
|
||||
```
|
||||
Reference in New Issue
Block a user