Overview

A/B Test Planner is a hybrid planner and analyzer. In planning mode it pulls live GA4 funnel data, finds the highest-$-impact test surfaces and ranks candidates with sample size and run time. In analysis mode it takes your test numbers and returns a rigorous verdict — with the math shown.

What it does

• Plan tests — ranked by expected revenue impact, with MDE and required sample size.

• Run-time projection — how long at your current traffic.

• Rigorous analysis — two-proportion z-test / chi-squared, p-value, 95% CI, power.

• Guardrails — SRM check, peeking, novelty and day-of-week flags.

• Clear verdict — Ship / Keep Running / Kill / Re-test, reasoning visible.
  
  

How to use it

1. Plan: connect GA4/Shopify and ask "what should I test next".

2. Analyze: paste sessions and conversions per variant + days run.

3. Shadow computes the stats inline and checks for bias.

4. Get the verdict — with the auditable math.

Run it in-app or from Claude / ChatGPT via the Shadow MCP.

What it needs

GA4/Shopify for planning; your four test numbers + days run for analysis.

What you get

Planning: a ranked test backlog with sample size and run time. Analysis: a full results table with p-value, CI, power and an explicit ship/kill verdict.

Not the right skill when…

• You want a full-funnel audit → use CRO Audit, instead

• You want a single-page review → use Landing Page Analyzer, instead

• You want to know why a metric moved → use Root-Cause Analysis, instead