EyeFly Ops › 09 — Split Test SOP

09 — Split Test SOP

⚡ Split test idea added to backlog in brain from any of: - Akash (strategy or perf👤 | Role | Responsibility | |---|---| | Creative Strategist / 📊 - Learnings documented and applied to active SOPs per quarter (target:
1
Backlog
  • Log test idea in brain with:
  • Hypothesis: "We believe [change] will produce [outcome] because [reason]"
  • Variable being tested (single variable only — no multivariate)
  • Success metric: specific, measurable (e.g., CTR, CPQBC, lead volume)
  • Minimum run duration (see Gap Flags for threshold TBD)
  • Control vs. variant definition
hypothesis, success metric, and minimum run duration are logged and approved
Branch
Test disrupted mid-run (accidental edit, platform issue):
Invalidate test. Log disruption reason in brain. Restart from Backlog with same hypothesis if disruption was external.
Branch
Test reveals a new hypothesis:
Log new hypothesis as a separate Backlog entry — do not extend the current test.
2
In Progress (SLA: 168h to launch after approval)
  • EA sets up test per protocol:
  • Duplicate control asset or workflow
  • Apply single variable change to variant
  • Confirm tracking is in place before launch
  • Log start date and baseline metrics in brain project record
  • Do not touch test mid-run — no optimizations, no budget changes, no creative edits
test has run for full minimum duration with no interruption
3
Completed (SLA: 72h post-run)
  • Pull results: winner, loser, delta on success metric
  • Log in brain project record:
  • Winner (control or variant)
  • Delta (absolute + percentage)
  • Sample size and run duration
  • Confidence level if calculable
result logged in brain with all fields above
4
Documented (terminal)
  • Write findings doc — include:
  • Hypothesis tested
  • Result summary
  • Recommended action (scale winner, update SOP, run follow-up test)
  • Update relevant SOP if result changes best practice
  • Link findings doc in `brain.resource_docs` with tag `split-test`
findings doc exists, SOP updated if applicable — WIN terminal
↓ Hands off to - **Winning variant (creative)** → Meta Ads Creative Pipeline at Not Started. Include: asset link, test summary, delta achieved. - **SOP-impacting findings** → relevant pipeline SOP file updated in same session as Documented stage. Akash approval required before update. - **Findings docs** → linked in `brain.resource_docs` with tag `split-test` for quarterly review.