Ab test statistical analyzer

Skill a5c-ai/babysitter/library/specializations/data-engineering-analytics/skills/ab-test-statistical-analyzer

Babysitter enforces obedience on agentic workforces and enables them to manage extremely complex tasks and workflows through deterministic, hallucination-free self-orchestration

Install

npx -y skills add a5c-ai/babysitter --skill ab-test-statistical-analyzer

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

What its author says it does

Copied from the file, not written here

Performs statistical analysis for A/B testing experiments

SKILL.md

2.1 KB, as published. Nobody here has run it

A/B Test Statistical Analyzer

Overview

Performs statistical analysis for A/B testing experiments. This skill provides rigorous statistical methods to determine experiment validity and significance.

Capabilities

Sample size calculation
Statistical significance testing
Bayesian analysis
Sequential testing
Multi-armed bandit analysis
Segment analysis
Novelty/primacy effect detection
SRM (Sample Ratio Mismatch) detection
Confidence interval calculation
Power analysis

Input Schema

{
  "experimentData": {
    "control": "object",
    "variants": ["object"]
  },
  "metrics": [{
    "name": "string",
    "type": "conversion|continuous|ratio"
  }],
  "analysisType": "frequentist|bayesian|sequential"
}

Output Schema

{
  "results": [{
    "metric": "string",
    "controlValue": "number",
    "variantValues": ["number"],
    "pValue": "number",
    "confidenceInterval": "object",
    "significant": "boolean"
  }],
  "srmCheck": "object",
  "recommendation": "string"
}

Target Processes

A/B Testing Pipeline
Feature Store Setup

Usage Guidelines

Provide complete experiment data for control and variants
Define metrics with appropriate types
Select analysis methodology based on requirements
Review SRM checks before interpreting results

Best Practices

Always check for sample ratio mismatch before analysis
Use appropriate statistical tests for metric types
Consider practical significance alongside statistical significance
Account for multiple comparison corrections
Document assumptions and limitations