Minimum Detectable Effect (MDE) is the smallest true difference in conversion rate between control and variant that an A/B test has sufficient statistical power to detect — given a defined significance level (alpha) and power (1 − beta). In practical terms, MDE answers: "What is the smallest improvement this test can reliably find?" If you set an MDE of 5% relative improvement, you are saying your test is powered to detect a conversion rate increase from (for example) 4.0% to 4.2%, but not smaller changes.
Simplified relationship: Smaller MDE = larger required sample size. Larger traffic volume = smaller detectable MDE = more sensitivity.
Formula approximation:
MDE ≈ (Zα/2 + Zβ) × √[2 × p(1−p) / n]
Where p = baseline conversion rate, n = sample size per variant, Zα/2 = z-score for significance level (1.96 for 95%), Zβ = z-score for power (0.84 for 80% power).
Why Minimum Detectable Effect Matters for Ecommerce
MDE is the bridge between statistical theory and business reality. It forces you to answer a critical question before running any test: "What size of improvement actually matters for this business?" If your product page converts at 3.5% and a 0.2% absolute lift would generate ₹5 lakh/month in additional revenue, then your MDE should be no larger than 0.2 absolute percentage points — and your test must be designed to detect that.
Many ecommerce teams set MDE too large, choosing an MDE that makes tests fast but misses commercially meaningful lifts. A test designed to detect only 20% relative lifts will miss a consistent 8% improvement — which, on a high-traffic store, may represent ₹50+ lakh/year in incremental revenue.
Conversely, setting MDE too small makes tests impractically long. A test powered to detect a 0.1% absolute lift on a 3% baseline conversion rate might require 200,000+ visitors per variant — months of traffic for most Indian D2C brands.
Real-World Example
WOW Skin Science gets 25,000 unique visitors/day to their homepage. Their homepage-to-PDP click rate is 18%. They want to test a new product grid layout. Before starting, they run a power analysis: at 80% power and 95% significance, with 25,000 daily visitors split 50/50, they can detect an MDE of approximately 1.5 percentage points relative (18% → 18.27% absolute) in a 14-day test. They assess whether a 1.5% relative improvement is commercially meaningful. Given their AOV of ₹850 and traffic volume, a 1.5% click rate lift translates to approximately 375 additional PDP visits/day, with an expected incremental revenue impact of ₹3.2 lakh/month. Yes — this is worth detecting. They proceed with the test at this MDE, and it returns a 2.1% relative lift — comfortably above the MDE.
How to Improve / Optimize Minimum Detectable Effect
- Start with business impact, work backward to MDE. Calculate what revenue a 1%, 2%, and 5% lift would generate at your current traffic and conversion rates. The smallest meaningful revenue impact determines your maximum acceptable MDE.
- Accept higher MDEs for lower-traffic pages. If a category page gets 2,000 visitors/day, you need a large MDE to run tests in a reasonable timeframe. That's fine — focus high-sensitivity tests on your highest-traffic pages.
- Reduce variance to reduce required sample size. Use CUPED (Controlled-experiment Using Pre-Experiment Data) or covariates to reduce the variance in your outcome metric. Lower variance means a given sample size can detect a smaller MDE.
- Run one metric as primary. Each additional primary metric requires a multiple comparisons correction, effectively shrinking the detectable effect for all metrics. Commit to one primary metric and treat others as guardrails.
- Use a sample size calculator before every test. Never set test duration by gut feel. Calculate the required sample size for your target MDE and divide by daily traffic to get the minimum test duration.
Minimum Detectable Effect in A/B Testing
MDE is one of four parameters that define a frequentist A/B test's statistical properties, alongside significance level (alpha), statistical power (1 − beta), and sample size. Setting any three determines the fourth. Most testing practitioners fix alpha (0.05) and power (0.80) and let MDE and sample size be the negotiable variables — the business MDE requirement determines how long the test must run.
Run smarter A/B tests with CustomFit.ai — 14-day free trial, no credit card required.