In the context of CRO and A/B testing, a hypothesis is a structured, falsifiable prediction that states what change you will make, why you expect it to work (based on evidence), and what outcome you anticipate on a specific metric. A well-formed hypothesis is the foundation of a rigorous experiment — it prevents teams from running tests without a clear rationale and forces them to commit to what "winning" looks like before the test starts.
There is no mathematical formula, but a reliable hypothesis structure is:
"Because [observation/evidence], we believe that [proposed change] for [audience segment] will cause [expected outcome] as measured by [metric]."
Example: "Because our session recordings show that 68% of mobile users who reach checkout drop off at the address form, we believe that reducing the address form to 3 required fields (instead of 7) for returning customers will reduce checkout abandonment rate from 68% to below 55%, as measured by completed purchases."
Every element serves a purpose: the observation justifies the test, the change is specific and buildable, the audience is defined, the outcome is measurable.
Why Hypothesis Matters for Ecommerce
Running an A/B test without a hypothesis is just changing things and hoping something improves. A hypothesis-driven process ensures every test contributes to a body of knowledge — even losing tests tell you something definitive about your customers. For Shopify brands with limited monthly traffic, this matters enormously. You may only be able to run 3–4 tests per month at statistical significance. A strong hypothesis makes each test count by grounding it in real evidence rather than opinion. Teams that document hypotheses and outcomes also build an institutional memory that prevents repeating failed experiments.
Real-World Example
The CRO team at Pilgrim (Indian D2C skincare brand) reviews their funnel data and notices that the PDP to add-to-cart rate for their hair care range is 12%, significantly below their skincare range at 21%. Session recordings reveal users are scrolling past the product description without reading the ingredients section. Hypothesis: "Because users are not engaging with our ingredient list (average scroll depth stops at 40% on hair care PDPs), we believe moving the 'hero ingredient callout' above the fold and replacing dense text with icons will increase add-to-cart rate from 12% to at least 16%, as measured over a 3-week test."
How to Improve / Optimize Hypotheses
- Always anchor to data: Every hypothesis should cite an observation from analytics, heatmaps, session recordings, or customer feedback. "I think users find it confusing" is not a valid observation.
- One change per hypothesis: Testing multiple changes in a single experiment makes it impossible to know which change drove the result.
- Define the metric upfront: Primary metric (the one that decides win/loss) must be specified before the test launches. Post-hoc metric selection is a common source of false positives.
- Specify the audience segment: A hypothesis that applies to "all users" may mask important differences between mobile and desktop users, or new versus returning customers.
- Set a minimum detectable effect: Decide what lift in your metric would be meaningful to the business. Running a test to detect a 0.1% change is rarely worth the traffic cost.
Hypothesis in A/B Testing
Every A/B test is the operationalisation of a hypothesis. When a test concludes, the outcome — win, loss, or inconclusive — should be recorded against the original hypothesis, contributing to a searchable experiment library that future test ideas can reference.
Run smarter A/B tests with CustomFit.ai — 14-day free trial, no credit card required.