A/B tests are powerful, but easy to get wrong. Misleading results can lead you to keep worse pages.
1. Stopping too early
A version that leads after a day or a handful of conversions may just be lucky. Decide sample size in advance with our sample size calculator.
2. Testing with too little traffic
Small websites may need weeks or months for a valid test. Test big changes, which show larger differences, rather than tiny tweaks.
3. Changing many things at once
If you change the headline, image and form together, you will not know which change mattered. Sometimes this is fine for a radical redesign, but know the limitation.
4. Ignoring weekly patterns
Run tests over full weeks so weekdays and weekends are both included.
5. Measuring the wrong goal
A version with more form fills may bring worse leads. Track through to customers where possible.
6. Uneven traffic
Make sure both versions get similar traffic from the same sources.
7. Not checking significance
Use our A/B test calculator before declaring a winner.
8. Not recording results
Keep a log of tests and lessons so you build knowledge over time.
Read A/B test ideas.
Want the full picture? This article is part of Landing pages: the complete guide, our in-depth guide with everything in one place.
