Is A/B Testing Doable With Your Website?

(And What You Should Do Instead)

When most people think of conversion optimisation, they think of A/B testing — showing two versions of a page and declaring a winner based on statistical confidence.

It’s a great idea in theory. But for websites with low traffic or long buying cycles, it quickly becomes a mathematical nightmare. But what is “low traffic”? Let’s see.

---

The Illusion of A/B Testing for Small Numbers

Let’s say your website converts 2% of visitors. You make a change you believe could improve conversions by 10%. That sounds significant, right?

Here’s the reality:
A 10% lift on a 2% conversion rate takes you from 2.0% → 2.2% — an improvement of just 0.2 percentage points.

To prove that difference with 95% confidence and 80% statistical power, you’d need roughly:

63,000 visitors per variant
or over 120,000 total visitors for one test.

That’s the equivalent of running an A/B test for months — or even years — on a site that only attracts a few thousand visitors a month.

---

The Numbers Don’t Lie

Here are some real examples using standard statistical models:

Even at 40,000 visitors a week — a strong audience for most B2B or high-value brands — you’d still need three months to detect a 5% improvement with statistical confidence.

And if your site attracts just a few thousand visitors a week?
You might be looking at 17+ weeks to get a statistically valid result.

---

Why This Matters for “Considered Purchase” Brands

If you sell to people who make big, thoughtful decisions — whether that’s financial planning, healthcare, education, or B2B services — your traffic tends to be:

• Lower volume

• Higher intent

• More variable

• Influenced by seasonality and multi-touch behaviour

That means traditional A/B testing isn’t just slow — it’s often statistically impossible to run meaningfully within a business cycle.

---

So What’s the Alternative?

At UserBoost, we developed a different approach:

B.O.B. — The Behavioural Observation Board

Instead of waiting months for A/B tests to reach significance, B.O.B. tracks how users actually behave before and after changes are made — then uses Bayesian probability to measure confidence in improvement.

Each optimisation is presented in plain language:

“Adding new social proof block — 58.3% improvement.”

You get clarity over complexity — with measurable insights that don’t rely on massive sample sizes or endless test durations.

---

The Bottom Line

A/B testing is powerful when you have the traffic and transaction volume to support it.
But for most considered-purchase brands, it’s not the right tool.

You don’t need more data.
You need better visibility — the kind that helps you see whether your changes are actually working, in real time, with the audience you already have.

That’s exactly what B.O.B. delivers.