Stop 13 · Marketing Tools & CollaborationTopics · 5/7

A/B Testing Tools (Optimizely, VWO Overview)

In short

Is the green button or the orange one better? Opinions are free and often wrong. A/B testing tools split real visitors between versions and tell you which one actually converts more.

Dedicated tools for running structured website experiments — beyond simple ad A/B tests, these let you test entire page variations and route traffic automatically based on statistical results.

The blob holds two plates of jalebis, one in each hand, comparing them.

Think of it like this

Two jalebi recipes

A halwai can't decide between his old jalebi recipe and a new crispier one. So for a week, every other customer gets one or the other, and he counts who comes back for seconds. He didn't ask his cousins' opinions. He let the customers decide. That's an A/B test.

Watch it happen

How a testing tool runs one experiment

Visitor

Test tool

Website

GET /pricing

1/5A visitor opens the pricing page.

The big ideas

What you need to know

Random split

Visitors are randomly divided so both groups are similar. That's what makes the comparison fair.

One goal, set first

Choose the main metric, like sign-ups, before the test starts. Not after seeing results.

Enough time and traffic

Run until you reach the planned sample, usually at least a full week to cover weekday patterns.

The tools

Optimizely and VWO are paid tools for page experiments. Google Optimize shut down in 2023.

See it in code

Walk through the code

excel

1       Visitors   Signups   Conv. rate2A      5000       150       =C2/B2      → 3.0%3B      5000       190       =C3/B3      → 3.8%4Lift                        =(D3-D2)/D2 → 26.7%

1/2Conversion rate is sign-ups divided by visitors for each version.

Working out conversion rate and lift in Sheets. Before trusting a winner, check statistical significance with a test calculator or your tool's report.

Save yourself the pain

Common mistakes

Oops

Stopping the test the moment one version pulls ahead.

Do this instead

Decide sample size and duration upfront. Early leads often disappear.

Oops

Changing headline, image and price all in one variant.

Do this instead

Change one main thing, or you won't know what caused the result.

Your mission

Plan a test

~25 min

Quick check

Why does an A/B testing tool split visitors randomly?

Go deeper

1Explore a free trial or demo of an A/B testing tool's interface.
2Learn the basic setup: defining a goal, creating variants, and setting traffic split.
3Compare this to manually running an A/B test without dedicated tooling.