A/B testing best practices
for conversion optimization.

What A/B testing is, and what it is not

An A/B test shows two versions of a page or element to similar visitors at the same time and measures which one produces more of the outcome you care about, such as form submissions or purchases. It is the most reliable way to learn whether a change helps, because it replaces opinion with evidence.

It is not a way to validate a decision you have already made, and it is not useful on a page that gets very little traffic. Getting the setup right matters more than the tool you use.

Start with a hypothesis, not an idea

A good test starts from a problem you can see in the data and states what you expect to change and why. For example: because most visitors leave the pricing page without clicking, making the primary button clearer and moving it above the plans will increase demo requests. That gives you a reason, a change and a metric.

Test one meaningful change at a time

If you change the headline, image and button together you will not know which one mattered. Test a single variable, or a single coherent idea, so the result teaches you something you can reuse elsewhere.

What is worth testing first

• Headlines and value propositions. They shape whether visitors keep reading.
• Calls to action. Wording, size, colour contrast and position.
• Forms. Number of fields, labels, layout and error handling.
• Social proof. Which reviews or logos you show and where.
• Pricing and offer presentation. Layout, anchoring and guarantees.
• Page structure. Order of sections and length of the page.

Get the sample size and duration right

Small differences need a lot of visitors to detect. Before you launch, use a sample-size calculator to estimate how many visitors each version needs, based on your current conversion rate and the smallest improvement worth caring about. Then commit to that number.

• Run the test for at least one or two full business cycles, usually whole weeks, so weekday and weekend behaviour are both covered.
• Do not stop the moment one version pulls ahead. Early leads often disappear.
• Avoid running tests during unusual periods, such as a big sale, unless that is the audience you want to learn about.

Understand significance without worshipping it

Statistical significance tells you how unlikely your result would be if the two versions were actually the same. Most teams look for a high level of confidence before acting, but a significant result on a tiny effect may not be worth the change. Consider the size of the lift and the cost of implementing it, not only the p-value.

Common mistakes

• Testing on pages with too little traffic to reach a result.
• Changing the test while it is running.
• Ignoring segments, such as mobile and desktop, that behave differently.
• Only counting clicks when the real goal is completed enquiries or sales.
• Not recording results, so the same test is run again a year later.

Keep a testing log

For every test record the hypothesis, screenshots, dates, sample size, result and what you decided. Over time this becomes a map of what your audience responds to, which is more valuable than any single win.

When not to A/B test

If a page has a clear usability problem, such as a broken form or a confusing checkout, fix it. Testing is for choosing between good options, not for confirming that broken things are broken.

How to choose your primary metric

Every test needs one main measure of success, chosen before it starts. If you sell online, it might be completed purchases or revenue per visitor. For a service business, it might be submitted enquiry forms, booked calls or clicks on a phone number. Avoid choosing a shallow metric such as clicks on a button if the real goal happens later. A headline that gets more clicks but attracts less qualified enquiries can look like a win while quietly hurting the business.

Add a few guardrail metrics that you will watch to make sure the winner does not cause harm elsewhere, such as bounce rate, page speed, refund rate or lead quality. If a variation improves sign-ups but lowers the share of people who become customers, you want to know before rolling it out.

A worked example

A small agency notices that its contact page gets plenty of visits but few form submissions. Session recordings show visitors scrolling through a long form and leaving at the phone number field. The hypothesis: because many visitors hesitate to share a phone number, making it optional and explaining how the details are used will increase completed forms. Version A keeps the form as it is. Version B makes the phone field optional and adds a one-line reassurance under the button.

The team calculates how many visitors each version needs, runs the test for three full weeks and finds that version B increases submissions by a meaningful margin without lowering lead quality. They roll it out, log the result and plan the next test on the headline above the form. Nothing here required advanced tools. It required a clear problem, one change and patience.

Tools and setup

Many platforms offer testing features, from website builders and email tools to dedicated experimentation software. Choose one that fits your traffic and technical comfort, and check that it does not slow your pages or cause flicker, where visitors briefly see the original version before the variation loads. Confirm that conversions are tracked correctly for both versions before you begin, and run a quick test to make sure each variation displays properly on phones and desktops. Google Optimize has been discontinued, so check that any tool you use is currently supported and privacy-compliant.

Testing beyond pages

The same thinking applies to emails and ads. You can test subject lines, send times, offers and calls to action in email, or headlines, images and audiences in paid campaigns. The rules are the same: one clear change, enough volume, a fixed duration and a metric tied to the real goal. Learnings often transfer between channels, so a headline that wins in an ad may deserve a trial on the landing page.

What to do with the result

When a test finishes, do not simply declare a winner and move on. Ask why it might have won. Check whether the effect was the same on mobile and desktop and across traffic sources. Consider whether the result surprised you, and what that says about your assumptions. If the test was inconclusive, that is still information: the change probably does not matter much, and you can spend your effort elsewhere. Then plan the next test. Small, steady gains compound over months and give you a clearer picture of your customers than any single big redesign.

Privacy and ethics

Testing involves showing different experiences to real people, so keep it honest. Do not use fake scarcity, misleading countdowns or hidden costs, even if they win a test. Respect cookie and privacy rules where you operate, explain data use in your privacy notice and avoid collecting more information than you need. A tactic that increases short-term conversions but damages trust costs more in the long run than it earns.

Segments and personalisation in tests

An overall winner can hide differences between groups. New visitors may respond differently from returning ones, mobile users differently from desktop users and paid traffic differently from organic. Once a test has reached its planned sample size, look at the main segments to see whether the result is consistent. If a variation wins on desktop but loses on mobile, you may want to use it only where it helps. Be careful not to slice the data so finely that random noise looks like a finding; treat segment insights as ideas for a follow-up test, not proof.

Multivariate and split URL tests

A standard A/B test compares two versions. A split URL test sends visitors to entirely different pages, which suits major redesigns. A multivariate test changes several elements at once to see how they combine, but it needs far more traffic and is rarely appropriate for small sites. For most small businesses, simple A/B tests on the highest-traffic pages are the best place to start.

Building a testing culture

The greatest benefit of testing is not any single win. It is a habit of asking, "How do we know?" Encourage everyone on the team to suggest ideas and record the reason behind them. Celebrate tests that disprove a favourite idea, since they save you from a costly mistake. Share results in plain language, with screenshots, so that decisions are transparent and lessons are easy to reuse. Over months, this culture shifts a business from arguing about opinions to learning from customers.

A simple test plan template

1. Problem: what the data shows and where.
2. Hypothesis: what you will change, why and what result you expect.
3. Primary metric and guardrail metrics.
4. Audience, page and devices included.
5. Sample size and planned duration.
6. Screenshots of each version.
7. Decision rule: what result leads to shipping, stopping or retesting.