A/B Testing Guide

What is A/B Testing?

A/B Testing is a testing method used to compare 2 versions of the application display, with the aim of being able to determine which type is the most superior.

If not done correctly, tests can fail to produce meaningful, valuable results and can even mislead. Generally speaking, running controlled experiments can help organizations with:

  • Solving UX issues and common visitor pain points
  • Improving performance from existing traffic (higher conversions and revenue, improve customer acquisition costs)
  • Increasing overall engagement (reducing bounce rate, improving click-through rate, and more.)
A/B Testing has 2 different options, namely (A or B) which are presented to the user. These 2 versions of choice can be monitored which one is the best, whether version A or version B.

Purpose of A/B Testing


A/B testing allows making the most efficient and effective decisions, because it is based on "User Actions" rather than words or observations of users.
A/B testing can be implemented to create the desired goals so that they can be completed more quickly and without problems.

Winner version is the winner between version A or B. It can be concluded which version is the best. Winner version can be implemented into the final product or can be used by all users.

MVT(Multivariate Testing)

Multivariate Testing is an A/B testing method but it has more than 2 variants. For example, we have versions 1-6 on the settings page, the test we use is MVT. Because with this MVT can measure the effectiveness of various design combinations.

A/B Testing at Instagram

One example of a company that uses A/B Testing is Instagram where the first Winner version is compared to the other Winner versions. In the end, you can get really good results and the best performance from the user's choice.

On the Instagram Register page, it can be seen that the A/B testing of version 1 focuses on the Button with the SignUp label and version 2 on the Button with the Continue label. The final result of A/B testing above version 1 as Winner version.

"Observe what people do, not what they say"

Some examples of A/B tests:


The application of A/B Testing is very diverse, for example the color difference between the CTA, the difference in the headline text, to the layout. In essence, a/b testing allows you to compare the responses from users regarding a product offered through simple measurements and practical ways. So that you can get which product has the best performance.

More resources

  1. A/B Testing 
  2. Exploiting A/B Testing for Fun and Profit
  3. A/B Testing Tutorial
  4. A/B Testing on Amazon, Etsy, AriBnb, and other large companies.
  5. What are Multivariate tests?

A/B Testing Guide
Nanang Prasetya 4 April, 2022
Share this post
Sign in to leave a comment
5 Stages of Design Thinking