What is an A/B Test?

For you, A/B testis a pompous, smoky term that doesn’t mean much? We show you otherwise!

Digital marketing is a constantly evolving discipline that requires continuous adaptation. This cannot be done without the implementation of testing and analysis of results: Welcome to A/B Testing!

A successfully tested tool

The principle of this tool is to offer your target audience several versions of your site, your application or your digital advertising campaign.

The idea is to compare in real time the results of versions A, B or even C, to see which one offers the best results. For this, each version must meet the same objective (conversion, traffic or reputation) and must be taught during the same period so that the results can be compared fairly and the best conclusions can be drawn.

In concrete terms, this is how it can be in the context of improving your website’s home page:

1. You will create several versions of your home page, with different variables.
2. When users arrive at your website, they will be segmented, which means that one part will see version A, and the second part will see version B.
3. Then comes the step of analyzing behavior. Which version was the best? Did people spend more time on your site, leave their contact information, or click more often on version A or B?
4. Finally the statistical analysis of the yields are taken, which will allow you to make a decision to improve the website conversion.

As you may have understood, it is primarily a means of measuring the effect of changing certain variables on your website.

The key to a conversion optimization campaign is also a good segmentation of your audience; it can be interesting to segment your audience into groups in a homogeneous way through well-defined criteria: purchase behavior, demographics, connection devices, browser used, etc.

Conversion optimization is important because most of the time, conversions do not occur by themselves.

Let’s say you have the “perfect” product; it’s cheap, it’s something we all need, and it’s something that generates a lot of interest. Still, you’re likely to encounter a number of potential problems:

• Your users may not understand how to buy your product.
• Your users may not realize that your product is for sale.
• Your users may lose interest or postpone their purchase.
• …..

On one level, conversion optimization is about making people want to buy your product (or commit to your brand), but more importantly, it’s about giving them the power and opportunity to actually do it. No doubt about it. No friction.

A/B tests for what uses?

This method is particularly effective in the following situations:

• Conversion Optimization Campaign (CRO):
  Not only can you work on improving the conversion rate of your website or application, but it can also be used to improve the ergonomics of your site or application.
  
• Emailing campaign:
  You can completely divide your database into two segments and send them the same email by changing only the content or the title. You can also try out the best time for your mailings, the images or the call to action button you use in the body of the mail.
  
• Digital advertising campaign:
  You can find out which ad headlines, main texts, images, calls to action or a combination of the above work best for your target audience. In addition, you can experiment with various Facebook/Instagram/Google audiences and ad locations to find out who your perfect audience is and which locations get the best results.

If the results are more conclusive in one version than in another, you can apply the one with more performance for the next marketing actions.

Each test will allow you to know your audience better and act accordingly.

What is the difference between an A/B test and a multivariate test (MVT)?

When Test A/B will offer a version A of your website to a user with one variant, and a version B to another user with a different variant, the multivariate test may propose several changes simultaneously in the same version A banner, a title, a description or a video, we will try to test a set rather than an enhancement.

In summary, the MVT allows you to test several combinations of changes in the same experiment. The interest is to be able to test the performance of a complete combination of optimizations (this title, with this banner, with this footer), instead of improving a single element as proposed by the A/B tests.

The main weakness of the MVT test is that it will be necessary to have a BIG volume of traffic to your website in order to have reliable statistics.

A/B tests, source of knowledge

The more you use A/B testing, the more you will know your customers and know how to reach them more effectively. The possibilities are endless to allow you to continually improve the way you approach your main goal, both in substance and form.

However, you cannot afford to create A/B tests before you are clear about your goals, the friction points on your page, and the indicators you will use to measure your A/B test results. The A/B test is one more element of the Conversion Optimization (CRO)process.

The advantages of A/B tests are numerous; they allow you:

• Find out which version and variant responds best to your audience.
• React immediately by stopping the versions that have the lowest performance, therefore, invest your marketing budget only in the version or versions that work best.
• Measure in real time the impact of the improvement and the behaviour of the users.
• Adopt best practices for your next campaigns based on real and reliable data.

Famous A/B test stories

NETFLIX:A/B testing is the key to the success of many giants, including Apple, but also Netflix. In 2015 at SWSX2015, Netflix’s vice president of product innovation, Todd Yellin, explained it very well in an interview. “A/B test is not just about showing two versions of the same product and watching what happens. It’s a matter of experience … What matters is not who people are or what they tell you, but what they do! “. These extensive tests have allowed to choose the final designs of: the Netflix interface on Apple TV in 2009, and the “children” part. More about Netflix and its A/B testing strategy: https://medium.com/netflix-techblog/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15

GOOGLE: Marissa Mayer’s attention to detail and perfectionism led her to take A/B tests to the extreme! First engineer to join Google in 1999 (20th employee of this future giant), she spent 13 years in various positions. To validate the final colour of the search engine pages, at that time I had tried no less than … 41 shades of blue!