Video A/B Testing: how they work and what are the factors to take into consideration

In marketing, you always learn from experience.

You start by setting your objectives over the short, medium, and long term. 

You try to isolate a precise profile of your target audience, even better if you can divide it into very specific targets.

You study the context, the emotion that is relevant for you and for your target audience.

You establish the budget and estimate the return on investment.

Having fixed these elements, you then design a strategy, which is always a large mosaic that is made up of several pieces, an overall design that is the sum of many actions to be put in place, in the right ways and at the right times.

But it doesn’t end there: a decisive step is still missing. 

The analysis of results: it’s from here that the “experience” we mentioned at the beginning takes shape and from which we can learn.

In short, it’s about measuring how each campaign performs, as accurately and broadly as possible. Here, you identify the strengths and weaknesses, understanding what worked and what didn’t. 

This triggers continuous improvement.

Now, let’s immediately ask a question that takes us right to the heart of this article: is this analysis phase always a conclusive moment? And, do you have to wait until the end of a campaign to analyze the results?

The answer is no. 

Limiting your analysis only to the end results would be a big mistake. 

Before launching a campaign and harnessing all its firepower, it’s a good rule of thumb to first do some testing and carefully evaluate these “intermediate results.” 

This is the case especially when it comes to video marketing.

The importance of testing in video marketing

Testing your marketing campaigns before moving on to a more massive launch phase is always a great idea. 

And why is this even more true when it comes to video-based campaigns?

First of all, cost is a big motivator. 

Creating a video campaign can require a lot of resources, time to invest, and skills that are not always available within an organization. 

Of course, video marketing always records excellent ROI. But if you can put a good testing phase in place, the returns will be even greater. It is a basic matter of optimization.

But as fundamental as this is, it’s not all.   

There is another element of video marketing that we want to focus on. 

Of all types of digital media, video has the highest data granularity. 

What does this mean? 

It means that video offers very in-depth information about how the audience interacts with a campaign. Here are just a few examples: metrics on the number of views, impressions, abandonment rate, viewing times, various metrics that measure engagement, and everything related to the click-through rate for your different calls to action.

In other words: a well-conducted analysis on your test campaign results can provide a very high-resolution snapshot of performance, audience reactions (broken down into the different targets and micro-targets), what works best, and what can be improved.

A great testing classic that is extremely effective? 

A/B testing.

Video A/B Testing: What it is and how it works

Video A/B Testing is a process of distributing two different versions of a video to evaluate its performance based on the behaviors and reactions of the target audience.

How does this work? 

Quite simply, it involves dividing your audience into two segments. 

For example, you would distribute the video with a specific thumbnail to one audience segment, while the second audience segment would see the same video with a different thumbnail. 

Which version has the best open rate? 

The data doesn’t lie: you’ll be able to choose the version with the most effective thumbnail based on actual results. 

So far, it’s all very basic. 

Of course, the deeper you go, the more complicated things get…and the more interesting, especially for those involved in designing marketing strategies.

First of all, it’s not enough to divide the audience into two clusters, 50/50. It depends on how the audience is composed, its characteristics, its variables; in other words, you need to have in-depth knowledge of your audience before you divide it (and this is possible with a deep and wide analysis of the available data). 

Moreover, Video A/B Testing can easily turn into so-called A/B/n Testing: in practice, the target audience can be broken down into even smaller segments than 50/50.

And finally, a decisive point.

We mentioned above the very high granularity of data that video provides. 

This also means that A/B testing can focus on several elements that always affect the same video, in some cases simultaneously.

Here are some elements that we feel are especially crucial.

Video A/B Testing: Key focus areas

Here is a list of elements that can be extremely decisive when conducting Video A/B Testing:

1) Title

It goes without saying that the title is very important: it’s one of the very first things a user notices and evaluates, especially if you post the video on social media. 

Moreover, it’s also a crucial aspect for search engines (thus for SEO marketing).

That’s why it’s a great idea to A/B test slightly or very different titles.

2) Previews and Thumbnails

There’s another element that is as important as the title. 

We’re talking about the video preview and the thumbnail, which is crucial for several channels, from Google to social, to messaging apps that allow you to copy video links.

Why is this such a crucial element? 

Because, together with the title, the audience forms an expectation of the video: in other words, what viewers expect to see in the video. 

In this way, it forms a hook to capture attention and stimulate curiosity. 

Finally, it provides an “at first glance” view that helps define the identity of the company or brand that produced and distributed the video.

There are a number of variables at play when choosing the video preview and thumbnail: the choice of images (with attention to the ideas they evoke), typography, colors, graphic layout, and much more.

3) Sound

So far we have focused on the visual aspects of a video. 

The sound aspects of a video may appear secondary, but it’s precisely because of this that they play a delicate and decisive role.

It is essential to pay attention to any background music: it can radically change the “atmosphere” of a narrative. 

Therefore, you’ll also want to pay attention to voice-overs and any other sound elements.

4) The first few seconds

The first 10 to 15 seconds of a video are extremely valuable. This is a well-known fact. It’s a matter of the attention threshold, which is becoming shorter and shorter as we are constantly bombarded with stimuli in today’s digital ecosystem. 

It’s here, in these 10 to 15 seconds, where the user decides whether or not to continue viewing, which determines whether or not they will reach the call to action and the end of the video.

Make sure that you run an A/B test of the same video with two different introductions to see which version provides better engagement and longer viewing time.

5) Length

There is no single answer to the question of how long a video should be. It depends on a great many factors: the campaign objectives (a video aiming at brand awareness, for example, has extremely different dynamics from one that promotes a single product or service), the target audience you’re addressing, the channel used through (TikTok is very different from your company website, etc.).

So, do some testing to assess the ideal length. But do it intelligently: you’ll want to take several factors into account, especially in terms of the distribution channels. 

Keep this additional awareness in mind: an excellent strategy is to start with a longer video and repurpose it for different channels and platforms (especially on social).

6) Call to action

The call to action is the crowning glory of every single video marketing campaign. Doing A/B Testing on this point, therefore, is not just important, it’s essential.

First and foremost, it’s a matter of focusing on where you insert the call to action: evaluate different positions, depending on the duration and storytelling of the video, and choose the one that guarantees the best performance.

Not only that: also watch out for the “form” of the call to action itself. There could be a huge difference between a “buy now” button and one that says “explore more”…don’t underestimate them!

In short: try, try, try. Use as many combinations as possible. And make sure you are guided by data every step of the way.