Description
The purpose of an A/B Test is to compare the performance between two different versions of a marketing tactic. The A/B Testing template allows you to clearly capture the critical elements of your test prior to execution. It also forces you to consider why a potential A/B test is valuable and what you will do if the result of your test supports your hypothesis. A/B Testing (also known as Split Testing) is an increasingly popular method to make incremental improvements to advertisements, websites, and other customer touchpoints. Marketers now have the opportunity to run several A/B tests, often at the same time. Using a common A/B Testing template can help you to clarify strategic and operational details with your team.
Question
How are we planning to execute an A/B Test?
Steps
- To plan an A/B Test, you need to first have an understanding of what you want to improve and how you might improve it. Capture these background details and explain why the A/B test is worth investing in.
- You likely have a Control (existing Version A) and a Challenger (modified Version B) that you are testing. Write a hypothesis for your test by explaining why you expect the Challenger (Version B) to ‘win’.
- Document your test by describing your independent variable (what is being changed) and your dependent variable (what is being measured). How will your Version A be different than your Version B?
- Capture what and how you will measure the difference in performance between Versions A and B. This metric should relate back to your original reason for investing in the A/B Test. What are you trying to improve?
- Finally, capture how you will execute the test and the actions you will take based on the results. This is a useful litmus test for value - if there are no specific actions coming out of this test than it is likely not worth doing.
Considerations
- Make sure that you are not running so many A/B tests concurrently that you may impact your results.
- Continue to review trends in your marketing performance data to identify opportunities for new A/B tests.
- Only run A/B tests if you can achieve the sample size required for your target confidence level.
Reference
Bland, D., Osterwalder, A., “Testing Business Ideas: A Field Guide for Rapid Experimentation”, John Wiley & Sons, Inc, 2020.