A critical aspect of successful personalization is the ability to test multiple experiences for different audiences and quickly identify the best performing option for each audience. Marketers can do this with targeted A/B testing, which lets you test customer experiences live and, once you have sufficient traffic, rapidly identify the most effective content, offers, messages, and more. Here are three of our top A/B testing tips to guide you to delivering successful personalized experiences:
1. Select goals carefully and examine results of all business goals when testing
Choose metrics that relate to your business goals. When optimizing for one metric, be aware of the impact on other important goals. The purpose of a campaign may be to increase average order value; however, a campaign targeted at this goal could end up causing a reduction in conversion rates and have an overall negative impact on revenue per visitor. Make sure you define all of the business goals in your system that can monitor campaigns for impact to all goals.
2. Understand factors that can impact testing
A common mistake is to ignore external factors that influence the outcome of a test. Tools often take the “all other things being equal” approach. However, all other things are almost never equal. Identify external factors that can impact testing results and examine outcomes with that in mind. Examples include: new product releases, changes in sale prices, new acquisition campaigns, design changes, out-of-stock alerts on popular items, and seasonal changes.
3. Consider statistical significance when drawing conclusions
Testing requires adequate samples in order to be sure that the results are meaningful. If you split your audience too much or have a large variation in user behavior you may have adequate data to provide confidence. Your testing tool should include a continuous confidence measure based on the data gathered.
Targeted A/B testing is a process that is simple, but requires sufficient attention to detail. In order to have results that are going to help your business, you need to run tests properly. Make sure you understand all your goals and all of the factors that can affect your tests. And make sure your sample sizes are sufficient so you can be confident in the results and make sound decisions about improving your business. Follow these three simple steps, and you’ll be well on your way!