Testing and conversion rate optimization (CRO) have been around for a long time. So most marketers — B2C and B2B alike — are already familiar with the importance of testing. They know why they need to test, the benefits that come from continuous optimization, and, at a minimum, the basics of designing successful tests. But testing can be more complicated than it appears. How do you decide what to test? How do you know when a test is done? What do you do if your results are inconclusive? If you aren’t an expert conversion rate optimizer, these answers are not obvious.
To give you a little motivation, we’ve compiled some testing tips from experts in the conversion rate optimization field.
Customer Research Will Tell You What to Test
The data tells you where the story is, not what the story is—it alerts you to an issue that you should be solving. In your customer research, look for answers that get repeated. That'll clue you into what your customers' main concerns and needs are. Then, go to your competitors' sites and collect what you're seeing from them, including the reviews they're getting. Gather all of that info and compare it, highlighting the main issues that come up over and over again.
Many people may know how to launch an A/B test, but most of them don't know what to actually test. By talking to your customers, you're essentially taking away the speculations and gut feelings and basing your decisions on what people are telling you. With that information in hand, you can start to test and fix any issues. Ultimately, if you haven't done the research, don't trust your gut feeling.
—Talia Wolf, founder and CEO of GetUplift (from the Wistia blog)
Pick the Highest Impact Pages — Not the Worst-Performing Pages
This is probably the most important step in your test creation process. If you pick a page that shouldn’t be tested, you’re going to have a bad test.
Tests should be selected based on opportunity. Remember, opportunity doesn’t mean fixing your worst-performing pages. It means testing pages that have the biggest impact on your bottom line.
In other words, the page you pick should have the potential for a big lift and should be directly connected to your key progress indicators.
One of my favorite identifying metrics is exit-pages. If you have a page with a high exit rate that isn’t the end of your page sequence, then you’ve identified a potential test page.
—Justin Rondeau, Director of Optimization at DigitalMarketer (from the DigitalMarketer blog)
Set Testing Timeframes to Account for Full Business Cycles
For some high-traffic sites, you would get the needed sample size in a day or two. But that is not a representative sample. It does not include a full business cycle, all weekdays, weekends, phases of the moon, traffic sources, your blog publishing and email newsletter schedule, and all other possible variables.
So for a valid test, both conditions – an adequate sample size and a long enough period to account for all factors (a full business cycle or, better yet, two) – should be met. For most businesses, this is 2-4 weeks. Always run tests full weeks at a time (stop tests at the 7, 14, 21 or 28 day mark).
—Peep Laja, founder of Conversion XL (from the Conversion XL site)
Test for Different Audiences
Most tests are run to determine the most effective experience for the least common denominator. Once a winning experience has been determined from a test, that experience is shown to everyone. But every visitor to your site is unique. They all have different preferences and in-the-moment intent. A winning experience may be effective for some percentage of visitors, but that means that you are still missing the opportunity to be relevant to the rest of your audience. Testing can tell you which product recommendation algorithm, homepage experience, CTA, etc. works better for a specific segment of visitors.
With that in mind, you should never react to negative or less-than-successful test campaign results without doing some additional filtering. Very often a test campaign that is performing worse than the control is actually performing better than the control for certain segments. For example, even basic filters such as first-time versus returning visitors, or organic versus paid traffic, can flesh out nuances in results across groups. These tests can help you target a campaign more effectively, leading to even more relevant, higher impact personalization efforts.
—Meera Murthy, VP of Strategy at Evergage
Don’t Forget that Circumstances Will Change
You shouldn’t assume that your winners will remain winners forever. Even if your website hasn’t changed, your users, and the way they access your site, are changing all the time. And, as a result, they may react differently than the population that was part of your initial test sample. New devices may change the way users browse your website. You only have to look at the way smartphones and devices are revolutionizing online retail to understand the impact a change in device can have on your conversion rates.
Re-test your winning elements to kick the tires and ensure the experience is still relevant. It’s also a great opportunity to test new and improved iterations of the old favorite.
—Suzi Tripp, Senior Director of Experimentation Strategy at Brooks Bell (from the Brooks Bell blog)
And Finally, Remember that Best Practices are Just Guidelines
If they are anything in conversion optimization, they are only starting points. You’ll never see conversion experts declare a generic color, CTA size and number of form fields as ingredients guaranteed to lift your bottom line.
Conversion optimization is an iterative cycle of hypothesizing, testing, learning, concluding and testing again. Always observe the environment under which successful testing studies were conducted. You’ll have better context, reasoning and learning from the positive results.
—Neil Patel, founder of CrazyEgg (from his personal blog)
Keep these tips in mind as you plan and execute tests to improve your site experience going forward. And to learn how Evergage can help you test personalized experiences for segments or individuals, request a demo today.