Test page identification is arguably the most important step in your A/B testing procedure. If you pick a test page that shouldn’t be tested, your results are going to suffer. The page you select for your A/B test should meet the following criteria:
1) The page should have enough traffic to test
2) The conversions on the page should be connected to long term goals, e.g., revenue, sales qualified leads (SQLs), etc.
3) The page should be a "high opportunity" page, i.e., not your worst performing page
If your page meets these three criteria, then you have a page worth testing!
I’ve said it once; I’ve said it a thousand times:
Not Every Page Is Testable
If you have a low traffic page, it might not be eligible for A/B testing. So take a look in your analytics platform and see if the page gets at least 2,000 unique visitors per month. (Note this is a bare minimum requirement.)
You need more than just eyeballs on your site. Site visits are just a heuristic; it’s better to identify test pages based on average conversions. Your page should get at least 100 conversions per variation. For larger stake changes, it is in your best interest to move the baseline conversion as high as 200-250 conversions per variation!
Let's play with these numbers a bit shall we?
Even at the bare minimum, it can be tough to generate enough traffic for an A/B test. My suggestion is to move up your funnel to a page that does have enough traffic to test.
As marketers, we love our metrics. Click rates, bounce rates, purchases, leads, they all fascinate us. Well they don’t fascinate a boss who is wondering why the 320% lift in clicks hasn’t made the company a single cent.
In the last section I gave a bit of advice for traffic-challenged sites. That advice comes with a big ‘but’ – and here it is:
When measuring micro-conversions as a progress indicator, make sure you can translate the micro-conversion to the bottom line, e.g., revenue.
When you select a test page, especially when you start your testing program, go for tests closer to the bottom of the funnel. Assuming you have the traffic to conduct these tests, the lower funnel pages make it much easier to attribute the test results with your major KPIs.
Sure, testing technologies allow you to track your tested visitors further down the funnel, but those results might not reach statistically significant results, i.e., a 10% lift in clicks on a top funnel page DOES NOT mean a 10% lift in sales or leads!
Moreover, there are many variables outside of the test page that could influence a visitor after they interact with the test page. So if you are just starting out, do yourself a favor – start looking for deep funnel pages.
The Worst Performing Page Isn't Always the Best Test Page
We all want to plug a sinking ship, but that doesn’t mean we have to waste testing resources on these poorly performing pages. A 10% lift on a page that is converting at .1% won’t help you as much as a 10% lift on a page that is converting at 1%.
Pick Your battles
Obviously, you should fix your bad pages, they just might not be a/b test worthy. Remember, not every page should be tested, but all pages can be optimized. Here’s an example:
Paula is running a PPC campaign and her landing page’s conversion rate is abysmal. She notices that the offer on the landing page does not match the offer promised in the ad.
Should Paula run a split test on her landing page to verify that it will increase conversions? No! She should make the change immediately and spend her time testing things that actually matter. Here’s an example of a test that matters:
Paula’s PPC landing page, because of best practices, included images of people on the page to ‘humanize’ the brand. Paula has been upset about her landing page’s performance lately and wanted to find a way to convert more of that paid traffic.
After running an eye-tracking test, she realized the faces are getting all of the attention – completely overpowering the call to action.
Should Paula run an A/B test on her landing page? Yes! She has hypothesized that a variation without the faces would increase conversions – she needs to verify this.
When you select your next test, please make sure that the page has enough traffic, your KPIs are in line with your business goals, and you aren’t just selecting the worst performing page.
Named one of the top 25 most influential CRO professionals, Justin Rondeau has spent his entire career working on optimization campaigns and has helped train some of the leading optimization teams at Fortune 500 companies. Rondeau has run hundreds of tests for both B2B and e-commerce brands and has analyzed 3,000+ split tests across virtually every industry.