Conversion Rate Optimization (CRO) is a research process with the goal of increasing your bottom line by improving your website. If you are serious about CRO, then make sure you add these 3 tactics to your everyday procedure.

1. Develop a Test Hypothesis

Screen Shot 2015-01-16 at 2.16.50 PM

Testing without a hypothesis is merely playing an expensive game of guess and check. To be frank, running the split test is the easiest part. Your success is dictated by the work you put into developing a solid test hypothesis.

The hypothesis is the second step in your split testing process. First, you have to identify a test page, then you can start researching what changes will improve the page. Your test hypothesis should look something like this:

I believe that changing A for visitors B will make C happen.

Here’s a hypothesis in context:

I believe that displaying the particular product that visitors spent the most time looking at for all traffic sources will increase product sales
(Click here to see this in action)

Selecting element "A" requires detailed web analytics analysis, qualitative data sources, and a bit of creativity.

It’s important to always have your customer in mind. This is why you need to identify the visitor type. Visitor types include new visitors, return visitors, return buyers, etc… The more you understand your visitor type, the more you can optimize for their needs.

Ideally, we hope that C always happens, but even the best testers get it wrong! Don’t worry if your test doesn’t make the gains you expected. If your test fails, look at your hypothesis and try to figure out why! Optimization is an iterative and evolving process, which is why your split tests will fail from time to time.

2. Design Tests to Better Customer Experience

It would do all marketers well to understand that the process of conversion rate optimization (CRO) is intimately tied to improving both the user experience and the customer value. Inane tests and conversion hacks have no place in your optimization tool kit.

Sure, you can use some black-hat conversion tactics, also described as ‘Dark Patterns’ by Dr. Harry Brignull. However, these tactics only provide short-term wins that are not scalable.

While developing your hypothesis you should ask a second question before launching a campaign:

Does this improve the customer experience?

If you answer ‘no’, find a way to make it better. Website visitors are looking for a quick solution to a particular problem or need – give this to them.

For example, I was on the Mass Health Connector site, and I really just wanted their fax number. I know, nobody faxes anymore, but that was the quickest way to send my information since they don’t take email attachments.

Anyway, the fax number was not listed on the site – despite the instructions to fax my proof of residency. I picked up a phone – gasp, I know – and called their service department. When I finally got through, they told me to mail it.

This is a great example of a terrible customer experience.  I was looking for an instant resolution to my problem, which was something their website was unable to provide. When I finally called them and wasted time in the queue I was told to use an even more archaic method.

To anyone who is worried - I resolved this issue by Googling the fax number, which I found in the body of some PDF.

3. Create a Test Schedule

A testing schedule is a must! Before you launch a test, you need to know how long you plan to run the test. Don’t ever call your test before or after this date.

I know plenty of testing tools will tell you that your test is complete, even if you don’t have a large enough sample size. This is because the statistics are not put in context; the numbers are in a vacuum and prior to applying the math the numbers check out.

Below is an example of a test that shouldn’t be called yet. There are two main reasons, 1) the confidence level is too low – aim for 95% 2) the test was only live for 5 days.

Screen Shot 2015-01-16 at 2.24.17 PM

Even if the confidence level were at 99%, I wouldn’t call this test until it reached my calendars end date. Why? Time is itself a major variable during tests and people act differently on different days. Just run an analytics report on conversions per day and see!

Screen Shot 2015-01-16 at 2.25.37 PM

Convinced? Good! Now there are a number of variables to consider when estimating your test duration.

  1. Expected Percent Lift – A larger lift requires a smaller sample size. Smaller expected lifts need more data to even identify the lift. Use conservative estimates!
  2. Number Of Daily Unique Visitors – The number of visitors will dictate how long it takes to run your test. More visitors also help increase the tests statistical power.
  3. Desired Confidence Level – A higher confidence level requires a higher sample size. Stick with 95% for now.
  4. Number Of Variations – More variations require more traffic – keep it simple.

When you have this data, toss it into a duration calculator. For a test with two variations, I recommend GPower 3. There are also simpler duration tools on most testing tech sites. For this example I’ll use VWO’s test duration tool:

Screen Shot 2015-01-16 at 2.26.54 PMNote: Remember to complete the week; this test should be scheduled to run for a total of 28 days.

Adding these tactics to your conversion rate optimization procedure is definitely easier said than done, but that is what the New Year is for. Start 2015 with a clean slate and optimize the right way.


Named one of the top 25 most influential CRO professionals, Justin Rondeau has spent his entire career working on optimization campaigns and has helped train some of the leading optimization teams at Fortune 500 companies. Rondeau has run hundreds of tests for both B2B and e-commerce brands and has analyzed 3,000+ split tests across virtually every industry.