Nurturing a testing culture in some companies can be a challenge — you may have to inspire, convince or cajole the nay-sayers to get them excited about conversion optimization.
That’s why we’ve written this essay, to help you get started. Below are five easy tips for implementing a successful conversion optimization strategy — plus a bonus tip for those who think they don’t have enough data to measure.
5. Find Small Optimization Wins to Build Support
You may have heard some conversion advocates say you should “just start with testing anything!”
I hate to break it to you, but that’s bad advice.
At WiderFunnel, we work with a lot of clients who have tried this guns-ablazin’ approach in the past and have ended up bringing in an expert after losing organizational support for their optimization program. If you start with a poor process or mistaken assumptions, you can waste a lot of time and momentum before you get the results that can help your business.
We get it, though — resources can be limited, while expectations are not. So how do you do it right?
Start with simple tests. When I say simple, I meansimple. But don’t worry — just because a test is simple doesn’t mean that it won’t have an impact. Some of our biggest wins in terms of insights and revenue have come from tests that didn’t even have to go through our design department. Don’t discredit a simple copy idea because your boss might think, “That’s all?”
Small changes can work. We recently ran a test on DMV.org where we changed the copy “Enter Your ZIP” to “Please Enter Your ZIP.”This test ended up increasing conversions by over 12%. Who knew, all you had to do was be polite!
Pro tip:Digging into the experiment results by segment showed that some states responded better to "please" than others. Alwaysconsider your message segmentation.
The beginnings of a CRO program are fragile, and one mistake or serious bug can get the whole program cut in a heartbeat. At WiderFunnel, we know that modest changes can get big results, so don’t be afraid to start with the basics. As your boss and colleagues gain confidence in the program and you grow confident with your process, the time will come to begin looking at the next steps. Remember, conversion rate optimization is a marathon, not a sprint, so walk before you run.
4. Prove Monetary Value Ahead of Time to Gain Organizational Buy-In
If you can show your boss that you have grown the business by 15% (with statistical significance!), you won’t have to advocate to her ever again —shewill be你的advocate. It takes a while to ingrain a testing culture into the company, but once it’s established, you begin to answer really hard questions that can change the strategy of the company. A few tips:
- Start by holding meetings to get ideas from all stakeholder departments; everyone will have questions that they want answered about what the customer’s real preferences are.
- 专注于你的同事的问题和设计经验eriments that can answer them. Keep in mind, not all questions are created equal. Even if a colleague has a great question, it is your job to weigh out the priorities and test the right questions first.
- There is no better way to get organizational support than by showing people their ideas in action. Start with the executive leadership. There is always a strategy to test the answer to any question, so be confident you can find it.
Once you start testing, don’t be shy about sharing positive results. If you do have a loser — which you will — show how you have taken that insight and applied it to your next test. If you have a great insight that the sales team might be able to use to its advantage, tell them! If you learn something that marketing could use in their next email promotion, share that with them too!
Get the whole company to share questions they want answers to, and be the person to deliver them those insights.
For more on this topic, check out thesestrategies for becoming an optimization champion.
3. Prioritize New Tests Emphasizing Potential, Importance & Ease
When you’re testing, order and priority matter. Starting with the wrong pages can waste tremendous effort, even if the result is positive. Prioritizing website tests is easy with thePIE framework: Potential, Importance and Ease.
Ideally, you’re analyzing your pages to find where you’ll get the most significant gains with the least amount of effort. You want to take a step back from your business and pick the spots that are going to really make a difference.
Start by breaking your site down into “like-pages” or “zones” — bundles of pages that you can run the same experiment on simultaneously. Examples include:
- Product detail pages
- Checkout funnel
- Site-wide navigation
- Product listing pages
- Homepage
Once you compare traffic and conversion rates for each zone, you should start to develop an idea of which areas will deliver the biggest impact.
To put this into perspective, if you have 10,000 visitors to your homepage, but it only has a conversion rate of 1%, it’s not going to be as effective a test-site as a product page that only has 5,000 visitors but a 4% conversion rate. The product page has twice as many total conversions as your homepage, despite receiving half the traffic. So, a 10% lift here is going to deliver more of an impact for your business than 10% on the homepage.
When you can only run a few tests at a time it is important to make sure you are spending your time focused in the right places.
2. Limit Variables to Improve Learning From Testing
One of the most common mistakes we see when we start working with clients is that there is a lack of understanding around the impact of variables.
When you’re testing, everything else you are doing matters. Some variables to keep in mind are:
- Competition:Your competitors could have changed their offering, or you may have offered discounts or successful promotions that affected competition.
- ob欧宝娱乐app下载地址市场趋势:There are up months and down months in the market and for your business — these trends may be affecting your results.
- Other tests:Running multiple tests simultaneously on your website will pollute your data.
- Press coverage:Were you in the news lately? This can affect a visitor’s perception of your brand both positively and negatively.
如果你有一个运行在一系列的实验same time, the insights can no longer be trusted. Multiple tests should be mutually exclusive because, if a visitor is entered in more than one test, it will skew the final results. How do you know which variation contributed to the conversion?
Don’t try to compare your pre-test conversion rate to your conversion rate while testing. All the external variables could have changed during the test. If you see that your conversion rate has been going down, consider that you could be even worse off if not for the lifts you’re delivering. Trust the data — it doesn’t lie. Results from A/B tests are much more reliable than "before and after" comparisons.
Aside from external variables, you also want to limit the variables within a test. Testing only one variable at a time is the best way to gain insights. We know this is sometimes impractical, so you need to prioritize which variables are most important to isolate for learning, and which should be clustered to get faster lift. This will help you maximize for both revenue lift and insights.
1. Wait for Statistical Significance to Avoid False Positives
我们知道和其他人一样令人沮丧的be to have a test declared a statistical winner only to see it begin an epic slide into negativity.
Fortunately for the people analyzing the data, things are starting to look up. VWO’s “Smart Stats” and Optimizely’s “Stats Engine” are market-leading stats tools designed to help marketers avoid declaring false positives. Here's an overview ofhow VWO and Optimizely's new stats models work.
Not every tool calculates significance the same way so it’s important to understand exactly what your tool is telling you. Get familiar withOptimizely’s Stats Engine; this will give you great insight into the differences between confidence levels, statistical significance, chance to beat control and so on.
The most important thing to try to avoid is declaring a false positive (implementing a winner that is really a loser). Here are a few tips you can apply to help you avoid this set back.
- Run your tests in full week periods. Often times, each day of the week has dramatically different conversions patterns. Sometimes you will run a test on weekdays and see a significant winner, but when you come into the office after a weekend, everything has changed. Running your tests in full week cycles can help you make sure you are getting the complete picture.
- Don’t just look at the numbers, look at the trends as well. Although metrics are great for calculating conversion rates, they are not the best at helping you make predictions. When a test is truly finished, all the variations will be pretty much flat-lined. If that conversion line is still jumping up and down, then it’s a good idea to take the confidence level with a grain of salt. Fortunately, the new results technology coming from the market’s leading tools is taking this into account. Never again will you have to wonder if your “statistically significant” results are really statistically significant.
- Make sure you have at least 100 conversions per variationbefore taking the confidence level seriously. A lot of testers get caught calling tests that are 20 conversions to 10 because there is 100% improvement (with confidence!). That being said, if your business is okay taking on some more risk, there is nothing wrong with acting on trends.
If you have the traffic and internal patience to run tests out to significance, does that mean it is always the right call? Nope!
Although significance is of paramount importance for testing, it’s not always the right decision to wait for it. It may sound counter-intuitive, but there are many times when calling the test without significance is actually the smarter move.
This is what separates the pros from the amateurs. As you gain testing experience, you begin to identify what “Winning,” “Negative Learning,” and “No-Effect” tests look like in a shorter period of time. This allows pros to be able make calls quickly and get to the big wins faster.
The most important thing to remember is that, if you call a test before it reaches significance, make sure to put an asterisk by those insights. Sometimes you will pause a variation that is losing by 20 conversions because you want to send more traffic to a more positive trending variation, not because you truly believe the idea is a loser. This distinction is a critical one, and once you gain experience you can make these calls with more accuracy for the business.
Bonus Tip: What to Do If You Don’t Have a Lot of Traffic
If you don’t have a lot of traffic, accept that you will not be able to get all the isolations you are looking for (and that’s okay). Try testing concepts to see where there is the most sensitivity and follow-up in that area.
For example, if you want to test improving your form UI, do an entire variation where you test all the UI on the page (leaving the copy and design). If you want to test the design, change the design elements only (leaving the copy and UI). If you want to test the copy and VP points change them all but leave the two sections above. If you learn there is no reaction to your UI changes, then you know that is probably not the most common barrier to conversion on your site and you can focus you efforts elsewhere.
