The Art of Conclusive A/B Testing

The ability to perform A/B tests on ad descriptions and landing pages in paid search is truly a beautiful (and powerful) thing. If you are not testing ad descriptions against one another for conversion and ROI, then you are missing out on a huge opportunity to gain more sales and leads for your company within your given budget.

However, a lot of times it is tempting to throw 4, 5, 6 or more different ad descriptions into the search engines’ systems and test them all at once. Yet the key to drilling down to conclusive evidence of what does and does not work in terms of driving more leads or sales to your website is testing 1 element at a time. There IS a reason why they call this A/B testing – not A/B/C/D/E/F testing!

Take this example. Say you set up 3 different ads for testing in Google AdWords:

Suppose that at the end of 2-3 weeks the 1^st ad significantly outperforms the others in terms of conversion rate. Was it the ‘5 Diamond’ designation? Was it the fact that the rate was included? Was it that it stated ‘downtown’? You really can’t know for sure because no single variable was isolated. Each ad differs too much from the others and therefore you simply can’t draw any great conclusions from this test.

Instead, what if you were set this up on a ‘testing schedule’ of sorts. For testing in Week 1-2 (assuming this is enough time to gain actionable data… but definitely run the ads for longer if needed):

(note that the only variable tested is the use of a room rate in ad description)

At the end of Week 2, say the first ad with the rate emerges as the clear ‘winner’. You can now conclude that the inclusion of rates in the ad description is a factor that contributes to more online sales.

Week 3-4? Stick with the rate in ALL descriptions now that you know it is important for sales, but move on to a new isolated variable. Say that this particular hotel has a number of honors to work with as key differentiators and now you’d like to see which resonates best with searchers:

Week 5-6? Perhaps the hotel wants to test the use of special offers such as ‘Free Breakfast’ v. ‘4^th Night Free’. Week 7-8: maybe 2 different landing pages.

By patiently remaining systematic in this approach and letting the numbers provide the proof, you’ll have solid evidence of which ad elements and landing pages produce the highest ROI for your company.

One important last word of note though! In Google AdWords, much to the conversion marketer’s chagrin, Google will by default start delivering the ad that has the highest click-through rate – not the ad with the highest conversion rate. If your goal for paid search is to drive the most leads or sales, not the most clicks, then I HIGHLY recommend that you turn off the setting that so-called ‘Optimizes’ your ad distribution.

To do so go to your Google Account and:

1. Select ‘Edit Settings’
2. Under Advanced Options select the radio button to ‘Rotate: Show ads more evenly’

Happy Testing!