Increasing Revenue: The basics of Split Testing
This topic, often called A/B testing is very large and complicated. It’s so big that there are companies devoted just to split testing; but for a majority of us out there hiring a company to help us with this is not an option. In this brief article I plan on summing up the basics and giving you enough information to get started.
Split testing from Wikipedia
A/B testing is a method of advertising testing by which a baseline control sample is compared to a variety of single-variable test samples. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.
Employers of this A/B testing method will distribute multiple samples of a test, including the control, to see which single variable is most effective in increasing a response rate or other desired outcome. The test, in order to be effective, must reach an audience of statistical significance.
So for our purpose this means we have one ad and we modify this ad in some way such as its color, size, location, etc. We then display both ads one at a time and see which ad gives the better CTR*, the original ad or the modified one. We then take the better ad and again modify one thing on that ad (color,size,location,etc..) and again see which one performs better. If the original ad performs better then you will want to change something else and try that test again. If the modified one works better you then change something new on that ad to see if you can get even better results. By doing this over and over and changing things as you go the perfect ad for your site will slowly emerge.
To see how this works in the real world let’s pick an ad. Since a lot of people use AdSense lets go with them. We first need a base case, we’ll use the basic AdSense ad Google assigns you when you sign up. Now we’ll pick one thing on the ad and change it. You will only want to change one thing on the ad at a time so you will have a better idea of the effect the change had on the ad. If you change multiple things and revenue increases you won’t know which one of the changes increased revenue so stick with changing one thing at a time. For this example I’m going to change the text color. Below is the original AdSense ad, this is the ad as it comes from Google.
The next ad is my modified one, as you can see all I’ve done is change the text color a little bit, in this case to match my sites colors a little better than the original.
Now for this simple case I’m going to run the ads for one week(seven days) each, first the original then the modified version. It’s important to try to keep as many variables the same as possible while your test two different cases. For example if you start the first ad Sunday morning at 8am you will want to start the second one at the same day and time. Also, don’t run the first ad the week before Christmas and the next one a week after and expect the results to be accurate; use your head when testing. Here are the results from the test.
Original Ad: 1669 hits, 18 clicks giving this ad a 1.08% CTR(Click through rate)
Modified Ad: 1456 hits, 27 clicks giving this ad a 1.85% CTR
So with some simple math we can see that the modified ad gave us a better CTR. We can then take this ad and test it even further by making the modified ad our new base case and start changing that ad. Maybe you’ll want to stick with the text color until you’ve maxed out it’s potential or maybe you want to jump in and start changing the font, border colors or size. It’s up to you and as long as you stick to this basic principle of testing one variable at a time you should continue to move your ads in the right direction.
I hope this gave you an idea of how to start split testing. Again, this is a very basic example but you have to start someplace and this is a pretty straight forward approach on how to maximize your text based ads. This can be carried over to many different sections of your website, for anything from improving navigation to deciding which links to put in the footer. Split testing is one of those things that never really ends, good luck and feel free to post up any questions in the comments section.
*CTR : Click through rate. Your click through rating can be calculated by taking the number of clicks that ad got and dividing it by the number of impressions the ad got then multiplying by 100.
Hey Ryan, thanks for the tip to your blog! I’ve never seen it before. Nice to meet you!