SPLIT TEST ACCELERATOR

"Discover How Chet Day Improved His Landing Page By 258.8% With The Split Test Accelerator, Using Only 209 Visitors In The Main Test, And Only 172 Visitors In The Confirmation Test!"

First, I want to say that you shouldn't expect to be able to have a test quite this good very often. I've never had one go quite this quickly, myself. Typically you need anywhere from 2,000 to 10,000 visitors for a good test. But I want to show you this case study for two reasons.

It shows you what's possible.
It shows you how you can get results very quickly if you take care to design good options at the beginning.

I can't show you the exact page Chet is testing, or the exact alternatives he's testing, because he wants to keep that private (very understandably!). I can only tell you that Chet's landing page pre-sells an affiliate product, and then sends the visitors to a merchant site.

But I can show you which kinds of factors he tested, and statistically how diverse they were.

First, let's look at the results of Chet's confirmation test:

(NOTE: the "confidence" measurement is the confidence you should have that the option can beat the leader in the long run. In this case, you should essentially have 0% confidence. Or, put another way, you should have greater than 99.95% confidence that the leader should continue leading in the long run.)

You can see that Chet's original landing page got just under 20% of visitors to click through to the merchant site.

And you can see that Chet's new page is getting just over 70% of visitors to click through.

With so few visitors in this confirmation test, these numbers aren't completely solid. I advised Chet to end the test at this point, because the main point was made very well -- the new page is very definitely better than the original. The statistical signficance is through the roof.

But the exact difference would fluctuate some if Chet were to run the test longer. It might end up 30% to 60%. Or it might end up 15% to 80%. We don't know. But it's very clear that in the long run the new page is much better than the original. So there's no point sending any more traffic to the original at this point.

Now, can you imagine going from 20% to 70% on one of your landing pages? It's a true breakthrough for this landing page, and for Chet's profits.

I've seen Chet's landing page, and the options he tested, and the new page doesn't use any gimmicks to get the visitor to click through.

Now, let's look a little more at how Chet got these results.

Here are the "ad by ad" results and the "factor by factor" results for Chet's Taguchi test:

Notice that Chet's test had 7 factors in it: "Main Headline", "Opening", "Offer Includes", "Testimony", "Limited Offer", "Order Link", "P.S."

There were 6 versions of the headline, and 3 versions of each of the other elements being tested.

A total of 4374 combinations were being tested.

And notice that these combinations were tested by running just 18 different ads.

And, amazingly, he sent only 10-13 visitors to each of these ads in the test. (again, getting results with so few visitors is not typical, but can be done)

When the results were decomposed for each factor, he had about 35 visitors for each headline, and about 70 visitors for each opening, offer include, testimony, limited offer, order link, and P.S.

So, why can he end the test
with so little traffic?

It's because of the variation in his results.

Three main factors created almost all of the improvement: The headline, the opening, and the Limited offer. And most of it came from the headline.

Now, truth be told, I think Chet got a little lucky. I would have run the Taguchi phase of this test a little longer. But the improvement indicated by the headline alone was encouraging enough that Chet decided to switch to the confirmation mode at this point. And it paid off in spades.

So what about those headlines? Was the first headline just an obvious dog? It looks like it with the stats we have before us.

It's kind of hard to tell in hindsight, but I'd be willing to bet that if I mixed up the headlines and quizzed the audience about which headline was the "dog", most people would get it wrong.

You can't always tell when you have a dog on your page -- or a winner, for that matter.

So you need to test. And . . .
The more factors you test,
the more chance you have
of finding a big winner
like Chet found.

Chet is testing 7 factors at at time. Others test 20 factors at time. This is much better than testing one at a time.

Testing one thing at a time, it might have taken Chet 7 times longer to find his big winner. And, actually, since most of the factors would have had to run longer before reaching statistical significance, Chet might have had to send 50 times as much traffic to this page to find the winner he found.

If you would like to start searching for breakthrough results 10 times faster, take another look at the Split Test Accelerator. It really can make your business grow many times faster.

Thanks for reading this.

Sincerely,

Jim Stone, Ph.D.

(c) Copyright 2004-2008 by Jim Stone Enterprises. - All Rights Reserved
20027 13th Drive SE Bothell, WA 98012
Disclaimer¦ Contact ¦ Free Adwords Accelerator Spreadsheet ¦ Free Multivariate Testing Course ¦ Free Confidence Measuring tool ¦ Split Test Accelerator ¦ Blog ¦