Start with Radical Redesigns
Are you thinking about running an A/B test to compare the performance of different elements on your page? That is great. Maybe you want to see whether a page with an image performs better than one without an image. Or you are curious to find out if the wording on your main call-to-action makes a difference. All good ideas!
But before you begin making small tweaks to your landing pages, make sure you have a good starting point. Launch a multivariate test, a Treatment page that is radically different from your Control page. Then continue improving with smaller iterations by running A/B tests.
Be Prepared for Validity Threats
Unfortunately, the validity of our live test was compromised because the the Control received more traffic than the Treatment. Some tweets to the offer included a direct link to the Control page, thus bypassing the splitting mechanism. If you are running similar experiments, be aware of such threats of validity and try to avoid them. As MarketingSherpa’s Austin McCraw wrote, “There are potentially real-world extraneous factors that can completely invalidate your results.”
The Offer Matters the Most
Even if the validity of our test was invalidated, we still wouldn’t have seen a significant difference between the two conversion rates. As Austin noted, “if the overnight drop hadn’t occurred, the Treatment would have outperforming the Control by 6% with a 96% statistical confidence level.”
On average, HubSpot’s landing pages convert at 35% and we attribute this success to the high motivation of our visitors. They arrive at our pages willing to fill out the form because they really want to receive the offer. So make sure you create strong offers to begin with!