5 A/B Testing Tools for Making Data-driven Design Decisions

A/B testing is becoming more and more common as teams realize how important it is for a website’s success. The Web is a huge, competitive marketplace with very few (if any) untapped markets, meaning that being successful by offering something unique is rare. Much more common is that you’re competing for the business of your customers with several other websites, so attempting to convert every visitor into a customer or upselling/cross-selling your services better could make all the difference to your bottom line.

Due to this, the market for A/B testing tools and CRO (conversion rate optimization) tools is growing exponentially. But choosing one can be quite a time-consuming challenge, so in this article I’ll compare the best A/B testing tools to help you decide which is most suitable for you or your team. If you want to get up to speed with A/B testing and CRO, check out our recent Introduction to A/B Testing article.

TL;DR: A/B testing is about experimenting with visual and content changes to see which results in more conversions. A/B testing often follows usability testing as a means of testing a solution to a flaw in the user experience identified using metrics like bounce rate in an analytics tool like Google Analytics, and thanks to the depth and quality of A/B testing tools available now, A/B testing is accessible to designers as well as marketers and developers.

1. Optimizely

• Summary: the leading A/B testing tool in 2017
• Price: contact sales team
• Who it’s for: designers, marketers and developers collaborating

Optimizely is one of the leading — if not the leading — A/B testing and CRO tools on the market today. It offers analytics tools to suit users of all levels, and a multitude of A/B testing tools. (You could think of it as the Google Analytics of A/B testing, with a much simpler user interface.)

Consider this scenario: You have an ecommerce store built with Magento. You’re aware that in certain cases it may benefit stores to add a one-step checkout solution instead of the standard multi-page checkout, but you’re not sure if your store fits that use case. You need to test both options and compare the results with/without the one-step checkout experience. You know that running two versions of the checkout simultaneously requires changes to the code, which is a complex matter.

With Optimizely, you can send a certain amount of your users to a totally separate checkout experience to collect conversion data. If the experiment yields negative results, you delete the experiment and the original checkout web page still exists and works fine. No harm done.

With their Web Experimentation tool, which offers an easy-to-use visual editor to create A/B tests without requiring a developer (optional), the ability to target specific user types and segments, and create experiments on any device, Optimizely has all your bases covered.

Although you can run A/B tests without a developer, your variations can be more targeted (for example, your variations can go beyond color, layout and content changes) if you have the skills and/or resources to develop custom experiments with code. By integrating your A/B tests into your code, you can serve different logic and test major changes before pushing them live.

Also, if your product extends beyond the web, Optimizely works with iOS, tvOS and Android apps. Optimizely’s Full Stack integrations makes it possible to integrate A/B tests into virtually any codebase, including Python, Java, ruby, Node, PHP, C#, Swift and Android.

2. Google Optimize

• Summary: A/B testing that seamlessly integrates with Google Analytics
• Price: free
• Who it’s for: anyone, being the easiest to learn of the bunch

Google Optimize is a free, easy-to-use tool that integrates directly with your Google Analytics Events and Goals to make A/B testing quick and easy! It’s ideal for traditional A/B testing, focusing on comparing different CTA (call to action) elements, colors and content.

Developers aren’t required for implementing Google Optimize, since it’s as simple as adding a line of JavaScript to your website and then customising your layout with the visual editor. With this you can change the content, layout, colors, classes and HTML of any element within your page.

It’s not as sophisticated as Optimizely, since it doesn’t allow you to create custom experiments with code/developers, but it’s free. It’s great for those starting out with A/B testing.

For each Google Optimize experiment, you’ll need to specify which Google Analytics Goals or Events will be the baseline for your A/B tests. For example, if you were A/B testing a product page, you could use an “Add To Basket” event that you’ve defined in Google Analytics to evaluate which of your variations converts the best. The Google Analytics report then gives you a clear indication of which variation converts best. It’s ideal for those on a low budget!

Just don’t get carried away, as Google famously once did, by testing 40 different shades of blue to see which converted best!

3. Unbounce

• Summary: A/B testing and conversion tools for landing pages
• Price: From $79/month
• Who it’s for: marketers looking to boost conversions on landing pages

Unbounce focuses on landing pages and convertible tools. Convertible tools use triggers, scenario-based overlays and sticky bars to A/B test offers and messages to learn when, where and why your visitors convert. An example? If a user tries to leave your site, they’re shown a discount code in a modal or a sticky header, and a test will determine which is more effective.

Landing pages can be an amazing way to validate your ideas, build excitement around a new product, and/or re-engage dormant customers. The problem with them is that they can result in false positives. If you get very few conversions you may feel like your idea is invalidated or demand for the new product doesn’t exist, when in reality users were just unimpressed and/or unconvinced by the landing. Unbounce helps you to determine what your landing is missing.

While you can choose from over 100 responsive templates designed for many markets, goals and scenarios, and then customize it with your own content using their drag and drop UI, you can also integrate Unbounce with your own design, making a terrific solution for designers and marketers who need to collaborate. Unbounce also works with Zapier and Mailchimp, so data can be transferred across the other apps and tools that marketers use.

It’s basically Optimizely for landing pages.

4. AB Tasty

• Summary: A/B testing with CRO features like Exit Intent Detection
• Price: $249+/month
• Who it’s for: designers, marketers, developers and stakeholders

AB Tasty is known for its deeper customizations. Rather than sending X visitors to Test A and Y visitors to Test B, AB Tasty allows you to target visitors based on many different factors, such as user data, user behavior, geolocation, or even the weather in their current location.

The visual editor is also easy to use, offering a widget library to add completely new elements to your variations rather than awkwardly customizing existing ones — allowing you to focus more on the targeting and results than tweaking the design. Think of it as rapid iteration, but for A/B testing. It’s very useful for early-stage websites finding their way.

5. Crazy Egg

• Summary: A/B testing combined with heatmaps and screen recordings
• Price: $29+/month
• Who it’s for: anyone, but with a slight focus on designers

Crazy Egg is somewhat new to the A/B testing market, having initially built its audience on heatmap tools and screen recordings. However, the addition of A/B testing tools to a usability testing app makes total sense. If usability testing highlights the problem, then A/B testing can help you narrow down the solution. Also, Crazy Egg’s pricing is very attractive!

Like most other services, Crazy Egg offers a visual editor for creating experiments, making the tool relatively accessible to anyone. It really focuses on making the creation of A/B tests extremely quick and easy, boasting that tests can be created and live within minutes.

In short, CrazyEgg offers you the what, the where, the why, and the ability to test a solution.

Conclusion

With these tools, you can A/B test and begin to improve conversions across practically any scenario and website. You don’t need to be a developer to take advantage of A/B testing tools, but if you do have a developer, you can create really powerful, custom experiments easily.

Whichever tool you use, be sure to establish your baseline performance using Google Analytics to know your conversion rates before you make any changes, and wear each 0.1% improvement as a badge of honor in your quest to maximize conversions! Good luck!