End of an era for Google Optimize: What's next?

From September Google has decided to sunset both the free and paid versions of Google Optimize. After this Google will not provide any A/B testing and will only integrate with third-party solutions.

Google Optimize has been the go-to solution for many companies starting out in A/B testing, with some estimates of nearly 60% of web domains engaged in A/B testing using Optimize. Its easy integration with Google Analytics and Google Ads and no cost option meant it was a great way for companies to experiment with testing. But through its limitations it also constrained companies’ experimentation strategy, especially around volume and scope of testing. Google admitted this is one of the key reasons for the sunset: 

"Optimize, though a longstanding product, does not have many of the features and services that our customers request and need for experimentation testing."

The sunset of Google Optimize presents a great opportunity for companies to rethink their experimentation strategy and lean into the exciting opportunities alternative testing software provides.

So which alternative is right for your organization? That depends on several factors, such as the software features, privacy, integrations and cost, as well as what you’re looking to achieve through experimentation. We’ll review some of the available tools for A/B testing and compare their features.

TL;DR

Before considering other A/B testing platforms check whether your Content Management System (CMS) already includes any A/B testing capabilities, and if so the scope (and possibly cost) of the A/B testing it provides. 

Similarly, companies using Adobe Analytics might want to first consider Adobe Target, before looking at other options, considering its simple integration with the rest of the Adobe universe.

For those who find their CMS doesn’t meet their needs and are using Google Analytics, it’s on to other platforms. If you’re just starting out in CRO testing, and have less than 50,000 site users each month, then VWO’s free starter plan is hard to beat. At the other end of the scale Optimizely or AB Tasty are great solutions for large enterprise A/B testing, especially with the discounts they’re offering for switching from Google Optimize. Whilst if you’re specifically looking for enterprise level personalisation then Dynamic Yield is worth researching.

We’ll go through each of these platforms in more detail below and we’d love to talk if you’d like to discuss A/B testing further.

What is A/B testing?

A/B testing, also known as Conversion Rate Optimization (CRO) testing, is the process of creating multiple versions of the same website, app or product feature to test the impact on users. It’s a data driven way of gathering user insights and continuously optimizing customer journeys which is primarily used to improve conversion rate and revenue/profit. Companies create a hypothesis on the effect of a change to the site or feature, test that hypothesis on live visitors to the site, and then measure the impact.

Platform Comparisons

We’ll discuss the different benefits of Optimizely, AB Tasty, Dynamic Yield, VWO, Adobe Target and CMS solutions. 

Optimizely

Optimizely is an enterprise level experimentation platform, offering client-side and server-side experimentation through its Web and Feature Experimentation products respectively. There are three tiers to the Web Experimentation product (Grow, Accelerate and Scale) and two (Accelerate and Scale) for the Feature Experimentation product. Each tier offers a different combination of features as standard, with further add-ons available at an additional cost. Optimizely is aimed at businesses which have around 500,000+ monthly active users to their site/app.

Software Features

• Compatibility: It can be implemented on any website, and you can check compatibility of test variants with eighteen browsers.
• Creating variants: It includes both a visual editor and code editor.
• Privacy: It can be integrated with consent management tools to ensure cookie compliance and provides an option to anonymize IPs. 
• Speed: All customers benefit from Optimizely’s Performance Edge, which hugely speeds up experiments, and makes it the best in the industry for performance.
• It doesn’t offer heatmaps or user recording, but it does integrate with tools, such as Fullstory, which do.

Integrations

Optimizely has a partnership with Google Cloud, and therefore integrates with all Google products, including GA4 and Google Ads. It also integrates with Salesforce, Fullstory, Hotjar, Segment, Contentful CMS and others, to enable you to fully understand and personalise your experiments.

Cost

Optimizely is a premium product, used by more than 20 of the Fortune 100 companies, and you should expect to spend a few thousand pounds per month on an Optimizely licence. Pricing varies with the number of monthly active users as well as the product plan.  However, they are offering substantial discounts for clients moving away from Google Optimize, through platinum partners such as 26. 

Overview

If you’re a large e-commerce retailer, with more than 500,000 users a month, with an existing CRO strategy built around Google Optimize and Google Analytics then Optimizely could be ideal for you. The speed of its testing would really open opportunities to scale experimentation and its server-side testing makes experimenting with product price and customer journey easy to do. 

AB Tasty

AB tasty is an all-in-one CRO platform which is also pitched at enterprise level A/B testing, but one that can cater to slightly smaller clients than Optimizely; from around 200,000 monthly active users. It also offers testing across web, app and IoT devices and has both client-side and server-side options. Similarly to Optimizely, it costs thousands of pounds per month, but they are offering a one-time discount to clients who move away from Google Optimize.

Software Features

• Compatibility: It works with most web frameworks including Single Page Applications.
• AI at its core: From traffic allocation to segmentation based on engagement and content interest, AB Tasty uses AI to improve conversion.
• Creating variants: It includes a widget library developers can use to make changes, such as adding social proofs, countdowns or banners, without writing code However the visual editor can be seen as less robust than Optimizely’s.
• Privacy: You can easily adjust cookies to only drop upon implicit consent or change the cookie storage method.
• It includes additional CRO tools such as heatmaps and user surveys

Integrations

It integrates with GA4, both in terms of reporting experiment impacts and with importing audience segments from GA4 into AB Tasty for experiment targeting.

AB Tasty has a web of technology partners and therefore integrates with most things, including Adobe Analytics and Salesforce DMP. Where there isn’t a ready-made integration there are options to build customer connectors via the AB Tasty API or their Universal DataCollect feature. 

Cost

Marginally cheaper than Optimizely, but still a premium product costing thousands per month. Pricing depends on monthly active users and on the features included in the package.

Overview

This is the best option for mid-sized enterprises who are looking to push forward their CRO testing strategy. The all-in-one nature of AB Tasty could mean other products such as Hotjar can be retired, limiting costs and the number of third parties involved in site testing.

VWO

VWO seem very interested in snapping up smaller companies who are currently with Google Optimize and they’ve launched a free starter package for businesses testing less than 50,000 users each month. This is ideal for those dipping their toes into CRO testing or who have a smaller online audience. They do have other packages for web testing (Growth, Pro and Enterprise) aimed at sites with increasing numbers of monthly users, as well as separate products for app and server-side testing. They’re also offering discounts on migration from Google Optimize 360.

Software Features

Features depend on the precise package you choose, with additional cost for deployment of test winners, personalisation, or user behaviour tools like heatmaps, session replays or on-page surveys.

• The free starter package includes:
  • A/B and split URL testing
  • Visual and code editors
  • Basic targeting by URL and device
  • Traffic allocation
  • Add new site elements (such as text, lists, images, buttons or links)
• Creating variants: Both visual and code editors are available in all web testing packages, but the elements library and customizable widgets are only available in the paid packages.
• Privacy: As with other platforms you’re also able to anonymise IP addresses and adjust behaviour based on users’ cookie consents. 

Integrations

VWO provides detailed instructions on how to integrate with GA4, with or without Google Tag Manager, which enable reporting on the performance of your experiments in GA4 reports. For other third parties VWO often offer plugins (such as for WordPress and Shopify), and it even connects up to other analytics tools like Adobe Analytics and Heap and user behaviour tools such as FullStory.

Cost

Unlike lots of other platforms VWO is very open about its pricing, clearly setting out costs by product, package and monthly tracked users. Whilst it has a free package available for smaller testing functions, for companies who are testing hundreds of thousands of users each month the costs for an enterprise package are very similar to packages available with AB Tasty or Optimizely.

Overview

If you’re looking for a free tool and you have less than 50,000 users each month then VWO could be a good option, however the free package is quite limited. It’s worth considering your ambition for your CRO function before signing up with VWO, it could be very easy to start off with their free package but find you need to add on paid packages quite quickly.

Dynamic Yield

Dynamic Yield was named as a Leader in Personalisation Engines by Gartner for the fifth consecutive year in 2022. It’s focused on AI-driven personalisation at scale, and provides machine learning powered content automation, segmentation and recommendations. It’s again aimed at enterprise level clients rather than those starting out in A/B testing or personalisation. 

Software Features

• AI and Machine Learning: Dynamic Yield includes options such as using predictive targeting to select audiences to target and contextual next-best action suggestions to improve optimisation. 
• Compatibility: Compatible with the most used Google, Mozilla, Apple and Microsoft desktop browsers as well as Samsung, Android and Apple mobile browsers and iOS and Android apps. Dynamic Yield is also fully compatible with Single-Page Applications (SPA).
• Variants: The no code visual editor is one of its core strengths, but it does also support code editing in HTML, CSS and JavaScript.
• Privacy: Dynamic Yield provides an ‘Active Cookie Consent’ mode which ensures users are not served personalised content unless they actively provide consent.

Integrations

Dynamic Yield lists a host of supported integrations, including Google Analytics, Google Ads, Shopify, Salesforce and FullStory.

Cost

Because it’s hidden behind a sales request, we can’t provide much information on the costs of this tool.

Overview

If you’re looking specifically for enterprise level personalisation, rather than optimisation agnostic to the user’s characteristics, then this could be the right tool for you.

Adobe Target

If you’re a large enterprise already enveloped in the Adobe universe, then this is probably the platform for you. Through the Analytics for Target integration personalisation features of all of Target, Adobe Analytics and the Adobe Experience Platform Real-Time Customer Data Platform (CDP) are enriched. The integration with the Adobe Real-Time CDP especially enables a much broader range of first party data to be used within experimentation than previously possible. Client-side, server-side and hybrid deployments as well as App and Email implementations of Target are possible.

Software Features

• Real-time personalisation: Segment users in-page through combining historical customer profile data with in-session behaviour for personalised experiences at each visit.
• Automation of traffic allocation to highest performing variants, along with a variety of testing types (A/B, multivariate and contextual bandit option).
• Compatibility: Target delivers variants to most browsers across Windows, Mac and mobile/tablet devices
• Variants: The Visual Experience Composer (VEC) is Adobe’s visual editor to create and test personalisation, and custom code insertion is possible within and outside the VEC, however it also provides a Form-Based Experience Composer. There is also a specific VEC for SPAs.
• Privacy: The way in which Target reacts to user consent choices is controllable through Adobe Experience Cloud.

Integrations

You’ll want the rest of the Adobe ecosystem to make the most of Adobe Target, especially Adobe Analytics and Adobe Experience Manager, and ideally the Real-time Customer Data Platform. Through Adobe Experience Manager many other integrations, such as Salesforce CRM data, are available, however Target can be integrated with 3rd parties directly through its Data Providers capability.

Cost

There’s both a standard Target and a Target Premium offering. Target Premium requires an additional license to Target Standard, but it includes most of the machine learning and AI functions such as Automated Personalisation, Auto-Target and Recommendations.

Target pricing is based on the product options, experimentation volume and number of channels targeting is delivered by (web, mobile, app, email, IoT etc). Specific pricing for a business is only available through to Adobe.

Overview

If you’re already part of the Adobe ecosystem and looking for an enterprise solution this could be a great solution. It offers both basic A/B testing but stretches to full AI-supported personalisation as well.

CMS solutions

It’s worth considering whether the same tool that you use to deliver your content also has an experimentation function. This is likely to be more limited than the platforms created specifically for A/B testing, but it may meet your needs. As with any other platform you should consider cost effectiveness, compliance with data privacy and cookie consent management, whether the tool integrates with every touch point of your customer journey and with your analytics and reporting solutions.

Looking for support?

There are lots of different A/B testing tools out there with different specialisations. To choose the best fit, you’ll need to consider cost, privacy compliance, software features, integrations and ease of use. Our data science and solutions teams can work hand-in-hand to run technology and platform reviews to ensure you have the right technology in place to meet both your business and customer needs, now and in the future. 

With the technology foundations in place, we can help you design and implement experimentation and CRO programs that will drive performance and help you reach your business goals. Want to chat? Get in touch here.