Optimizely
The Optimizely app empowers editors and content managers to run experiments on structured content in a simple way.
Why Contentful and Optimizely?
Contentful and Optimizely work together to facilitate experimentation without the need for developer involvement. This means that once it is setup, editors can run experiments on content using the Contentful web app and do not need to ask for code changes to clients. This enables a powerful and flexible workflow for editors and content managers.
Getting Started
In order to successfully integrate Optimizely into your content model, we will cover the following topics:
Prerequisites
Terminology
Installation and configuration
Usage
Integration with your front end
FAQ and troubleshooting
Prerequisites
The prerequisites to run experiments with Optimizely and Contentful are as follows:
A Contentful account with a space for your website.
An Optimizely account with a Full Stack project.
A developer to integrate the initial solution with your front end.
Terminology
We will start by explaining term definitions.
What is a variation?
A variation is simply one possible option or variant in a test. If you are A/B testing, you will test two different options: option A and option B. Translated to real-world terms: if you have two variations of a button, you may want to see which one gets more clicks.
What is a reference field?
A reference field is Contentful specific. It is a field within a content type that doesn't hold its own value but instead points to one or more entries. In the image below, we have a content type called page. Page has a reference field called "component". The "component" reference field is a list of items (content types) that can be found on a page.
What is a variation container?
A variation container is a content type created by the Optimizely app which is used in place of a value in a reference field. In the image below, we've replaced our Call To Action (CTA) entry with a variation container that holds two different CTAs.
This is how the Optimizely app transforms your content model and prepares it for experimentation. As noted in the "What you will need" section, you will need a developer to handle the new response from Contentful.
Install and configure the Optimizely app
Step 1 - Install the app
Optimizely will ask you to sign in and authorize access to your account in order to connect it with Contentful.
Select an Optimizely Full Stack project you wish to use for your experiments.
NOTE: if you don't see a list of projects, this means that you do not have any projects starting in Optimizely. You will need to start a project and create an experiment first before continuing.
Click Install.
Step 2 - Configure the app
Now that the app is installed, it is time to set up an experiment. We are going to walk through an example setup where we run an experiment on the CTA button of a page.
We will enable experimentation on the component reference field by enabling it in the Optimizely app. Clicking the "Add content type" button will present a modal where we will pick the page content type and enable "component".
As you'll remember from our example above, "component" is a reference field:
In Contentful, it looks like this:
Usage
Now that we have the app installed and configured to experiment on "component", we will edit our content to use the variation container. Again, the variation container is a custom content type that is installed by the Optimizely app.
In the flow shown above we do the following:
Add a variation container to our "component".
Pick the CTA experiment from the dropdown menu in the Optimizely app interface.
Pick the existing "Buy Now!" CTA as our control group.
Create a new CTA entry with the value "20% Off!" and use that as our variation.
Publish the new CTA entry.
Publish the variation container.
Remove the old CTA block.
Publish our page.
Editors can easily swap out values and turn them into variation containers like in the example above. This makes it easy to take any reference field which is used to display content and change it into a variation container that displays content based on an experiment.
Integration with your front end
Variation containers are a new content type that is introduced into the Contentful response. They are containers that point to two or more actual values that we'd like to test. We now need to now integrate our front end with Optimizely and the variation containers. At this point, we need the help of a developer to change how our front end works to accept and programmatically handle variation containers.
From here, we need to take a more technical dive into the setup of the client and how it integrates with Optimizely. The guide below uses the same CTA example we have been following so far. It is beneficial for developers to get acquainted with the content we outlined above before moving on to the technical guide below.
The goals of the setup are:
Enable server-side experimentation without pushing new code per experiment.
Separating concerns: controlling content in Contentful and experiments in Optimizely.
Speed: server-side selection of variations for fast delivery and to avoid the "flash of content".
How the Optimizely App changes the Contentful API response
As mentioned above, the Optimizely app creates a new content type: the variation container. Let's look again at the CTA example that says "Buy now!":
Using the Optimizely app, the same content model is transformed. Where we originally received the CTA of "Buy Now!", we now are receiving a variation container holding all the different possible CTAs:
The variation container is simply a content type that nests the possible values for the CTA and holds metadata about the Optimizely experiment and the variation names.
Using Optimizely to pick the right variation
We are going to create a pseudo-backend that takes the Contentful API response and uses the Optimizely SDK to determine which variation to show to the user.
Let's start by looking at the variation container JSON response returned by Contentful. Notice that the content type exposes a meta, variations and experimentKey property.
Optimizely variation container JSON response
{
"sys": {
"space": { ... },
"id": "41nZggHEplcBOsrPXLOEU",
"type": "Entry",
"createdAt": "2019-07-17T14:32:24.306Z",
"updatedAt": "2019-07-17T14:32:24.306Z",
"environment": { ... },
"revision": 1,
"contentType": {
"sys": {
"type": "Link",
"linkType": "ContentType",
"id": "variationContainer"
}
},
"locale": "en-US"
},
"fields": {
"experimentTitle": "CTA Variation Experiment",
"experimentId": "15249150297",
"meta": {
"control": "2hoYOxnZVV3Imvry1DhmtG",
"variation": "5IlB5PCsca3zZnzlVd7WvI"
},
"variations": [
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "2hoYOxnZVV3Imvry1DhmtG"
}
},
{
"sys": {
"type": "Link",
"linkType": "Entry",
"id": "5IlB5PCsca3zZnzlVd7WvI"
}
}
],
"experimentKey": "CTA_test"
}
}
Before we can choose which entry to pick out of our variation container, we need to see how to get the right variant from Optimizely. Below is an example with a hardcoded test to get a variation for a user.
Optimizely variation test
import Optimizely from 'optimizely';
import datafile from './optimizelyDataFile';
import { getUser } from 'user';
const optimizelyClient = new Optimizely({ datafile });
const user = getUser();
const variation = optimizelyClient.activate(
'CTA_test',
user.userId
);
// variation => 'variation'
In the code above, Optimizely has determined that this user should see the variation option. This is a value we can get from the variation container meta field.
Now we will update our code to include pulling the variation container itself and using its values to populate the Optimizely experiment.
Optimizely pull variation container
import Optimizely from 'optimizely';
import datafile from './optimizelyDataFile';
import { getUser } from 'user';
import sdk from 'contentful-sdk';
const optimizelyClient = new Optimizely({ datafile });
const user = getUser();
// we are using the id of the variation container from the JSON sample above// In a real world implementation the ID would be set dynamically through e.g. a slug.const variationEntry = await
sdk.getEntry('41nZggHEplcBOsrPXLOEU');
const variation = optimizelyClient.activate(
CTA_test
variationEntry.experimentKey,
user.userId
);
// variation => 'variation'
const ctaEntryId = variationEntry.meta[variation];
// ctaEntryId => '5IlB5PCsca3zZnzlVd7WvI'
We now have the entryId of the CTA content block we want to display! This means that the content we should show is "20% off!" CTA from Fig 1.2 above. From here, you should be able to make minimal changes to your display logic in order to show the correct entry.
Important notes about this example
In this example we used userId. Optimizely requires that you identify the current user so they can determine which experiment group to place them into. This ID should stick with the user and properly identify them throughout different sessions. If you use a random number as an ID, your A/B test will not work correctly and the results will be meaningless.
A datafile was used in the example above to create the Optimizely client. This is supplied by Optimizely and will be unique to your project. You can read more about the data file here.
Only published content on Contentful will be exposed in the variation container. Using a Contentful environment, you can test how a variation container works before promoting your experiment to production.
Full Stack Experimentation vs Feature Experimentation
To learn about what needs to be considered when migrating projects from Optimizely Full Stack Experimentation to Feature Experimentation, refer to the migration guide by Optimizely.
FAQ and troubleshooting
How does the Optimizely app affect my environment?
The app needs to create a new content type called the variation container. The variation container is a custom type that makes use of a custom editor interface inside the web app and collates data from Optimizely and Contentful.
Where can I activate my experiments?
In Optimizely. We want to keep a clear separation of concerns. That means Contentful controls the content and Optimizely controls the experiment. The Optimizely app provides deep links on the sidebar into the Optimizely dashboards for easy access to your experiments where you can set up and launch experiments.
Why is my app configuration not being saved?
You must remember to click "Save" at the top right of the Optimizely app in order to save the full configuration.
Why does one or more of my variations not show up?
When you pick variations inside of a variation container, you will need to ensure that they are published and not a draft. Furthermore, you will need to ensure that your variation container itself is published for it to show up in the API response.
How can I define a fallback or default value for my experiment?
This is up to you. However, a default value or fallback is usually the control group. If you have a certain audience that you don't want to expose the experiment to, you will need to ensure your front end can handle the scenario of falling back to your control group.
How do I target a certain audience?
This is done in Optimizely. You will need to provide a unique value (usually a user ID) to let Optimizely know who is currently seeing an experiment. If you need a custom solution, you will have to provide more data points in order to correctly target your audience. In the case of excluding an audience, see above about defining a default or fallback value.
What do I do after an experiment is over?
There are a few options available. Currently, Optimizely offers the option to automatically start showing the winning variation after your experiment has gathered a meaningful data set. This means you would leave your variation container in place and allow Optimizely to always choose the winner.
Another option is to manually remove the variation container and replace it with the value that has won the experiment. This may be beneficial as you won't need to rely on Optimizely to serve the right content anymore.
Does uninstalling the Optimizely app break my experiments?
Your existing experiments will still run. We never tell Optimizely what to do with your experiments, we simply read experiment names, variation groups, and experiment traffic data. However, the editor's experience when dealing with experiments inside of the Contentful Web App will be highly degraded. Variation containers will still exist but the custom editor they use will be uninstalled.
Why does the variation container not show up as an option in my reference field?
Your reference field may have specific validations enabled. This means you will need to enable the variation container as a possible valid content type for that field. This can be done in the app or by finding the specific content type and changing the validations to include the variation container.