A simple effort at personalisation
This post first appeared here, on the Secret Escapes Tech Blog.
Personalisation is one of the things tipped to be massive in 2015. With so much data lying around about our customers, and their behaviour, we now have a great opportunity to optimise their experience by providing more relevant sales and information. Personalisation can be as simple as adding a name to an email subject line, or it can be as complex as completely changing a user experience of a website.
In 2015 I’ve been keen to experiment with personalisation of the web, and how we at Secret Escapes present our offers to our users. Currently our ‘homepage’ for signed in users is a list of sales, often 200 long. Whilst we curate and hand-pick every offer that features on the site, we don’t do a great job at creating a personal list for our users. Using Optimizely, and a hack-day, I was keen to implement something that would prove the concept and be sufficient for a very rudimentary A/B test.
There were multiple ways I looked at using data about our users, as well as how we may be able to quickly create some. I explored 3 different types of personalisation:
- Search terms. Can we personalise the site and increase the prominence of sales which match previous searches made by the user?
- Sale type, based upon most visited sales. We group and tag sales by location and sale type (All-inclusive, Spa). If a user shows a certain propensity per type, for example 80% of sales visited are London based, should we promote those sales?
- PPC keyword registration. When we acquire our users via PPC, they are often acquired via a specific keyword. For example “Spa weekend breaks”. If they are looking for these sales, should we promote them for the next x days?
I used Optimizely and Jquery cookie to set a cookie with the above information for the user. We set the Optimizely experiment to run for 100% of users, on the appropriate URLs. For search terms, I stored the last 3 search terms made by a user. For sale type, we incremented a cookie for each sale type match they made. And for PPC keyword we simply set a cookie which stored the keyword that the user reached the site on (this was extracted from a query string).
I then set up multiple Optimizely experiments, and used their targeting to enter users into the appropriate experiment. If the user matched the cookie, they were entered into an A/B test. The test promoted all relevant sales to the top of the listing page. For example, a visitor who signed up under the keyword of New York, would see the following page:
The complexity of this hack would lie in how we treat users who fall into multiple experiments. So for V1 of the experiment we opted to only run with a single cookie type, which was PPC keyword registration. This was the simplest of the implementations, as it wasn’t a dynamic or changing cookie. Once they had signed up it would remain the same until they expired.
Our initial results are incredibly promising, for the New York keyword we saw the following increases after running for 140 days:
Clicks to sale from listing page – +24.4%
Clicks to purchase page – +30.4%
Completed purchases – +210%
Other keywords, including “All-Inclusive” saw very similar results. Seeing significant results for clicks to sale from the listing page.
Whilst the numbers on completed purchases are relatively low, all of the KPI’s are positive and it’s something we’re soon to be extending even further across other keywords.
Naturally this type of personalisation isn’t scalable, however it’s enough to validate an initial hypothesis and spend further time investigating more robust solutions.
It’s worth considering the amount of users who will fit into each bucket. The more granular you get with your personalisation, the harder it is to validate the result as significant. We initially chose our highest performing (in terms of signed-up users) keywords, to test good levels of traffic.
Cookie compliance. The EU has specific directives on what cookies can be set without opting in, this needs to be taken account and if running this experiment in the EU your users may need to be aware if you are using cookies to personalise their experience