The Analyst’s Diary: Online A-B CX Testing

The Analyst Diary, August 2017. Hello diary. As you know, it’s been a busy couple of weeks with me zipping between France, the US and even the wilds of the north of England, experiencing the Great British Summer!

It was good to catch up with one of my retail clients following up on some earlier work we did around credit card payment failure scenarios.  Previously we had configured events to report on customers experiencing payment gateway failures.  That made it easy to look at as we had the graphs and data at our fingertips.

Case Study: Market America The Key To Delighting Your Customers

In the world of Online Customer Experience, it is often worth investigating a little further and in this case, it exposed some interesting behaviours.  It was clear that very few customers were retrying following a failure and were instead abandoning their purchases.  So, we focused on those abandoning customers who only tried once, and a quick look at the replay highlighted an obvious problem: the failure message gave no option to try again, and gave no guidance on what to do next.  Pretty much the only option visible was to click back to the home page and go through the whole process again.  Of course, some decided to take this option (clearly desperate to buy from this retailer!) but many just left.

It was agreed that simple A-B CX testing would be performed to see how presenting another chance to enter card details (a different card or re-enter the same) following a failure would impact abandonment.  The results were clear: when presented with a retry option, most customers had a go. The impact on conversion was also clear: On average, the number of people successfully checking out following an initial card failure was more than double those presented with the original (lack of) options.

The CRO team were really pleased because, without UserReplay, they just wouldn’t have realised what was happening.  The original discovery of the payment obstacle and its quantification in UserReplay really allowed them to focus their A-B cx testing efforts for maximum return.  They admitted that many previous A-B tests were based on no more than a “hunch”.  

Conversion and revenues are up! Customers achieve their purchase! Everyone’s happy!

Anyway, I wanted to write this one down, as really pleased with my days’ work.

Here’s to a happy Online Customer Experience.

Until next time….

 

The Analyst’s Diary is a collection of scenarios encountered by UserReplay’s Online Customer Experience analysts.  Names and some details may have been changed to protect the innocent!