Example 1: Looping User Journeys
On a major e-commerce site, anecdotal evidence suggested customers sometimes had to enter their postcode (UK Zip Code) multiple times before placing an order. Thorough testing did not create any examples of this, and the problem was initially put down to user error.
When UserReplay was adopted a flag was created for any user journeys with multiple postcode entry. A report of these journeys was created, ordered by the number of times the postcode was entered. One determined customer had entered their postcode 27 times before it had been accepted. This report showed that multiple postcode entry on a single journey was happening hundreds of times per day, and that there was a significant negative correlation between the number of times the customer had been returned to the postcode box, and conversion. The problem was real and well worth fixing.
Example 2: Login Failure
UserReplay can proactively monitor KPIs so that severe conversion problems can be identified and dealt with extremely quickly. For e-commerce sites with login, any problems around logging in create a severe and immediate impact on conversion. This means login attempts and failures are important KPIs to track.
One customer had a typical login failure rate of 400 per hour. These reflected user error and most typically forgotten passwords. UserReplay noticed this rate rapidly accelerate to 7000 failures per hour representing about 1/8th of all user journeys.
None of the affected journeys could convert, creating a plunge in conversion rate. Investigation of a sample of the impacted journeys showed they all related to one backend server. This server was immediately taken out of use, restoring conversion.
Further investigation showed that a password encryption software update had been wrongly applied to this server, this was rolled back and the server placed back online. The problem was costing £50,000 per hour as it occurred during peak hours, and therefore resolving the problem within 30 minutes saved significant sales.
Example 3: Combating Fraud
One multi-channel e-commerce customer was encountering an increased incidence of fraud after launching a “collect from store” option. The customer established a set of criteria that represented reasons for suspicion. Most of these criteria did not represent fraud, but where a number of them happened in a single journey this was sufficient reason to investigate further. UserReplay is an open system and through an API, SQL queries can be developed. The customer quickly established a bespoke report, which runs every 15mins and furnishes the fraud team with an ordered list of the most suspicious journeys.
Despite successfully passing the core fraud detection system, many of these were blatantly fraudulent. Store security is then advised and is waiting for the fraudster when they arrive. Evidence from UserReplay’s visual Session Replay is subsequently used in prosecution. Given the open nature of UserReplay the reports can be refined to test new hypotheses and keep on top of the fraudsters.
Example 4: Disappointing Mobile Conversion Rates
One customer noticed a significantly lower rate of conversion for mobile, following a website upgrade. No error messages or problems were identified in the customer’s Application Performance Management tool, but the poor conversion was clearly visible in Web Analytics and was costing over £5k per day in lost sales. Flags were set up in UserReplay to identify these failed conversion journeys. Interestingly the Android conversion rates were broadly comparable to web, but the iOS rates were much lower. Many of the failed journeys were stopping at the same point.
A sample of the affected user journeys were replayed in a number of iOS devices up to the problematic point. It was then discovered that on the iPhone 4 a vital button was being rendered outside the visible page, preventing further progress with the user journey. This problem was fixed and the conversion rates for mobile immediately increased significantly.
Example 5: Customer Dispute
One disgruntled customer recently managed to contact the CEO of a UserReplay customer, accusing him of defrauding them. The accusation was that discounts offered in a promotion were stripped from their purchase after they had been calculated and shown at checkout. The customer claimed they were charged an incorrect amount and were demanding immediate compensation. The CEO called the IT Director to ask for an explanation of exactly what had happened in the case.
The IT Director searched UserReplay and found the disgruntled customer’s journey. On visual session replay it could be seen that the customer had removed an item from his basket, right at the end of the process, and had been legitimately disqualified for the discount. The IT Director was able to explain that the site had worked correctly, and exactly what had happened within minutes of receiving the call from the CEO. Useful lessons were learned. An investigation in UserReplay showed that this scenario happened around 100 times a day and hence had the potential to cause wider problems. Warning messages were added to advise customers when they disqualified themselves for promotions by item deletion.