An e-Commerce site is a delicate balancing act. Inventory, pricing, content, CRM, search, recommendations, cart, payments and many other components need to be tied together to create a harmonious customer experience.
If any of these components fall short of customer expectations, then online retailers are losing potential revenue. It’s therefore paramount that they measure the performance of each of these components, and understand how they all scale up to meet the overall goal: revenue.
In today’s digital-first, data-driven age, it’s relatively easy to measure the performance of each of these components. Collecting, processing and surfacing data to those who need it costs less and less each year. As data moves to the cloud and new analytics software becomes available, businesses can start getting access to meaningful data in days, rather than months, and for a negligible cost.
Advertisement
However, we still see organizations struggle to turn this data into a measurement framework that drives the right actions across the e-Commerce site.
What you measure drives your decision making. If you measure the wrong thing, you will draw the wrong conclusions and make the wrong decisions. Choosing the right metrics will help your product and development teams to focus on building the functionality that will achieve the right outcomes, and pull together in the same direction.
When it comes to retailers determining how they should measure search, there are several bad practices and red herrings that they fall for. Here are a few that retailers should be aware of, and how successful retailers measure their search performance.
Beware Of The Site Conversion Rate
The overarching goal of most e-Commerce sites is to generate revenue. Generating revenue can be broken down into three distinct components:
- Traffic: How many people are visiting your site?
- Conversion: How many people are buying from your site?
- Basket size: How much are people buying from your site?
The first goal of an online retailer is to measure these three KPIs and map all their current initiatives against how they expect them to impact these three KPIs.When it comes to search, we often see that retailers tie the performance of search directly to the session conversion rate on the site. Sometimes, we see that sessions are split into those with a search and those without.
It’s very important to measure the session conversion rate; however, it’s also important not to draw conclusions from it alone.
An e-Commerce journey is a complex one. There are thousands of different journeys that shoppers will take towards their conversion, with many of them dropping out along the way There are a huge number of factors that will contribute to whether a shopper converts or not. They could be struggling to find the product they are looking for using the search box, they may take issue with the price, they may not accept a type of payment or there could be dozens of other reasons why that shopper did not convert on the site.
The graphic below is a simplified version of the possible journeys that shoppers will take.
When it comes to search, successful retailers focus on search conversion metrics. The primary measurement should be the search click-through rate. This is the percentage of searches that lead to a product view and is the purest indication as to whether shoppers are viewing the products in the search results page.
The search add-to-cart rate and search order rate also are important. Similar to the click-through rate, these are the percentage of searches that lead to a product add-to-cart or an order. However, these metrics can be distorted by factors outside of the control of search, such as price, availability or issues with the cart or payments.
Beware Of The Average Click Position
A popular metric to track is the average click position. This can be calculated as a median or mean, but the principle is that it shows how far down the search results shoppers are having to go before they click.
Whilst the average click position is a useful metric for measuring the ranking of the search results, it can be misleading if it is used on its own.
Shoppers use the search box for a variety of purposes. Some searches are specific, where the shopper has a particular product in mind and they use the search box for the shortest path to that product. For instance, a search for ‘iPhone XR 128GB.’ Here, a low average click position indicates that search is successful, as the most relevant product is at the top of the search results.
Other searches are more exploratory. Shoppers use the search box in a similar way that they would navigate to a category page and take a look at the products that are available. For instance, a search for ‘shirts on sale’ or ‘Valentine’s Day.’ In this instance, a higher average click is more favorable, as retailers want to encourage shoppers to scroll through the search results and look at as many products as possible.
A more thorough way of measuring the ranking of products is the Normalized Discounted Cumulative Gain metric. It’s a mouthful to say, and it’s not that much easier to explain. This article from eBay does a great job of explaining what it is and how they use it to measure their search performance.
Essentially, it measures whether the products are ranked in descending order of clicks or orders for that search query. Each search query has an NDCG score from 0-100, with a score of 100 implying that the results are perfectly ranked, and you can take an average across your site to get a measure of how well your search results are ranked.
Be Smart About How You Measure Revenue
Revenue is the ultimate goal for an e-Commerce platform, and all KPIs and metrics should be traceable back to the goal of increasing revenue. However, there are ways that revenue KPIs can be misused when in the context of search.
Some retailers will measure the revenue generated from search, others what percentage of sales originate from search, as opposed to sales through browsing or other methods. This can lead to the wrong conclusions being drawn. There are a number of scenarios that will lead to a shopper using the search box. Sometimes shoppers will use the search box out of frustration with the navigation options.
If the percentage of revenue that is attributed to search is going up or down, it doesn’t necessarily mean that search performance is improving or reducing. It can be a result of a shift in consumer behavior, or an improvement of your site navigation.
A far better way to tie revenue with search is to look at the revenue per search and the percentage of shoppers using search separately. If the revenue per search is stable but the user behavior across the site is changing, then you know that there isn’t an issue with search. However, if there are fluctuations in the revenue per search, then your attention will be more likely needed within the search component.
Focus On The Right KPIs
It’s important to remember that setting KPIs depends on what your company is trying to achieve. KPIs are purely an indication as to whether you are achieving your goals. If you have a different goal, KPIs should be changed to match.
Whatever your goals, it’s important to take the time to make sure that the KPIs and metrics used to track whether you meet the goals are driving the right behavior among your organization. Poorly chosen KPIs can drive counterproductive behavior, and mask success or failure within your e-Commerce ecosystem.
George Hodgson-Abbott is a Product Manager at GroupBy Inc. where he owns the analytics, recommendations and personalization components of their e-Commerce platform. Prior to that, he was a consultant specializing in analytics, advising global brands on how to extract value from their data through better decision making and automation through machine learning.
