AI-Driven Analytics: Taming The Wild World Of Grocery Data

0aaGary Saarenvirta DaisyIntelligenceWhen it comes to gathering data, few retail categories handle a greater volume than the grocery business. With potentially hundreds of thousands of SKUs and millions of transactions, it’s no surprise that supermarkets have been at the forefront of many important data-driven innovations over the decades, from barcode scanners and panelist data to loyalty programs and even the humble coupon.

But this is also a crowded marketplace with fierce competition, including brick-and-mortar specialty food retailers and the expansion of grocery sections in big box stores. Then, of course, there is the loud march of e-Commerce into the grocery space providing a myriad of pickup and delivery options, while the biggest online behemoth, Amazon, is shaking up the entire space with its recent acquisition of Whole Foods. Grocers are well aware that in order to keep up, they need to leverage their hard-earned data and glean the best insights from consumer behavior and shopping preferences.

The largest retailers, such as Walmart, have gotten a big head start in this regard — for years, they have used predictive analytics to forecast demand and set prices. Midsize and smaller retailers have been relegated to the limits of traditional business intelligence and reporting in an attempt to derive insights. However, these older, legacy business intelligence and predictive technologies are simply not able to handle the vast amounts of data grocers now have — the millions of variables in complex environments are well beyond traditional tools — and are certainly beyond human capacity.


Over the past several years, with the additional computing power of GPUs, it has become possible to analyze all of a retailers’ data using more sophisticated mathematics than was previously available. The emergence of artificial intelligence (AI) is tailor-made for high-frequency, low-margin grocers, where even slight improvements can deliver big increases to the bottom line. The transformation of the grocery business through the use of AI is changing everything from demand forecasting and promotional planning to price modeling and assortment shifts. These are some of the most important best practices to keep in mind as grocers expand their use of AI-driven predictive analytics:

The Future Of Forecasting

With traditional predictive analytic solutions, most grocery assortment forecasting is done at an aggregate level by product by week, or perhaps product by store by week. Each product or SKU is considered independent of all other products, ignoring complex interdependent factors — such as product affinities, cross-price elasticity, customer proximity to stores, customer proximity to competitive stores, and store layout — resulting in forecasting error rates topping 38%.

Advances in AI technology and computer power allows more complex mathematics to be implemented, enabling grocers to include all the more intricate cross-product relationships and affinities into forecasting models. A host of additional variables, including spatial geography, weekly promotional attractiveness or store draw compared to competitors, store layout and planograms are more easily factored into forecasting results. 

The Power Of Price Optimization

Retailers face massive challenges when it comes optimizing prices when trying to maximize profits across tens of thousands of SKUs, multiple pricing zones, rapidly changing costs, increasing competitive activity and ranges of customers across various channels. At the core of price optimization is forecasting. Traditional approaches to price optimization therefore had all the drawbacks previously described in older generation forecasting tools.

Taking into account millions of additional features compared to traditional approaches creates more complex price optimization requirements, which necessitates the use of artificial intelligence solutions powered by GPU computing hardware. Improvements to the math results in more accurate pricing and improved sales without sacrificing additional gross margin.

Pushing Promotions Higher

Grocers still rely heavily on weekly promotions to boost sales. However, the math involved in selecting which items to promote in each marketing medium each week from the potential of available SKUs is mind-boggling (a retailer with 50,000 SKUs selecting 500 items to promote each week in a month, faces 103600 possible choices).

AI systems are able to understand the customer intent by analyzing the patterns apparent in millions of transactions. Some products belong to a use case which require consumers to purchase multiple items — for example to make an Italian dinner, consumers would take advantage of a ground beef promotion and purchase additional products like pasta, tomato sauce, bread etc. to complete the use case. This is commonly referred to as “halo sales” or product affinities. 

In contrast, promoting water which requires no additional purchases to complete the consumer’s use case results in smaller transactions. By not promoting the halo products that are automatically purchased retailers can grow the blended gross margin and average size of transactions. AI systems are able to handle these complex challenges by being able to find all “halo” patterns and automatically simulating the outcome of hundreds of millions of choices,far exceeding human capability, thus selecting better promotional assortments that maximize sales and gross margin. This allows retailers to squeeze every last incremental dollar from each promotional activity.

The Promise Of Big Data: Fulfilled Through AI

Many retailers have struggled to fulfill the promise of Big Data: They have so much data but it has generally remained wild and untamed, unable to deliver the insight grocers truly need to boost profits and impact margin. At the same time, grocers are facing a revolutionary change in the high-frequency retail industry, so, more than ever, they require advanced solutions that will keep them on a level playing field with their large competitors.

For many, it is AI solutions that will make the data-driven differenceby finding the best solutions to highly complex situations that require analyzing massive data sets. As the cost of AI technologies comes down and computing capacity rises, grocers can take full advantage of the latest best practices when it comes to forecasting, price and promotion. Those that maximize this opportunity will be transformed. Those that don’t take advantage of AI-driven decision making will be at an extreme competitive advantage. 


Gary Saarenvirta, the founder and CEO of Daisy Intelligence, is an expert on AI technology and evangelizes about the societal benefits of AI and machine learning as it pertains to improving business decision making. Saarenvirta has a proven record-of-success applying AI-powered technology to automated core business decisions. Previously, he was head of IBM Canada’s analytics and data warehousing practices. Saarenvirta holds both a BASc and MASc in aerospace engineering from the University of Toronto and continues to lecture and advises in curriculum development for the Engineering Science and AI programs at U of T. 

Feature Your Byline

Submit an Executive ViewPoints.


Access The Media Kit


Access Our Editorial Calendar

If you are downloading this on behalf of a client, please provide the company name and website information below: