Big Data & Machine Learning Scenarios for Retail

In the retail industry, customer-centricity & personalized experiences are a very high priority. Retailers are realizing that the data that they own combined with the abundance of public & purchased data can give them a competitive advantage that was not easily possible in the past. Competing with data has become feasible and real due to the convergence of the democratization of data and the availability of tools and technologies for handling the size and variety of data.

Microsoft provides Azure Machine Learning which is a highly capable machine learning toolset providing a comprehensive capability for Retail data scientists to quickly build out Advanced Analytics and Predictive Models. It includes pre-defined models for key retail activities enabling personalized customer experiences through recommendations and market basket analysis; and enhancing the ability to forecast inventory & demand at a hyperlocal level using a combination of analytic models and diverse data sets. The Azure Marketplace also enables retailers to acquire & re-use pre-built models rather than build from scratch. By using predictive analytics to forecast which products customers might want next, retailers can create online and offline retail experiences that are personal and relevant .

Big Data has become a hot topic in the Retail Industry as it addresses one of the most critical issues challenging retail today. The world of data is changing and retailers are challenged by the increasing scale, complexity and velocity of data. The past few decades have seen exponential growth in computing and storage: driven by Moore's Law, computing power has increased dramatically, making the modern laptop more powerful than a supercomputer from 1980s. At the same time the amount of data stored has grown dramatically, thanks to rapidly declining hardware cost and the emergence of new data sources such as RFID, the web and social media.

These are just some of the scenarios/use cases that we see frequently in the Retail Sector:


Recommendation engines are critical for Retail since they enable personalization online as well as in-store (through assisted selling solutions). Recognizing that they had an exceptionally rich vein of customer data, JJ Food Service, saw an opportunity to use this data to further boost customer satisfaction. An area where they felt they could save their customers’ some time was by anticipating customer orders, i.e. recommending products to them even before they had entered anything into the system. Here is a link to the story.

A/B Testing

In marketing and business intelligence, A/B testing is jargon for a randomized experiment with two variants, A and B, which are the control and treatment in the controlled experiment. It is a form of statistical hypothesis testing with two variants leading to the technical term, Two-sample hypothesis testing, used in the field of statistics. Retailers could run A/B Testing to determine the best Store or Shelf Layout. Consumer Goods companies could run A/B Testing to determine product packaging or shelf layout. Solutions like the one deployed by Shopperception can be easily used for A/B Analysis to determine the most optimal shelf layouts, store layouts and product packaging.

Predictive Maintenance

Predictive maintenance techniques are designed to help determine the condition of in-service equipment in order to predict when maintenance should be performed. This approach promises cost savings over routine or time-based preventive maintenance, because tasks are performed only when warranted. From a Retailer perspective, this could mean collecting data from in-store equipment like Refrigeration Units in Groceries and QSR, Coolers for beverages, Coffee Machines and so on. ThyssenKrupp teamed up with CGI to develop a solution that securely connects ThyssenKrupp’s thousands of sensors and systems in its elevators that monitor everything from motor temperature to shaft alignment, cab speed and door functioning, to the cloud with Microsoft Azure IoT services. The solution provides technicians with instant diagnostic capabilities and rich, real-time data visualization.

Retail Fraud

Retail Fraud often occurs in patterns, consisting of many small transactions each just below a certain threshold. Fraud patterns are always changing, big data and anomaly detection can help improve fraud detection rates and reduce false positives. Sysrepublic has an interesting solution called Secure Store and is an advanced exception reporting and POS data mining solution used by retailers to identify, prevent, and investigate: fraud, theft, and operational/systemic breakdowns that can lead to shrink. Solutions like Secure Store are able to rank fraud cases and highlight the ones to investigate or prevent.

Price Optimization

Dynamic pricing is a pricing strategy in which businesses set highly flexible prices for products or services based on current market demands. Business are able to stay competitive by changing prices based on algorithms that take into account competitor pricing, supply and demand, and other external factors. KSS Retail provides regular and promotional price optimization, market basket analytics and other services to leading retailers including 7-Eleven, O’Reilly Auto Parts, Raley’s, McKesson, Sonae, and many others.

Inventory Forecasting

Inventory management concerns the fine lines between replenishment lead time, carrying costs of inventory, asset management, inventory forecasting, inventory valuation, inventory visibility, future inventory price forecasting, physical inventory, available physical space for inventory, quality management, replenishment, returns and defective goods, assortment planning and demand forecasting. Balancing these competing requirements leads to optimal inventory levels, which is an on-going process as the business needs shift and react to the wider environment. Retailer Pier 1 Imports wanted to better connect with its customers using insights and data. To do that, the company took to the cloud to pilot a predictive analytics solution based on Microsoft Azure Machine Learning and Microsoft Power BI. As a result, Pier 1 Imports may use data insights to predict which products customers will want in the future, create a dynamic website using predictive modelling and create more efficient and effective marketing campaigns. Here is a LINK to the story.

Location Analytics

By combining demographic data like Median Income, Education Levels, Median Age and customer purchasing data such as preferences, past purchases, and online behavioral data, retailers gain a more in-depth understanding of customer needs and wants than with just past purchase data. While planning new store locations, combining this data for potential locations can give retailers insights into the best locations for new stores. Valuable insights can be gained knowing proximity to competitors, how proximity to hospitals correlates to flower sales, and how nearby sports clubs can drive athletic apparel sales. Esri has some interesting implementations in Retail around Location Analytics. Here is some information about Retail Location Analytics from Esri.