Welcome back to our exploration of the retail promotion landscape. In our first act, we painted the broad strokes of what makes promotions tick and why measuring their effectiveness is not just beneficial but essential for retail success. We delve deeper, moving from the 'why' to the 'how', in our sequel: ‘Decoding the Metrics: A Deep Dive into Calculating Promotion Effectiveness.’
In this installment, we roll up our sleeves and focus on the nuts and bolts of promotion analysis. We will dissect the vital components – the baseline, uplift, subsidization, cannibalization, pull forward, and halo effect – and reveal how each can be meticulously calculated to form a cohesive strategy.
Uplift Quantization Through Data-Driven Analysis
Data Collection (Solution Pipeline): The journey begins with gathering many data points. This includes transaction data, promotional information, and holiday and weather data — all pivotal in understanding sales patterns. This data is aggregated to form a comprehensive training dataset.
Data Segmentation (Product Hierarchy): The collected data is then organized into a hierarchy that classifies products by category, subcategory, and individual product levels. This hierarchical segmentation allows for a nuanced analysis of promotions at different granularities.
Time Series Resolution (Process Resolution): Data is analyzed over varying time resolutions—short-term daily, medium-term weekly, and long-term monthly—offering a multi-dimensional view of sales patterns and enabling precise measurement of promotion effects over time.
Data Preparation (Data Range and Preprocessing): A data range spanning 2-3 years is considered to ensure a robust analysis, accounting for seasonal trends and annual variations. Preprocessing steps like normalization, aggregation, stationarity checks, and outlier detection are applied to clean and ready the data for modeling.
Feature Engineering: This step involves creating new data variables (features) that could influence promotion outcomes, like lagged sales growth or weather impacts, enhancing the models' predictive capabilities.
- Machine Learning Models (ML Models)
Various machine learning models are employed. Endogenous models, like ARIMA and Croston, focus on patterns within the sales data itself. Exogenous models, including LSTM, XGBoost, and specialized tools like Google TIDE, incorporate external factors affecting sales.
- Modeling and Forecasting: With the data prepped and features engineered, two types of forecasts are generated:
- Baseline Forecast: This predicts sales without any promotional activity, serving as a control scenario.
- Promotion Forecast: This predicts sales during promotional periods, enabling the measurement of the actual lift due to the promotion.
Quantization of Uplift and Subsidization
- Promotional Lift Quantization: The difference between the promotion and baseline forecasts reveals the promotional lift — the direct impact of promotional activities on sales.
- Subsidization: The analysis identifies any subsidization effect, where promotions might reduce profit margins or cannibalize non-promotional sales.
- Gross Lift Quantization: Finally, the overall effectiveness of the promotions is encapsulated in the gross lift quantization, which combines the incremental sales due to the promotion with the broader market and inventory implications.
From Measuring Uplift to Understanding Cannibalization and Halo Effects
Having delved into the nuances of uplift quantization, where we unearth the additional sales generated by promotions, we now pivot to a critical aspect of retail strategy – cannibalization.
- Identifying Potential Cannibals and Victims: Correlation-Based Identification
We focus on finding substitutes and propose a straightforward measure based on the correlation of common purchases.
The total number of baskets is denoted as n, the number of baskets containing product i as n(i) , i = 1, . . . . .m.
Number of baskets containing both products i and j as n(ij) I.e., number of times products i and j were purchased with another product k.
High values of ρij are caused by similar purchase patterns of products i and j with respect to the other products k; more specifically, by similar sets of complements of products i and j.
Let's consider two products, A and B. Coefficient values close to +1, it implies that when product A is bought more frequently with a set of other products, product B also tends to be bought more frequently with the same set of other products.
- Market Basket Analysis by Using Apriori Algorithm
Defining Terms for Cannibalization: In the context of cannibalization, let X and Y be two different products where the promotion of product X might lead to a decrease in the sales of product Y.
Calculating Support: Support(X, Y): This measures how often products X and Y are purchased together. A high support value indicates that the two products are frequently bought together, which may imply a complementary relationship rather than a cannibalistic one.
Calculating Confidence: Confidence(X, Y): This shows the likelihood that product Y is purchased when product X is bought. If confidence is high, it might suggest that Y is often an add-on to X, rather than being cannibalized by X. However, if promotions on X lead to a significantly lower sale of Y, it may indicate cannibalization.
Calculating Lift: This is the key metric to identify cannibalization. If the lift is significantly greater than 1, it suggests that products X and Y are purchased together more often than would be expected if they were statistically independent. If the lift is less than 1, it could imply that the products are substitutes for each other; when one's sales increase, the other's decrease, indicating potential cannibalization.
Identifying Cannibals: Substitutes vs. Cannibals: If you find that the lift between two products is less than 1 consistently, especially during promotions, it suggests that these items are potential substitutes and that the promotion of one may be cannibalizing the sales of the other.
- Quantifying Cannibalization and Visualizing Impact
Impact Modeling and Sales Estimation with Cross Price Elasticity:
Cross-price elasticity allows us to quantify the relationship between products, particularly identifying competitors or substitutes. It measures how the demand for one product, let's call it Product Y, is affected by the price change of another product, Product Z. A positive cross-price elasticity indicates that the two products are substitutes, meaning if Product Z's price decreases, sales of Product Y are likely to decrease as consumers switch to the less expensive option.
The cross-price elasticity of demand can be integrated into a multilinear regression framework, where the coefficient (slope) derived from the regression analysis, when multiplied by the price mean of Product Z and divided by the quantity mean of Product Y, gives us the cross-price elasticity. For instance, if the calculated cross-price elasticity is 13.56, it indicates that a 10% price reduction in Product Z could lead to approximately a 130.56% decrease in the quantity sold of Product Y.
- Impact Modeling and Sales Estimation with BSTS
Implementing a Bayesian Structural Time Series Model predicts the counterfactual scenario—what sales would have been without the promotion. Comparing actual sales with this counterfactual quantifies the net effect and the extent of cannibalization.
Cannibalization Graph: A network graph is created to visually map the relationships between potential cannibals and victims, illustrating the magnitude and direction of the impact.
Illuminating the Halo Effect in Promotion Analysis
- Identifying Potential complementary products
As we transition from the complexities of cannibalization to the opportunities presented by the halo effect, it becomes vital to identify not just any related products but specifically those that are complementary. The halo effect thrives on the positive relationships between products, where promoting one item can lift the sales of another. We can employ a statistical approach known as the Yule phi coefficient to identify these relationships systematically.
The Yule Phi Coefficient Approach
The Yule phi coefficient is calculated using a contingency table that counts the number of transactions where both products are sold together, where each is sold separately, and where neither is sold. The phi coefficient is a number between -1 and 1, where a value closer to 1 indicates a strong positive association and, thus, complementary products.
- Quantifying the Impact of the Halo Effect on Complementary Products
While our analysis of cannibalization scrutinized the competitive dynamics between products, the halo effect invites us to explore the cooperative side. To quantify the impact of the halo effect, we apply analogous methodologies used in cannibalization analysis, this time seeking out the uplift in complementary products.
Navigating the Pull-Forward Effect in Promotion Strategy
To understand promotion effectiveness, there's an often-overlooked but significant phenomenon known as the "pull-forward" effect. This effect can substantially impact sales patterns for individual items and whole categories over time. It's particularly prevalent among Hi-Lo retailers who frequently change prices to entice consumers.
- Understanding Pull-Forward
Pull-forward occurs when a promotion prompts consumers to purchase larger quantities of a long-shelf-life product than they typically would. Think of detergent on sale; shoppers might buy several bottles, causing subsequent purchases to lapse until they've used up their stock. This behavior disrupts their usual buying frequency and can lead to a temporary dip in sales following the promotion.
- Calculating Pull-Forward
To quantify this effect, a retrospective analysis of promotional data is key. Here's how we can approach it:
- Historical Data Analysis: Examine sales data for the weeks following a promotion. The goal is to observe any deviations from the expected purchase cycle of the product.
- Baseline Comparison: Compare the post-promotion sales to the established baseline. A decrease in sales suggests a pull-forward effect where future sales have been "borrowed".
- Time Frame Consideration: For products bought every three weeks with a four-week shelf life, analyze at least one month of post-promotion data to capture the full effect.
Category Impact Assessment: Evaluate the broader category performance. If a promoted item sees a post-promotion slump, it's essential to ensure this isn't offset by a lift in a different, non-promoted item within the same category.
- Crafting a Cohesive Strategy
By integrating these elements, retailers can craft promotions that not only spike sales in the short term but also build sustainable growth. The equation serves as a strategic compass, guiding businesses to consider the full breadth of a promotion's influence.
With all factors laid bare, we arrive at an equation that seeks to balance the scales, providing a holistic measure of promotion effectiveness:
Overall Promotion Effectiveness=(Promoted Product Uplift−Cannibalization Loss+Halo Effect Gain)−Pull-Forward Impact
Conclusion: The Symphony of Strategy
Promotion effectiveness is not a single note played in isolation but a symphony of interrelated movements. As we conclude our series, we reflect on the harmony of uplift, the cautionary tales of cannibalization, the serendipity of the halo effect, and the tempo set by the pull-forward effect.
AUTHOR - FOLLOW
Vyshagh A
Senior Data Scientist, Tredence Inc.
Topic Tags
