Deciding Where to Shop
Can Newton’s idea of gravity be used to estimate grocery store revenue?
With many different brands, formats and channels to engage with, deciding where, when and how to do your grocery shop can be a complex decision. Some of us will do a regular weekly shop at a large superstore, some will do smaller top-up shops throughout the week, and others will use their mobile to order groceries to their home at a designated time. Can the idea of gravity be used to account for all these different behaviours?
In this article I will summarise some of the findings from my recent Advanced Spatial Analysis PhD Thesis conducted at the Centre for Advanced Spatial Analysis UCL, and in partnership with dunnhumby. I will discuss where the idea of gravity modelling in retailing comes from and how the model is used to estimate grocery store sales in the UK. I will cover the results from my thesis and how they suggest that gravity models are no longer as relevant as they once were. I will then identify how shopping habits have changed in recent years and how they have affected the ability of the model to estimate revenue reliably.
Gravity Modelling
Gravity is a concept most of us will have been taught during our high school Physics classes. You are likely to recall the story of Isaac Newton sitting under a tree when an apple fell onto his head, leading him to wonder why the apple fell straight down as opposed to sideways or even upwards. This line of questioning eventually led to Newton identifying gravity as a force that holds us on the ground and keeps the moon and planets in their orbits. The famous equation holds that the attractive force between two plants is equal to G (the gravitational constant) multiplied by the product of the masses (m1 and m2) and divided by the square of the distance between them (R²):
This idea was then linked to the idea of movement of individuals by Ravenstein in the late 1800s and Reilly was the first to link gravity to retailing in 1929. Reilly stated that:
“two cities draw trade from a smaller intermediate city or town approximately in direct proportion of the first power of the population of these two larger cities and in an inverse proportion to the square of the distance of each of the large cities from the smaller intermediate city” (Reilly, 1929, p. 16)
Thus drawing parallels with the concept of gravity and formulated an equation:
Which could be used to determine trade boundaries between different cities. This would be the point at which an individual would be indifferent from visiting city A or city B for shopping. The idea was that this formula could be used to determine how much retail sales would come from an intermediate town to either of two competing destinations. It could then be used by either retailers or city planners to determine where to locate retail outlets.
Modern applications
The idea and formulation of gravity modelling in retail has been adapted and improved since the original idea by Reilly in 1929, but the foundation of the concept has remained fundamentally unchanged. Namely that the flow of goods between two locations is proportional to a measure of the size of the origin and destination and inversely proportional to the distance between them. In grocery retailing, this means the amount of money available to spend from each origin, the attractiveness of the store and the travel time between them.
Most modern implementations of the gravity model in retailing now take the form of the model derived by Wilson in 1969, 1971, known as the origin-constrained or retailing model. This takes advantage of the mathematics of entropy-maximisation from thermodynamics to place the model on more sound mathematical footing and allows for easier calibration. This model takes the form:
Where:
Within this Tij represents the flow of value from origin, i, to destination, j, Oi is the total amount of revenue available to spend in each origin, Wj is a measure of attractiveness of the destination, cij is a measure of distance between and origin and destination and Ai is a balancing factor to constrain the total outflow from each origin. The calibrated model has two main parameters of γ, which measures the strength of attractiveness, and β which measures the distance decay in the model.
To calibrate this model we can use anonymised loyalty card data which shows the total amount spent per origin at each destination within a given timeframe. The calibrated model can then be used to predict total store revenue based on the estimated revenue available from each origin.
Modelling a region
Previous applications of gravity models using loyalty card data suggested that they could be highly accurate. Namely that at a small spatial scale they could be used to predict store revenue to within +/-10% across a whole year. Our research thus began with attempting to scale these models to predict store revenue for a whole region within the UK. That is to estimate the revenue for over 100 supermarket size stores for each week across a whole year.
The idea was that if we could create a model for a whole region in the UK, then we could potentially use this to identify changes in the retailing environment and how best retailers could adapt. This included the opening of new competitors, estimating the revenue of a new store and detecting underperforming stores.
The gravity model was thus trained on a dataset of anonymised loyalty card data from a UK national grocery retailer. This data showed the amount spent per week, per output area (A UK census area with around 150 households) and per store. The calibrated parameters from this model were then used to estimate total store revenue for the whole region, which could then be compared to the actual revenue.
Our results showed however that at this scale the gravity model was unable to account for the variation in store conditions and consumer behaviour to reliably predict actual store revenue. Namely that while a few stores were well predicted on a consistent basis, the majority were poorly predicted such that the model would unlikely to be used in practice to make new store location decisions. This meant that with more recent data were were unable to replicate the results seen in the previous literature.
Replicating previous results
With these results then the next step was to see whether we could replicate the performance seen in previous papers. These papers suggested that a gravity model could be reliably used to estimate grocery store revenue to within +/-10% across a whole year for four stores and that +/-30% for sixteen stores. It thus became a question as to whether we were just modelling too many stores all at once?
To make sure that our attempts at replication were consistent, instead of using a single group of either four or sixteen stores, we replicated the model across 47 different groups of each size. These stores were selected from a single region based on stores that were located next to each other, replicating the conditions from the previous papers.
Our results from this analysis however showed that while there could have been conditions that led to the performance seen in the previous literature, their results could not be consistently replicated. This was such that while a single group of each size showed results within the previous bounds, the other 46 groups did not.
These results supported the argument from the regional model application that at this scale and with the current data and modelling formulation, the gravity model was unable to account for the variance in underlying store characteristics and consumer behaviour at the regional scale.
Alternative models
This left two questions. Was it the form of the model that was affecting performance? Or was it changes in behaviour by consumers that meant that a gravity model was too simple?
An alternative model to try was thus a competing destinations model. This model differs from the original gravity model by adding in an extra factor that could account for two level decision making. This is such that the influence of either competition or agglomeration could potentially be identified. The results showed that there was a trade-off in modelling performance between agglomeration or competitive forces, but ultimately no improvement in performance was seen. Notably, no parameter pairing for the model was seen to result in improved modelling performance relative to the original model and stores reacted differently to agglomeration or competition.
The second modelling adaptation then explored whether instead integrating age as a measure of attractiveness, alongside size, could improve modelling performance. However again, the results showed that the integration of store age did not alter the original model performance. It did however show that the older a store was then the more attractive it would be to consumers. Based on this it was potentially suggested that this is due to the amount of information that consumers have about a store, but could be an interesting avenue for future research!
Finally, we attempted to see whether we could instead identify a subset of the population who still did regular weekly shops at a supermarket by car based on large basket sales. This aimed to reduce the influence of convenience and multi-purpose trip behaviour on modelling performance. Once again however the results did not show improvements in performance but did suggest that one of the key difficulties in creating gravity models is being able to estimate the total amount of revenue available at each origin.
Modern shopping trends
What these results suggested was that there was something missing that the model wasn’t able to capture or that behaviour had fundamentally changed from what it was before. This can clearly be detected in shopping trends that have emerged since the beginning of the 21st Century in the UK such as:
- Convenience Shopping — This trend evolved since the early 2000s and is characterised by more people shopping within a much smaller travel time (often walking 5–10 minutes), shopping with a greater frequency and doing so at “convenience stores” of big retailers. These stores are smaller than 3,000 sqft, locate close to consumers' homes or a large daytime population and have a wide but shallow product range. Retailers responded to this change by developing more of these small-format stores than any other format between 2004–2012.
- Rise of the deep discounters — Continental discount stores began to muscle their way into the UK market from the 1990s but saw some of the largest gains in market share in the 00s and 10s. This was in response to a perceived gap at the bottom of the market where consumers could be enticed by low-cost goods, thereby changing the nature of competition within the market and opening up a new segment. The success of these retailers can clearly be seen today with Aldi and Lidl driving the charge and continuing to open new stores all over the country.
- Online shopping — The adoption of e-commerce in grocery retailing has followed the general trend of broader e-commerce over the last 20 years with increasing utilisation and value relative to the overall market. While this uptake has been slower than the broader retail economy due to the nature of goods sold, it is still likely to have a significant effect on the geography of grocery retailing. Namely, distance is no longer likely to be seen the same way by consumers as goods can be delivered to their homes, the size of the store is unlikely to be the main driver of attractiveness, and e-commerce is another channel through which consumers can engage with a retailer.
These changes in both consumer and retailer behaviour can be seen to influence interactions within the grocery retailing market. This affects the underlying assumption of the gravity model that shopping would be undertaken on a weekly basis, with consumers travelling by car and with a clearly defined regular basket. This may mean that for the gravity model to reach levels of accuracy seen before, it would need to adapt or potentially that newer models such as Random Forests, Neural Networks and Agent Based Models (ABMs) need to be used.
Summary
There has been a long history of ideas from the physical sciences being adapted into social science scenarios. Gravity is one of those concepts and suggests that the flow of people, goods or value is proportional to the size of the origin and destination and inversely proportional to distance. In retailing, this is given as the amount of revenue available to spend, the attractiveness of the store and the travel time between them.
These models have been continuously adapted over their history to integrate advancements in new techniques and ideas. Today we can calibrate and evaluate these models using anonymised loyalty card data which shows where people live, where they shop and how much they spend.
My work however suggests that changes in consumer and retailing behaviour may mean that these models are no longer able to represent behaviour in the grocery retailing market. Namely, we show that at the regional scale and with the current method and data, the model is unable to account for the variances in consumer behaviour and store conditions to consistently and reliably estimate store revenue in the UK.
This could be due to changes in behaviour in the way in which we shop for groceries. This includes the rise in influence of convenience shopping, the increasing influence of deep discounters on competition and new channels of engagement such as e- and m-commerce. These results therefore leave open the question as whether the gravity model can be adapted to model these new circumstances or whether new data sources and methods are needed.
A huge thank you to everyone that contributed to this research and to dunnhumby and the Centre for Advanced Spatial Analysis for supporting this PhD.
