How much does it cost to analyse supermarket data?
Most supermarket suppliers want to analyse the data that they collect from their customers’ data portals. The data available varies significantly between the different supermarket retailers but most provide some form of sales data (value and quantity) which provides a partial, in some cases very detailed, picture of shopper demand for each supplier’s products.
In its simplest form, analysing supermarket data is entirely free! Any supplier to the ‘Big Four’ UK supermarkets, for example, can perform useful sales analysis with nothing more than an account on each of the ‘retail portals’ and a suitable spreadsheet tool (Microsoft Excel or Google Sheets, for example).
Most organisations, however, will want to understand their recent performance in the context of long-term patterns and this presents a few challenges:
- Retailers tend to provide detailed data only for recent events; usually the last few weeks
- Some data (e.g. stock levels) is only presented as a daily snapshot
- Downloading all available data every day takes a long time and is impractical
Downloading and integrating large datasets
Accepted practice is to download as large a dataset as possible, on a very infrequent basis (once-off, or annually) and then add download a few days of data at a time. This is an effective approach but requires the analyst to store all of the retrieved data somewhere and stitch it together in such a way that all the individual pieces form a coherent whole. This can be achieved in a spreadsheet tool – if tackled with considerable care, skill and attention to detail – but becomes increasingly challenging as data volumes increase.
Options for storing trading data: cost, skills, and technology
If you want to build a detailed, long-term picture of your trading performance then it’s highly likely that you will want to store the data that you collect in a database. There are several approaches to this, each with different skill requirements and cost implications:
- Implement an open-source database on your own computer(s), build your own database design and develop the necessary data management code
- Use a cloud data platform and its tools to design, develop and manage a solution that addresses your needs
- Implement a fully-managed retail reporting/analytics SaaS solution
Option 1 has the lowest direct cost – if you have all of the necessary skills within your organisation – but the total cost of ownership could be high. You will need to factor in all of the maintenance costs for your solution, including information security; hardware maintenance; data backup; software upgrades; code changes to reflect changing retailer data etc.
Option 2 will cost tens to hundreds of pounds a month – for use of the cloud data platform – but this will simplify some of the total cost of ownership; reducing information security risks, eliminating hardware maintenance and simplifying data backup, for example. However, you will still need to design, develop, implement and maintain your data processing solution.
Option 3 will cost hundreds, or possibly thousands, of pounds per month – depending on the sophistication of the solution you choose, the volume of data under management, the number of users you have etc. Many of the SaaS products in this category provide reporting and analysis tools alongside their data management capabilities; you are usually moving beyond simple spreadsheet analysis at this point and are using interactive visual dashboards and, perhaps, some degree of prescriptive or predictive analytics.
