Big data vs. small data?
The whole telecom world – at least the vendor community – is talking about the paradigm shift towards Big Data. This is very peculiar since telcos have been handling huge volumes of data for many decades now. So – really – there is nothing new under the sun. There is just a whole industry trying to sell something to telcos, that telcos already are perfectly capable of handling.
The IT departments at telcos are usually super-skilled at taking large volumes of data from a wide variety of sources and store that in a data warehouse. And this is exactly what the Big Data companies are trying to sell. No wonder that many telco CIOs don’t know if/when to implement a big data strategy – it is already implemented (Gartner is talking about the Big Data confusion here).
What might be lacking is a strategy on how to make the most out of the collected data. Which data is really interesting and business critical and which data will never be used.
We would like to introduce the concept of Small Data. The Big Data approach is usually bottom up, i.e. collect as much data as possible and then see if we can find something interesting and then maybe base your business processes on these findings. It sounds rather ineffective. The Small Data concept starts with the business in focus. Isolate a process that can benefit from having better tools. Then design the tool that really helps the business. Then build a world class data layer that supports the tool you have designed. The data layer shall be optimized for one specific purpose. This will give you a very small and extremely fast data layer – that gives direct benefits for your business.
In short: Big Data – collect as much data as possible then see if you can use this data to make your business more effective. Small data – make your business more effective by only storing the data you know is needed.
What would you choose?