Big Data: Past, Present and Future

Big data. 

It’s a phrase you’ve almost certainly heard of, but few understand it’s real potential. More specifically, few understand its real potential to do harm. 

Big data is the “macro-economic” view. It’s the collective interpretation of all the individual items/records into a summary/distilled soundbite. As a basic hypothetical example, 80% of people who eat an apple a day have better than average health. 

On the face of it, big data has huge potential. If the statistic above is true, then all we need to do is eat more apples and we all have better health. But there lies its simultaneous weakness. If absolutely everyone ate an apple a day, the average health of the population overall increases; and therefore the marginal benefit of eating an apple a day disappears. 

Most disturbingly of all, the inverse position now becomes true: now, if you DON’T eat an apple a day, there’s an 80% chance that you will have below average health. 

So when using big data for decision making, trend analysis, or any other purpose, you must remember the three golden rules:

  1. Past: your data is collected from things that have happened already. It is not perfect. It is not future proof. It is already old. 
  2. Present: make the right decisions and choices based on what you know and can deduce today. 
  3. Future: embrace change, encourage new thinking, accept you don’t know what will be true. Innovate. Then do it again. 

At Future Solutions, when we build advanced automated processing routines for our clients, we always seek to understand before being understood: we want to learn the past. That helps us understand the present: what are the pain points, the frustrations, the peaks and the troughs. Then we provoke dreams of the future: if X were no object, what would be true?

We know however that as soon as we start building the dream future state – in the present – it will soon become the past. As soon as our clients see what their system can now do for them in a fraction of the time it took previously, they start to dream even bigger and what could be possible given their new present state. 

And that’s why we do what we do. 

And we love it. 

Big (data) time.