Big Data Success: Delivering Data is Not Enough

Many years ago I was pitching a Big Data solution to the Production Manager of a large manufacturing plant. After describing all the data we could collect, and the metrics we could turn it into, I thought I had done pretty well. What Production Manager couldn’t be impressed and want our system to get his finger on the pulse of his operation?

Instead, his next question floored me: “If I don’t do anything with the data your system collects, then it doesn’t create any value for me, does it?”
I had never imagined that someone presented with real-time, detailed information wouldn’t immediately grab it and use it to improve their business. I was so taken aback I could not think of an intelligent response, and needless to say, we didn’t win that deal!

Now I’m not going to suggest that this person was absolutely right, but there is a gem of wisdom in what he said for the Big Data community:

Merely delivering data does not deliver value.

Even lots of accurate data, even in real time. There are many Big Data systems that think that this is where their responsibility ends: systems where data is collected and put in a data repository or historian; systems where data is collected an put on on-line graphs.

The Information Value Chain is only just starting when you collect the data. Turning that data into information and ultimately into ACTION is even harder, and if anything your Big Data system has made it 10 times worse: understanding a small amount of data to turn it into information and then persuade people to take action is extremely taxing, and takes many different skills. Doing that with a torrent of data is overwhelming.

What to do? It is our responsibility — as Big Data professionals — to understand the End User’s action goals, the information required to trigger that action and then to set up a Big Data infrastructure that gets the End User as far along that Information Value Chain as possible.

Then we are creating value!