Big Data - As Seen on TV!

Hi everyone! Now that the Olympics is over, I’ve been left with nothing to watch on TV.. until now!

Being a fan of stuff like Lost, I thought I’d give ‘Person of Interest’ a go. Shown on CBS in the US; the first episode was aired here in the UK by Channel 5 on Tuesday. It’s got a good pedigree with people like Jonathan Nolan and J. J. Abrams involved and stars Jim Caviezel (the guy who played Jesus in that film about Jesus) and Michael Emerson (Ben from Lost).

Now, I’m not going to review the show itself here.. All I’ll say is that Jim Caviezel wore an obviously fake beard, acted a little robotic and Michael Emerson had a fantastic fake limp.

Anyway, the show is set in modern day America with an ex-CIA agent (Caveizel) being picked up by a strange Billionaire (Emerson) who offers him a job. The pitch went something like this:

“Imagine if you knew that something bad was going to happen to someone. It could be a terrible accident or murder or something else. The question is, if you knew in advance; would you try and save them?”

Now, this is a little similar to Minority Report I know but without the people in the paddling pool and the cool X-Box Kinect stuff. Person of Interest has its very own McGuffin - “The Machine”.

The Machine comes from a job that the Billionaire was asked to do by the US Government after 9:11. 

“Build us something that could predict another terror attack” is probably what they asked for. 

He went away and did that but found that his ‘bad thing prediction machine’ also told him about individuals who could be in danger.

It seems that the US Government weren’t too bothered about these singular incidents so set up a protocol to delete these individual events every night. Rather cold! However, our Billionaire friend built a back door into ‘The Machine’ so he could pull these details out before deletion each night. The problem is, all he could get was the Social Security Number.

Building the Machine, Finding the People

Ok, Let’s review where we are.

1. Someone has built a tool that can predict the future. How does it do this?
2. From this tool, he can get hold of a unique identifier. But nothing else. So how does he find the person to save them?

Let’s tackle number 1 first: What the billionaire has built is a way to pull data from multiple sources (CCTV, audio recordings, web traffic, mobile phone data, addresses, names, social security ID etc), mash it together and then use it to predict likely events. Now, don’t all shout at once: I agree! He’s built a Big Data platform! We’ve got a lot of data (Volume) in a lot of formats like text, audio, video, pictures and so on (Variety) and The Machine is able to predict a few days in advance each day (Velocity). I wonder if he’s using Hadoop?

The second point is also interesting. He only gets one piece of data from his ‘back door’ into The Machine. In order to find out the name and address of the individual I’m pretty sure he’ll be trying to get hold of other things like an address file, electoral roll, demographics and so on to match the number to an individual so Jim Caviezel can go and save them. Does this ‘data enhancement’ process sound familiar to anyone?

The Play for Product Managers

Now, you may not all be Big Data geeks like me and may not give a monkeys about the guy who played Jesus, the Billionaire and his Machine. Nor may you care about crime procedural TV shows with a twist. However, what I think we should all realise is that if Hollywood is paying attention to Big Data and the clever people who wrote Lost and The Dark Knight think it’s interesting – maybe your customers feel the same?

So, how could your business create a Machine to give data, analysis, predictions and decisions to customers? I work for a business that does a lot of this already (hence why I’m so interested) but I do think that we’re now starting to see ‘Big Data’ go mainstream. The general public get the premise.. Will they go to work and want to do something about it?

Probably.

So go, find some tools, find some data; predict the future!