Big Data Can’t Measure Instinct



Below is a blog piece I wrote for Junior Strategy, a great project which I’ve recently begun to contribute to, which aims to be a resource for any current or aspiring junior planner or strategist. Started by Ben and Ashly who are both currently based in Amsterdam, over the last year Junior Strategy has grown to include contributors from Europe and the US, with more hubs springing up all the time. 

Up until now the format of the site has been in-depth interviews with planners or strategists at a senior or director level, probing them for pearls of wisdom gained along their journey that could be of use to anyone just starting out. But this week sees the launch of the newest ‘arm’ of Junior Strategy – a blog aimed to provide some food  for thought around issues in planning or strategy today. My first outing as a regular written contributor was to the ‘Wildcard’ section, talking about the impact of big data on consumer insight for brands, and the perils of taking big numbers at face value.

Millie Findlay

‘Big data’ is increasingly cited by commentators as the future of insight for brands and agencies alike, with 2013 already christened as ‘the year of big data’. In a study conducted by Winterberry Group which surveyed over 150 senior figures in advertising, marketing, publishing and technology, 77 percent said data management platforms will play either a “critical” or “major supporting” role in analysing and improving their advertising and marketing efforts in the long term.

This isn’t just about marketing either – brands are using consumer data to create and improve their services and products – the Nike + and Fuelband ecosystem being just one example. Netflix has even taken big data to the next level – using in-depth knowledge of their consumer’s viewing habits in their new venture as content producer with their new series House of Cards.

Despite the hype, this isn’t a new phenomenon: the use of algorithm and analysis to make sense of the huge streams of information left in our digital wake has been on the cards for some time. In 2008, Wired’s Chris Anderson wrote:

“This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear. Out with every theory of human behavior, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”

But this might be just the problem – the numbers can’t always speak for themselves. Or rather, they can say a number of things, depending on who’s reading them. What’s missing is a more human intuition about the consumer – vital to making sense of data sets, giving them a context beyond the numbers. And increasingly, companies are struggling to use the data to make the decisions that matter.

In his final all-staff email after being fired, Groupon CEO Andrew Mason wrote:

“If there’s one piece of wisdom that this simple pilgrim would like to impart upon you: have the courage to start with the customer. My biggest regrets are the moments that I let a lack of data override my intuition on what’s best for our customers.”

Reliance on data (whether you have too much or too little) risks being at the detriment of a real understanding of the customer and their needs and motivations. For strategists and planners both client side and in agencies, this intuition about the consumer comes from being immersed in their world – in all its contradictions and immeasurability. Multinational brands like P&G are beginning to tackle this by sending their product teams in-field for days or even weeks to better understand their consumers.

In an article for the Boston Globe, Samuel Arbesman refers to the ‘bias towards measurable information’:

“Throughout history, in one field after another, science has made huge progress in precisely the areas where we can measure things—and lagged where we can’t… The result, over time, has been that we know a lot about the things that are closer to our size, our altitude, our spot in the universe—and less about things that are hard to reach, hard to dig up, and hard to quantify. What we know has a bias, in other words, and is biased in favor of what we can measure.”

So while the never-ending streams of big data coming out of social media, online transactions and smartphone use might seem to reflect a huge amount of information about the consumer’s life, it doesn’t reflect everything. It might be possible to track how someone shops, or how they interact with their peers on Facebook, but it would be a mistake to imagine that this represents how they are all the time, in every situation. We also need to be aware of assuming that this data even represents all people – 76% of people globally still don’t use smartphones, and while it may seem that Facebook is everywhere, only 14% of the world’s population are active users. As Arbesman says, “big data might be deep… but it’s not wide”.

It’s here that human intuition about the consumer comes into play – data tells us that something happened, but not the complex emotions and drivers behind it, the ‘why’. A balance needs to be struck between using big data to see behaviour on a grand scale, while acknowledging its limitations. Offsetting huge data sets with a more intuitive understanding of the small-scale drivers and emotions behind behaviours is vital to avoid the perils of believing in numbers over people:

“Data-driven predictions can succeed — and they can fail. It is when we deny our role in the process that the odds of failure rise. Before we demand more of our data, we need to demand more of ourselves.” Nate Silver: The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t.

comments powered by Disqus